feat(sandbox): add boxd sandbox provider (#8314)

* feat(sandbox): add boxd sandbox provider

Adds deerflow.community.boxd_sandbox:BoxdSandboxProvider. Each user/thread
scope runs in an isolated boxd machine. New machines get the thread's files
and skills before they are handed out, release mirrors outputs back to
thread storage, and released machines wait in the shared warm pool.

The boxd SDK is an optional extra (deerflow-harness[boxd]). Unit tests run
against a fake client; tests/test_boxd_sandbox_live.py runs against real
machines when DEER_FLOW_RUN_LIVE_TESTS=1 and BOXD_API_KEY are set.

* fix(boxd): retain failed cleanup and bound output mirror calls

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
Michiel Voortman 2026-11-18 14:29:15 +01:00 committed by GitHub
parent 5d69e6e21c
commit e0cccd6b37
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
18 changed files with 3103 additions and 2 deletions

View File

@ -92,6 +92,11 @@ EXA_BASE_URL=https://api.exa.ai
# E2B cloud sandbox API key — required only when using E2BSandboxProvider.
# Sign up at https://e2b.dev/dashboard
# E2B_API_KEY=your-e2b-api-key
# boxd API key, required only when using BoxdSandboxProvider.
# Create one in the boxd console: https://boxd.sh?ref=deer-flow
# BOXD_API_KEY=your-boxd-api-key
# FEISHU_APP_ID=your-feishu-app-id
# FEISHU_APP_SECRET=your-feishu-app-secret

View File

@ -112,6 +112,7 @@ Each guide is the full reference for its topic. Open the one you need.
| DeepSeek web search, thinking mode and citation excerpts | [DeepSeek Web Search](backend/docs/CONFIGURATION.md#deepseek-web-search) |
| Docker, local development, upgrades, production deployment | [Running the Application](docs/guide/deployment.md) |
| Sandbox modes, file system, uploads and outputs | [Sandbox](docs/guide/sandbox.md) |
| boxd machines, bounded output copies and cleanup retries | [boxd sandbox provider](backend/packages/harness/deerflow/community/boxd_sandbox/README.md) |
| Skills, built-in tools, script workflows, Claude Code integration | [Skills & Tools](docs/guide/skills-and-tools.md) |
| MCP servers | [MCP Server](docs/guide/mcp.md) |
| Sub-agents, task delegation and optional SQLite checkpoint history | [Sub-Agents](docs/guide/subagents.md), [Tool restrictions](docs/guide/subagents.md#tool-restrictions) |

View File

@ -1781,6 +1781,67 @@ Downloads are restricted to `/mnt/user-data` and all file paths reject
traversal. Multi-process discovery and ownership coordination are not yet
implemented, so `replicas` is a per-Gateway-process soft cap.
**boxd Machine Sandbox** (runs each sandbox as an isolated [boxd](https://boxd.sh?ref=deer-flow) machine):
```yaml
sandbox:
use: deerflow.community.boxd_sandbox:BoxdSandboxProvider
boxd_api_key: $BOXD_API_KEY # falls back to the BOXD_API_KEY env var
# org: my-org # optional; the key's own org when omitted
# region: fra # optional; fra or nyc
# vcpu: 2 # optional; 1, 2, 4 or 8 (memory 4G, 8G, 16G or 32G follows)
# from_snapshot: my-snapshot # optional; start machines from a prepared snapshot
# auto_suspend_timeout: 300 # optional; idle seconds before a machine goes to standby
# ready_timeout: 120 # seconds to wait for a new machine to boot
# name_prefix: deer-flow- # machine names are <prefix><scope>-<random suffix>
# labels: # extra labels on every machine
# team: research
replicas: 3 # active + warm cap per gateway process
idle_timeout: 600 # warm seconds before the machine is deleted; 0 disables
environment:
PYTHONUNBUFFERED: "1"
```
Install the optional SDK before selecting this provider:
```bash
pip install "deerflow-harness[boxd]"
```
Each `(user_id, thread_id)` scope gets its own Ubuntu machine, created with
`isolated: true`, so it joins no private network and holds no boxd
credentials. The API key stays in the Gateway. Machines carry the labels
`deer-flow: sandbox` and `deer-flow-scope: <scope id>`, so you can find them in
the boxd console or with the CLI. The provider creates `/mnt/user-data` and the
skills root (`skills.container_path`) as real directories on the machine, so
shell commands and file tools use the same paths.
A new machine for a thread receives the thread's uploads, workspace and outputs
files and the skills projection before `acquire()` returns, each as one archive
of at most 5,000 files and 128 MiB; files past that limit are skipped with a
warning, and a failed transfer deletes the machine and raises. An Agent with a
skill policy gets its own skills projection, and the machine sees only what was
uploaded. Work outside `/mnt/user-data` is lost when the idle reaper deletes a
machine. At the end of a turn, release copies
`/mnt/user-data/outputs` back to the thread's output directory (bounded by file
count, bytes and one shared 120-second deadline), then parks the machine in an
in-process warm pool. A stopped machine's output copy is skipped until a normal
sandbox call restarts it. The same scope reclaims it after a health check;
the idle reaper deletes it after
`idle_timeout`. Shutdown deletes every machine the provider tracks and can be
retried when a delete fails.
Machines discovered after a failed create use the same tracked deletion retries.
Commands run through `bash -lc`. Per-call environment variables travel as exec
options, never inside the command string. The command deadline (the caller's
timeout, else `bash_command_timeout`) is enforced in the machine, so a command
that runs over it ends with `Exit Code: 124` and keeps its partial output. A
stopped machine is started on the next call; a command that hit the stopped
machine is not repeated and returns an error. Machines in standby or
hibernation wake on the call. Downloads are restricted to `/mnt/user-data` and
capped while streaming. Like the other remote providers, `replicas` is a
per-Gateway-process soft cap.
Choose between local execution or Docker-based isolation:
**Option 1: Local Sandbox** (default, simpler setup):

View File

@ -0,0 +1,33 @@
# boxd sandbox provider (`deerflow/community/boxd_sandbox/`)
Runs sandboxes as isolated boxd machines. General sandbox rules are in the
[sandbox guide](../../sandbox/AGENTS.md); user docs are in [README.md](README.md).
```bash
cd backend && python -m pytest tests -q -k "boxd"
```
- **Address machines by id, never by name.** Names are unique per owner and
are not free again right after a delete, so every machine generation gets a
random suffix.
- **Every machine is created with `isolated=True`.** The agent runs untrusted
code; an isolated machine holds no boxd credentials and joins no private
network. A test pins it.
- **A stopped machine answers with a conflict.** Start it; repeat file calls
once, but never a command, which may have been running when it stopped.
Standby and hibernated machines wake on the call.
- **Every SDK call has a deadline.** Commands run under coreutils `timeout`,
so exit code 124 and partial output come back. `machines.fs` read streams take
no deadline, so reads run as `head -c` through exec.
- **Release copies share one deadline.** Pass it through listing and downloads.
Budgeted calls skip stopped-machine recovery and failure probes because those
control-plane calls have separate timeouts.
- **Failed-create cleanup stays owned.** A machine found by generation label
goes through `_close_quietly`, so a failed delete is retried by the reaper
and shutdown.
- **`MachineFileNotFoundError` is also a `NotFoundError`.** Test for the
filesystem error, and confirm with `machines.get` before treating a machine
as gone.
- **Claims decide when a cancelled acquire releases.** Each caller that
receives a sandbox holds a claim; a cancelled caller gives its claim back
under the scope serializer, and the last one releases the sandbox.

View File

@ -0,0 +1,101 @@
# boxd sandbox provider
Runs DeerFlow sandboxes on [boxd](https://boxd.sh?ref=deer-flow) machines through
the `boxd` Python SDK. Each user/thread scope gets an isolated Ubuntu microVM.
The machine lives while the thread is active and while it waits in the warm
pool; after `idle_timeout` it is deleted, and a later turn gets a new machine
with the thread's files restored (see below). Anything the agent left outside
`/mnt/user-data` is gone at that point.
## Installation and configuration
Install the optional SDK, then select the provider:
```bash
pip install "deerflow-harness[boxd]"
export BOXD_API_KEY=... # an API key from the boxd console
```
```yaml
sandbox:
use: deerflow.community.boxd_sandbox:BoxdSandboxProvider
# boxd_api_key: $BOXD_API_KEY # falls back to the BOXD_API_KEY env var
# org: my-org # optional; the key's own org when omitted
# region: fra # optional; fra or nyc
# vcpu: 2 # optional; 1, 2, 4 or 8 (memory 4G, 8G, 16G or 32G follows)
# memory: 8G # optional; give either vcpu or memory, or a matching pair
# from_snapshot: my-snapshot # optional; start machines from a prepared snapshot
# auto_suspend_timeout: 300 # optional; idle seconds before a machine goes to standby
# ready_timeout: 120 # seconds to wait for a new machine to boot
# name_prefix: deer-flow- # machine names are <prefix><scope>-<random suffix>
# labels: # extra labels on every machine
# team: research
# replicas: 3 # active + warm machines per gateway process
# idle_timeout: 600 # warm seconds before the machine is deleted; 0 disables
# environment: # merged into every command's environment
# PYTHONUNBUFFERED: "1"
```
Invalid values (an unknown region, a size boxd does not offer, a malformed
environment name) fail at provider startup, before any machine is created.
`from_snapshot` is the way to give every sandbox extra tools: prepare one
machine, save it as a snapshot in the boxd console or with the CLI, and name the
snapshot here. New machines start from it.
## Lifecycle and contract
- **One machine per scope.** The local sandbox id comes from
`derive_sandbox_scope_token(user_id, thread_id)`. Every SDK call addresses the
machine by the id that `create` returned.
- **Isolated.** Machines are created with `isolated: true`: no private network,
no boxd credentials inside the machine. The API key stays in the Gateway, and
per-call secrets travel as exec environment, not in the command string.
- **Bootstrap.** `/mnt/user-data/{workspace,uploads,outputs}` and the skills root
are created as directories owned by the machine user, so the paths the agent
sees are the real paths.
- **Thread files and skills.** A new machine receives the thread's uploads,
workspace and outputs files (in that order) and the skills projection, each
as one archive of at most 5,000 files and 128 MiB. Files past that limit are
skipped with a warning. A failed transfer deletes the machine. An Agent with
a skill policy gets its own projection through `sync_agent_skills`, which
replaces the managed skill categories; the machine sees only what was
uploaded.
- **Release.** Output files are copied back to the thread's output directory
(at most 2,000 files, 512 MiB and 120 seconds per release), then the machine is
parked in the warm pool. Names that differ only by case or Unicode
normalization, unsafe names and linked destinations are skipped. A copy
failure keeps the previous copy. Listing and downloads share the remaining
120-second budget. A stopped machine's copy is skipped, and recovery waits
for the next normal sandbox call.
- **Reuse.** Only the same scope can reclaim a warm machine, after a health
check. A failed check deletes the machine and creates a new one.
- **Stopped machines** are started on the next call. A file call then runs
once more; a command does not, and returns an error that asks to run it
again, so a command never runs twice. Machines in standby or hibernation
wake on the call.
- **Commands** run through `bash -lc` with a deadline: the caller's timeout,
else `bash_command_timeout`. The deadline ends the command inside the machine
with exit code 124 and keeps its partial output.
- **Reads** run as a bounded `head -c` with a deadline, so a large or stalled
file cannot hold a Gateway thread or its memory.
- **Cleanup.** The idle reaper deletes warm machines after `idle_timeout`;
shutdown deletes every tracked machine. A failed delete is kept and retried
by the reaper (when `idle_timeout` is above 0) and by the next shutdown. If
`create` raises after the server made the machine, the provider finds it by a
per-create label and tracks it until deletion succeeds. When every caller
that acquired a machine was cancelled, the machine is released to the warm pool.
- **Leftovers.** Machines carry the labels `deer-flow: sandbox` and
`deer-flow-scope: <scope id>`. A Gateway that crashes cannot delete its
machines; find them by these labels in the boxd console or with the CLI.
## Tests
```bash
cd backend
python -m pytest tests -q -k "boxd"
DEER_FLOW_RUN_LIVE_TESTS=1 BOXD_API_KEY=... PYTHONPATH=. uv run pytest tests/test_boxd_sandbox_live.py -v -s
```
The live test creates one machine, runs the checks against it and deletes it
at the end.

View File

@ -0,0 +1,38 @@
"""boxd machine sandbox provider for DeerFlow.
This package implements DeerFlow's :class:`Sandbox` / :class:`SandboxProvider`
contract on top of `boxd <https://boxd.sh?ref=deer-flow>`_ machines through the
``boxd`` Python SDK. Each sandbox is an isolated Ubuntu microVM.
Configuration example (``config.yaml``)::
sandbox:
use: deerflow.community.boxd_sandbox:BoxdSandboxProvider
# boxd specific options (read via SandboxConfig's ``extra="allow"``):
boxd_api_key: $BOXD_API_KEY # falls back to the BOXD_API_KEY env var
# org: my-org # optional; the key's own org when omitted
# region: fra # optional; fra or nyc
# vcpu: 2 # optional; 1, 2, 4 or 8 (memory follows: 4G, 8G, 16G, 32G)
# from_snapshot: my-snapshot # optional; start machines from a prepared snapshot
# auto_suspend_timeout: 300 # optional; idle seconds before a machine goes to standby
# ready_timeout: 120 # seconds to wait for a new machine to boot
# name_prefix: deer-flow- # machine names: <prefix><scope>-<suffix>
# labels: # extra machine labels
# team: research
replicas: 3 # active + warm machine cap per Gateway process
idle_timeout: 600 # warm seconds before the machine is deleted; 0 disables
environment: # merged into every command's environment
PYTHONUNBUFFERED: "1"
Install the optional SDK before selecting this provider::
pip install "deerflow-harness[boxd]"
"""
from .boxd_sandbox import BoxdSandbox
from .boxd_sandbox_provider import BoxdSandboxProvider
__all__ = [
"BoxdSandbox",
"BoxdSandboxProvider",
]

View File

@ -0,0 +1,521 @@
"""``BoxdSandbox``: DeerFlow :class:`Sandbox` backed by one boxd machine.
A boxd machine is a full Ubuntu microVM with a persistent disk. The provider
bootstraps ``/mnt/user-data`` and the skills root as real directories owned by
the machine user, so virtual paths need no remapping: the agent's shell and the
file APIs see the same paths.
Commands run through ``bash -lc`` with the per-call environment passed as
structured exec options, never spliced into the command string. Writes use the
SDK's streaming ``machines.fs`` API; reads run as a bounded ``head -c`` exec
with a deadline. Search (``list_dir`` / ``glob`` /
``grep``) shells out to ``find`` / ``grep`` and is parsed with the shared
``deerflow.sandbox`` helpers, like ``community/e2b_sandbox``.
The ``boxd`` SDK is never imported here. Errors are classified by the names in
their class hierarchy, so this module imports without the optional dependency.
"""
from __future__ import annotations
import errno
import logging
import math
import re
import shlex
import threading
import time
from typing import TYPE_CHECKING, Any
from deerflow.config.paths import VIRTUAL_PATH_PREFIX
from deerflow.sandbox.download import DEFAULT_DOWNLOAD_LIMIT as _MAX_DOWNLOAD_SIZE
from deerflow.sandbox.download import download_limit
from deerflow.sandbox.read_file_contract import split_file_lines
from deerflow.sandbox.remote_list_dir import parse_remote_list_dir_output, remote_list_dir_command
from deerflow.sandbox.remote_search import parse_remote_search_output, remote_search_command
from deerflow.sandbox.sandbox import Sandbox, _validate_extra_env
from deerflow.sandbox.search import GrepMatch, path_matches, should_ignore_path_under_root, truncate_line
if TYPE_CHECKING:
from collections.abc import Callable
from boxd import Boxd
logger = logging.getLogger(__name__)
# Extra seconds the SDK call waits after the in-machine ``timeout`` fired, so
# the command's own exit code 124 and its partial output come back first.
_TIMEOUT_GRACE_SECONDS = 30.0
# Seconds between SIGTERM and SIGKILL for a command that hit its deadline.
_KILL_AFTER_SECONDS = 5
# Bound for internal helper commands (search, listing, health probes).
_HELPER_TIMEOUT_SECONDS = 120.0
# How long a started machine may take before commands reach it again.
_START_PROBE_SECONDS = 60.0
_START_PROBE_INTERVAL = 1.0
# Upper bound for one read_file call, so a huge file cannot exhaust Gateway memory.
_MAX_READ_FILE_SIZE = _MAX_DOWNLOAD_SIZE
# Deadlines for file reads, which run as a bounded ``head -c`` exec.
_READ_TIMEOUT_SECONDS = 120.0
_DOWNLOAD_TIMEOUT_SECONDS = 300.0
# Command deadline when the caller gives none (SandboxConfig.bash_command_timeout default).
DEFAULT_COMMAND_TIMEOUT = 600.0
def _error_names(error: BaseException) -> set[str]:
return {cls.__name__ for cls in type(error).__mro__}
def is_machine_gone_error(error: BaseException) -> bool:
"""The machine itself no longer exists (not a missing file inside it)."""
names = _error_names(error)
return "NotFoundError" in names and "MachineFilesystemError" not in names
def is_stopped_machine_error(error: BaseException) -> bool:
"""A conflict that a later ``start`` can clear (the machine is stopped)."""
names = _error_names(error)
return "ConflictError" in names and "MachineFilesystemError" not in names
def is_missing_file_error(error: BaseException) -> bool:
return isinstance(error, FileNotFoundError) or "MachineFileNotFoundError" in _error_names(error)
class BoxdSandbox(Sandbox):
"""DeerFlow Sandbox adapter that delegates to a live boxd machine.
Args:
id: DeerFlow-side sandbox id (the provider's cache key).
client: A ``boxd.Boxd`` client. It is shared by every sandbox of one
provider; this adapter never closes it.
machine_id: The boxd machine id. Every call addresses the machine by id.
region: Region the machine lives in, forwarded to every call.
default_env: Static environment merged into every command, overridden
per call by the ``env`` passed to :meth:`execute_command`.
default_command_timeout: Deadline in seconds for commands whose caller
gives none.
on_terminal_failure: Optional callback ``(sandbox_id, reason)`` invoked
when the machine is gone, so the provider can evict this sandbox.
"""
#: Every call is a fresh ``bash -lc`` exec; no shell state survives.
persistent_shell_sessions = False
artifact_root = VIRTUAL_PATH_PREFIX
def __init__(
self,
id: str,
client: Boxd,
machine_id: str,
*,
region: str | None = None,
default_env: dict[str, str] | None = None,
default_command_timeout: float = DEFAULT_COMMAND_TIMEOUT,
on_terminal_failure: Callable[[str, str], None] | None = None,
) -> None:
super().__init__(id)
self._client = client
self._machine_id = machine_id
self._region = region
self._default_env = dict(default_env or {})
self._default_command_timeout = default_command_timeout
self._on_terminal_failure = on_terminal_failure
self._lock = threading.Lock()
# Serializes the append read-modify-write. Distinct from _lock so the
# eviction callback, which reaches back into the provider, never runs
# under it.
self._write_lock = threading.Lock()
self._closed = False
# ── Properties / lifecycle ───────────────────────────────────────────
@property
def machine_id(self) -> str:
return self._machine_id
@property
def is_closed(self) -> bool:
with self._lock:
return self._closed
def close(self) -> None:
"""Delete the machine (idempotent).
The sandbox is marked closed only after the delete was accepted or the
machine is confirmed gone. Any other failure is raised, so the provider
keeps the handle and retries instead of leaving a machine running.
"""
with self._lock:
if self._closed:
return
try:
self._client.machines.delete(self._machine_id, region=self._region)
except Exception as e:
if not is_machine_gone_error(e):
logger.error("Error deleting boxd machine for sandbox %s: %s", self.id, e)
raise
logger.info("boxd machine for sandbox %s was already gone at close", self.id)
self._closed = True
def ping(self) -> bool:
"""Return True when the machine answers a trivial command."""
try:
result = self._run(["bash", "-lc", "echo ok"], timeout=30.0)
except Exception as e:
logger.warning("boxd sandbox %s health check error: %s", self.id, e)
return False
return result.exit_code == 0 and "ok" in _text(result.stdout).split()
# ── SDK bridge ───────────────────────────────────────────────────────
def _note_failure(self, error: Exception) -> None:
if self._on_terminal_failure is None or not is_machine_gone_error(error):
return
# Confirm before the provider drops the sandbox: a not-found answer
# from one call is not proof that the machine itself is gone.
try:
self._client.machines.get(self._machine_id, region=self._region)
except Exception as e:
if not is_machine_gone_error(e):
return
else:
return
try:
self._on_terminal_failure(self.id, str(error))
except Exception:
logger.exception("Terminal boxd failure callback errored for %s", self.id)
def _ensure_open(self) -> None:
with self._lock:
if self._closed:
raise RuntimeError("sandbox has been closed")
def _start_machine(self) -> None:
"""Start a stopped machine and wait until it runs commands again.
``wait_until_ready`` reports the machine state; a no-op probe confirms
that commands reach it before the original call is repeated.
"""
logger.info("boxd machine for sandbox %s is stopped; starting it", self.id)
machines = self._client.machines
machines.start(self._machine_id, region=self._region)
machines.wait_until_ready(self._machine_id, region=self._region)
deadline = time.monotonic() + _START_PROBE_SECONDS
while True:
try:
machines.exec(self._machine_id, ["true"], timeout=_START_PROBE_INTERVAL * 5, region=self._region)
return
except Exception:
if time.monotonic() >= deadline:
raise
time.sleep(_START_PROBE_INTERVAL)
def _call(self, op: Callable[[], Any], *, repeat: bool = True, recover: bool = True) -> Any:
"""Run one SDK call, starting the machine if it is stopped.
A machine in standby or hibernation wakes on the call itself. A stopped
machine is started, and the call is repeated once when ``repeat`` is
true. Commands pass ``repeat=False``: a machine stopped while the
command ran must not run it a second time, so the caller gets an error
and decides. Budgeted mirror calls pass ``recover=False`` to skip the
control-plane calls used for recovery and failure confirmation.
"""
self._ensure_open()
if not recover:
# Budgeted output copies must not enter control-plane recovery or
# failure probes, whose SDK calls have separate request deadlines.
return op()
try:
try:
return op()
except Exception as first:
if not is_stopped_machine_error(first):
raise
try:
status = str(getattr(self._client.machines.get(self._machine_id, region=self._region), "status", ""))
except Exception:
raise first from None
if status.rsplit(".", 1)[-1].lower() != "stopped":
raise
self._start_machine()
if not repeat:
raise RuntimeError("The sandbox machine was stopped and has been started again; the command did not run to completion, run it again") from first
return op()
except Exception as e:
self._note_failure(e)
raise
def _run(self, argv: list[str], *, env: dict[str, str] | None = None, timeout: float | None = None, binary: bool = False, repeat: bool = True, deadline: float | None = None) -> Any:
if deadline is not None:
remaining = deadline - time.monotonic()
if remaining <= 0:
raise TimeoutError("boxd output synchronization deadline exceeded")
timeout = min(timeout, remaining) if timeout is not None else remaining
return self._call(
lambda: self._client.machines.exec(
self._machine_id,
argv,
env=env or None,
timeout=timeout,
encoding="buffer" if binary else "utf-8",
region=self._region,
),
repeat=repeat,
recover=deadline is None,
)
def run_shell(self, script: str, *, timeout: float = _HELPER_TIMEOUT_SECONDS, binary: bool = False, deadline: float | None = None) -> Any:
"""Run an internal helper script with the configured environment."""
return self._run(["bash", "-lc", script], env=self._default_env, timeout=timeout, binary=binary, deadline=deadline)
# ── Paths ────────────────────────────────────────────────────────────
@staticmethod
def _guard_path(path: str) -> str:
if not path:
raise ValueError("path must be a non-empty string")
normalized = path.replace("\\", "/")
if any(segment == ".." for segment in normalized.split("/")):
raise PermissionError(f"Access denied: path traversal detected in '{path}'")
if not normalized.startswith("/"):
raise ValueError(f"path must be absolute: '{path}'")
return normalized
# ── Command execution ────────────────────────────────────────────────
def execute_command(
self,
command: str,
env: dict[str, str] | None = None,
timeout: float | None = None,
) -> str:
"""Run ``command`` through ``bash -lc`` and return combined output.
The deadline (``timeout``, else the provider default) is enforced
inside the machine with coreutils ``timeout``, which ends the whole
command with exit code 124 and keeps the output it produced so far.
The SDK call gets a slightly longer deadline as a backstop.
"""
_validate_extra_env(env)
if self.is_closed:
return "Error: sandbox has been closed"
merged_env = {**self._default_env, **(env or {})}
if timeout is None or not math.isfinite(timeout) or timeout <= 0:
timeout = self._default_command_timeout
seconds = max(1, math.ceil(timeout))
argv = ["timeout", "-k", str(_KILL_AFTER_SECONDS), str(seconds), "bash", "-lc", command]
call_timeout = seconds + _KILL_AFTER_SECONDS + _TIMEOUT_GRACE_SECONDS
try:
result = self._run(argv, env=merged_env, timeout=call_timeout, repeat=False)
except Exception as e:
logger.error("Failed to execute command in boxd sandbox %s: %s", self.id, e)
return f"Error: {e}"
stdout = _text(result.stdout)
stderr = _text(result.stderr)
if stdout and stderr:
output = f"{stdout}\n{stderr}"
else:
output = stdout or stderr
exit_code = result.exit_code
if exit_code not in (0, None):
output = f"{output}\nExit Code: {exit_code}" if output else f"Command exited with code {exit_code}\nExit Code: {exit_code}"
return output if output else "(no output)"
# ── File operations ──────────────────────────────────────────────────
def _read_bytes(self, path: str, limit: int, *, timeout: float = _READ_TIMEOUT_SECONDS, deadline: float | None = None) -> bytes:
"""Read at most ``limit`` bytes with a deadline.
``head -c limit+1`` bounds what the machine sends, so an oversized
file is detected without buffering more than one byte past the cap,
and the exec deadline bounds a stalled transfer.
"""
result = self._run(["head", "-c", str(limit + 1), "--", path], timeout=timeout, binary=True, deadline=deadline)
data = result.stdout if isinstance(result.stdout, bytes) else _text(result.stdout).encode("utf-8")
if result.exit_code != 0:
message = _text(result.stderr).strip() or f"head exited with code {result.exit_code}"
if "No such file" in message:
raise FileNotFoundError(errno.ENOENT, "No such file or directory", path)
if "Is a directory" in message:
raise IsADirectoryError(errno.EISDIR, "Is a directory", path)
if "Permission denied" in message:
raise PermissionError(errno.EACCES, "Permission denied", path)
raise OSError(f"cannot read '{path}': {message}")
if len(data) > limit:
raise OSError(errno.EFBIG, f"File exceeds maximum download size of {limit} bytes", path)
return data
def read_file(
self,
path: str,
start_line: int | None = None,
end_line: int | None = None,
) -> str:
try:
resolved = self._guard_path(path)
content = self._read_bytes(resolved, _MAX_READ_FILE_SIZE).decode("utf-8", errors="replace")
except Exception as e:
logger.error("read_file %s failed: %s", path, e)
return f"Error: {e}"
if start_line is None and end_line is None:
return content
lines = split_file_lines(content)
# Clamp like LocalSandbox.read_file: a negative start must not wrap
# around through negative-index slicing.
start = max(start_line or 1, 1)
end = max(end_line, 0) if end_line is not None else len(lines)
return "\n".join(lines[start - 1 : end])
def write_file(self, path: str, content: str, append: bool = False) -> None:
self._write_bytes(self._guard_path(path), content.encode("utf-8"), append=append)
def update_file(self, path: str, content: bytes) -> None:
self._write_bytes(self._guard_path(path), content, append=False)
def _write_bytes(self, resolved: str, data: bytes, *, append: bool) -> None:
fs = self._client.machines.fs
if not append:
# fs.write creates missing parent directories.
self._call(lambda: fs.write(self._machine_id, resolved, data, region=self._region))
return
with self._write_lock:
try:
existing = self._read_bytes(resolved, _MAX_READ_FILE_SIZE)
except Exception as e:
if not is_missing_file_error(e):
raise
existing = b""
payload = existing + data
self._call(lambda: fs.write(self._machine_id, resolved, payload, region=self._region))
def download_file(self, path: str, *, max_bytes: int | None = None, deadline: float | None = None) -> bytes:
"""Download under the size cap and optional monotonic mirror deadline."""
limit = download_limit(max_bytes, _MAX_DOWNLOAD_SIZE)
normalized = self._guard_path(path)
if normalized != VIRTUAL_PATH_PREFIX and not normalized.startswith(f"{VIRTUAL_PATH_PREFIX}/"):
raise PermissionError(f"Access denied: path must be under '{VIRTUAL_PATH_PREFIX}': '{path}'")
# head -c stops one byte past the cap, so a file that grows during
# the transfer still cannot exceed it.
try:
return self._read_bytes(normalized, limit, timeout=_DOWNLOAD_TIMEOUT_SECONDS, deadline=deadline)
except OSError as e:
if e.errno == errno.EFBIG or is_missing_file_error(e):
raise
raise OSError(f"cannot read '{path}' from sandbox: {e}") from e
except Exception as e:
raise OSError(f"cannot read '{path}' from sandbox: {e}") from e
def list_dir(self, path: str, max_depth: int = 2) -> list[str]:
resolved = self._guard_path(path)
result = self.run_shell(remote_list_dir_command(resolved, max_depth))
return parse_remote_list_dir_output(_text(result.stdout), resolved, pipeline_exit_code=result.exit_code)
def glob(
self,
path: str,
pattern: str,
*,
include_dirs: bool = False,
max_results: int = 200,
) -> tuple[list[str], bool]:
resolved = self._guard_path(path)
types = ("f", "d") if include_dirs else ("f",)
type_expr = " -o ".join(f"-type {t}" for t in types)
hard_limit = max(max_results * 4, max_results + 50)
# -H follows a symlinked search root, as list_dir does.
search = f"find -H {shlex.quote(resolved)} \\( {type_expr} \\) -print 2>/dev/null"
result = self.run_shell(remote_search_command(search, resolved, limit=hard_limit))
# A missing root or a failed find must not read as "no files matched".
output = parse_remote_search_output(_text(result.stdout), resolved, tool="find", limit=hard_limit)
matches: list[str] = []
root = resolved.rstrip("/") or "/"
root_prefix = root if root == "/" else f"{root}/"
# Records are LF-delimited; other splitlines() boundaries can be path characters.
for entry in output.text.split("\n"):
# Do not strip: trailing whitespace can be part of a file name.
if not entry or (entry != root and not entry.startswith(root_prefix)):
continue
if should_ignore_path_under_root(entry, root):
continue
rel_path = entry[len(root) :].lstrip("/")
if not rel_path:
continue
if path_matches(pattern, rel_path):
matches.append(entry)
# Look one match past the cap so an exhausted tree that held
# exactly max_results is not reported as truncated.
if len(matches) > max_results:
return matches[:max_results], True
return matches, output.truncated
def grep(
self,
path: str,
pattern: str,
*,
glob: str | None = None,
literal: bool = False,
case_sensitive: bool = False,
max_results: int = 100,
) -> tuple[list[GrepMatch], bool]:
# grep gets the raw pattern: -F matches it literally, -E as a regex.
# The Python compile only catches gross errors at the boundary.
if not literal:
re.compile(pattern, 0 if case_sensitive else re.IGNORECASE)
resolved = self._guard_path(path)
# --null ends the file name with NUL, so a colon in a path cannot be
# mistaken for the line-number separator.
flags = ["-r", "-n", "-H", "-I", "--null"]
if not case_sensitive:
flags.append("-i")
flags.append("-F" if literal else "-E")
if glob is not None:
# --include matches basenames only; it is a coarse pre-filter and
# path_matches below enforces the real directory scope.
include_pattern = glob.split("/")[-1] or glob
flags.append(shlex.quote(f"--include={include_pattern}"))
per_file_cap = max(max_results + 1, 50)
total_cap = max(max_results * 4, max_results + 50)
flags.append(f"-m{per_file_cap}")
search = "grep " + " ".join(flags) + f" -- {shlex.quote(pattern)} {shlex.quote(resolved)} 2>/dev/null"
result = self.run_shell(remote_search_command(search, resolved, limit=total_cap))
# A missing root, a missing grep or an unreadable tree must not read as "no matches".
output = parse_remote_search_output(_text(result.stdout), resolved, tool="grep", limit=total_cap)
root = resolved.rstrip("/") or "/"
root_prefix = root if root == "/" else f"{root}/"
matches: list[GrepMatch] = []
for raw in output.text.split("\n"):
try:
file_path, rest = raw.split("\0", 1)
line_no_str, line_text = rest.split(":", 1)
line_number = int(line_no_str)
except ValueError:
continue
if should_ignore_path_under_root(file_path, root):
continue
if glob is not None:
if file_path != root and not file_path.startswith(root_prefix):
continue
rel_path = file_path.rsplit("/", 1)[-1] if file_path == root else file_path[len(root) :].lstrip("/")
if not path_matches(glob, rel_path):
continue
matches.append(GrepMatch(path=file_path, line_number=line_number, line=truncate_line(line_text)))
if len(matches) > max_results:
return matches[:max_results], True
return matches, output.truncated
def _text(value: str | bytes | None) -> str:
if value is None:
return ""
if isinstance(value, bytes):
return value.decode("utf-8", errors="replace")
return value
__all__ = ["BoxdSandbox"]

View File

@ -0,0 +1,949 @@
"""``BoxdSandboxProvider``: DeerFlow :class:`SandboxProvider` backed by boxd.
Runs each DeerFlow sandbox as an isolated `boxd <https://boxd.sh?ref=deer-flow>`_
machine: a full Ubuntu microVM with a persistent disk. The provider creates
one machine per ``(user, thread)``, reuses it within the process, and parks
released machines in a warm pool (shared :class:`WarmPoolLifecycleMixin`
machinery) for fast reclaim. The idle reaper deletes warm machines after
``idle_timeout``.
A fresh machine for a thread gets the thread's Gateway-side files (uploads,
outputs, workspace) and the skills projection before it is handed out.
Release mirrors ``/mnt/user-data/outputs`` back to the Gateway, so artifacts
stay downloadable after the machine is gone.
Config is read off :class:`SandboxConfig` (``extra="allow"``); see this
package's ``__init__`` docstring for the full set. The ``boxd`` SDK is imported
lazily, so the harness installs without it.
"""
from __future__ import annotations
import asyncio
import atexit
import logging
import math
import os
import posixpath
import shlex
import stat
import tarfile
import tempfile
import threading
import time
import unicodedata
import uuid
from collections.abc import Iterator
from contextlib import contextmanager
from functools import partial
from pathlib import Path, PurePosixPath
from typing import TYPE_CHECKING, Any
from deerflow.community.provider_credentials import resolve_api_key
from deerflow.config import get_app_config
from deerflow.sandbox.acquire_serialization import AcquireSerializer, AcquireSerializerClosed
from deerflow.sandbox.download import DEFAULT_DOWNLOAD_LIMIT
from deerflow.sandbox.identity import derive_sandbox_scope_token
from deerflow.sandbox.sandbox import Sandbox, _validate_extra_env
from deerflow.sandbox.sandbox_provider import SandboxProvider
from ..warm_pool_lifecycle import WarmPoolLifecycleMixin
from .boxd_sandbox import DEFAULT_COMMAND_TIMEOUT, BoxdSandbox
if TYPE_CHECKING:
from boxd import Boxd
from deerflow.skills.projection import SkillProjectionPaths
logger = logging.getLogger(__name__)
DEFAULT_NAME_PREFIX = "deer-flow-"
DEFAULT_READY_TIMEOUT = 120.0
USER_DATA_ROOT = "/mnt/user-data"
OUTPUTS_ROOT = f"{USER_DATA_ROOT}/outputs"
SKILL_CATEGORIES = ("public", "custom", "legacy", "integrations")
# Machine sizes boxd offers, as vCPU -> memory in GiB.
MACHINE_SIZES = {1: 4, 2: 8, 4: 16, 8: 32}
VALID_REGIONS = frozenset({"fra", "nyc"})
_BOOTSTRAP_TIMEOUT = 60.0
_UPLOAD_EXTRACT_TIMEOUT = 300.0
# Deadline for each unary SDK call; an upload of up to 128 MiB is one call.
_CLIENT_TIMEOUT = 180.0
_LABEL_APP = "deer-flow"
_LABEL_SCOPE = "deer-flow-scope"
_LABEL_GENERATION = "deer-flow-generation"
_PROTECTED_ROOTS = ("/bin", "/boot", "/dev", "/etc", "/lib", "/lib64", "/proc", "/root", "/run", "/sbin", "/sys", "/usr", "/var")
_WINDOWS_RESERVED_NAMES = frozenset({"CON", "PRN", "AUX", "NUL", "CONIN$", "CONOUT$"} | {f"{prefix}{digit}" for prefix in ("COM", "LPT") for digit in "123456789¹²³"})
def _import_client() -> type[Boxd]:
"""Import boxd's ``Boxd`` client lazily (optional dependency)."""
try:
from boxd import Boxd
except ImportError as e: # pragma: no cover - depends on the optional dependency
raise ImportError("BoxdSandboxProvider requires the optional 'boxd' dependency. Install it with: pip install 'deerflow-harness[boxd]' or pip install 'boxd>=0.3.2'.") from e
return Boxd
def _validate_skills_root(container_path: str) -> str:
"""The skills categories are replaced recursively, so the root must be a dedicated path."""
normalized = posixpath.normpath(container_path or "")
if not normalized.startswith("/") or normalized == "/" or normalized != (container_path or "").rstrip("/"):
raise ValueError(f"skills.container_path {container_path!r} is not a safe boxd skills root: use a normalized absolute path such as /mnt/skills")
if normalized in {"/home", "/home/boxd", "/mnt", "/tmp"} or any(normalized == root or normalized.startswith(f"{root}/") for root in _PROTECTED_ROOTS):
raise ValueError(f"skills.container_path {container_path!r} is not a safe boxd skills root: it is a system directory")
if normalized == USER_DATA_ROOT or normalized.startswith(f"{USER_DATA_ROOT}/") or USER_DATA_ROOT.startswith(f"{normalized}/"):
raise ValueError(f"skills.container_path {container_path!r} must not overlap {USER_DATA_ROOT}")
return normalized
class _ShutdownIncompleteError(RuntimeError):
"""Cleanup still owns acquisitions or machine deletions that need a retry."""
class _AcquireTicket:
"""Hands an acquire's result to a caller that may have been cancelled."""
def __init__(self) -> None:
self.lock = threading.Lock()
self.abandoned = False
self.sandbox_id: str | None = None
class _UploadBudget:
"""File-count and byte limits for one tar upload into a machine."""
def __init__(self, max_files: int, max_bytes: int) -> None:
self.max_files = max_files
self.max_bytes = max_bytes
self.files = 0
self.bytes = 0
def admit(self, size: int) -> bool:
if self.files + 1 > self.max_files or self.bytes + size > self.max_bytes:
return False
self.files += 1
self.bytes += size
return True
class BoxdSandboxProvider(WarmPoolLifecycleMixin[BoxdSandbox], SandboxProvider):
"""Run each DeerFlow sandbox as an isolated boxd machine."""
uses_thread_data_mounts = False
needs_upload_permission_adjustment = True
# The machine sees only the skills uploaded into it, so a per-Agent
# projection from sync_agent_skills cannot be bypassed from the shell.
supports_agent_skill_isolation = True
_idle_checker_thread_name = "boxd-idle-reaper"
_MAX_SYNC_FILES = 2000
_MAX_SYNC_TOTAL_BYTES = 512 * 1024 * 1024
_MAX_SYNC_LISTING_BYTES = 512 * 1024
_SYNC_DEADLINE_SECONDS = 120.0
_MAX_UPLOAD_FILES = 5000
_MAX_UPLOAD_BYTES = 128 * 1024 * 1024
_UPLOAD_CHUNK_BYTES = 1024 * 1024
def __init__(self) -> None:
self._lock = threading.Lock()
self._sandboxes: dict[str, BoxdSandbox] = {}
self._thread_sandboxes: dict[tuple[str, str], str] = {}
self._warm_pool: dict[str, tuple[BoxdSandbox, float]] = {}
self._acquire_serializer: AcquireSerializer[str] = AcquireSerializer(thread_name_prefix="boxd-acquire-wait")
self._idle_checker_stop = threading.Event()
self._idle_checker_thread: threading.Thread | None = None
self._shutdown_called = False
self._shutdown_complete = False
self._shutdown_lock = threading.Lock()
self._accepting = True
self._acquires = 0
# Keyed by object identity: a replacement can reuse the logical id
# while an older machine still needs deletion.
self._pending_closes: dict[int, BoxdSandbox] = {}
self._closing: set[int] = set()
# Callers that received each active sandbox and have not released it.
self._claims: dict[str, int] = {}
self._client: Boxd | None = None
self._config = self._load_config()
atexit.register(self._shutdown_at_exit)
self._start_idle_checker()
# ── Config ───────────────────────────────────────────────────────────
@staticmethod
def _optional_int(name: str, value: Any, *, minimum: int) -> int | None:
if value is None:
return None
if isinstance(value, bool) or not isinstance(value, int) or value < minimum:
raise ValueError(f"sandbox.{name} must be an integer >= {minimum}")
return value
@staticmethod
def _memory_gib(value: Any) -> int | None:
if value is None:
return None
if isinstance(value, int) and not isinstance(value, bool):
return value
text = str(value).strip().upper().removesuffix("IB").removesuffix("B").removesuffix("G")
if text.isdigit():
return int(text)
raise ValueError("sandbox.memory must be a size such as 8G")
def _load_config(self) -> dict[str, Any]:
app_config = get_app_config()
sandbox_config = app_config.sandbox
def option(name: str) -> Any:
return getattr(sandbox_config, name, None)
# $VARS in config.yaml are already resolved by AppConfig; resolving
# again would rewrite a secret that itself starts with $.
environment = dict(option("environment") or {})
_validate_extra_env(environment)
region = option("region")
if region is not None and region not in VALID_REGIONS:
raise ValueError(f"sandbox.region must be one of {sorted(VALID_REGIONS)}")
vcpu = self._optional_int("vcpu", option("vcpu"), minimum=1)
memory = self._memory_gib(option("memory"))
if vcpu is not None and vcpu not in MACHINE_SIZES:
raise ValueError(f"sandbox.vcpu must be one of {sorted(MACHINE_SIZES)}")
if memory is not None and memory not in MACHINE_SIZES.values():
raise ValueError(f"sandbox.memory must be one of {[f'{gib}G' for gib in MACHINE_SIZES.values()]}")
if vcpu is not None and memory is not None and MACHINE_SIZES[vcpu] != memory:
raise ValueError(f"sandbox.vcpu={vcpu} goes with sandbox.memory={MACHINE_SIZES[vcpu]}G")
if vcpu is not None and memory is None:
memory = MACHINE_SIZES[vcpu]
if memory is not None and vcpu is None:
vcpu = next(cpu for cpu, gib in MACHINE_SIZES.items() if gib == memory)
labels = option("labels") or {}
if not isinstance(labels, dict) or not all(isinstance(k, str) and isinstance(v, str) for k, v in labels.items()):
raise ValueError("sandbox.labels must map strings to strings")
name_prefix = option("name_prefix") or DEFAULT_NAME_PREFIX
if not isinstance(name_prefix, str) or not name_prefix.replace("-", "").isalnum() or name_prefix != name_prefix.lower():
raise ValueError("sandbox.name_prefix may hold lowercase letters, digits and '-'")
ready_timeout = option("ready_timeout")
try:
ready_timeout = float(DEFAULT_READY_TIMEOUT if ready_timeout is None else ready_timeout)
except (TypeError, ValueError) as exc:
raise ValueError("sandbox.ready_timeout must be positive") from exc
if not math.isfinite(ready_timeout) or ready_timeout <= 0:
raise ValueError("sandbox.ready_timeout must be positive")
command_timeout = option("bash_command_timeout")
try:
command_timeout = float(DEFAULT_COMMAND_TIMEOUT if command_timeout is None else command_timeout)
except (TypeError, ValueError) as exc:
raise ValueError("sandbox.bash_command_timeout must be positive") from exc
if not math.isfinite(command_timeout) or command_timeout <= 0:
raise ValueError("sandbox.bash_command_timeout must be positive")
replicas = option("replicas")
idle_timeout = option("idle_timeout")
return {
"command_timeout": command_timeout,
# Blank, absent or template values read as None; the SDK then
# reads BOXD_API_KEY from the environment.
# A boxd-specific name, so a key configured for another provider
# under sandbox.api_key is never sent to boxd.
"api_key": resolve_api_key(option("boxd_api_key")),
"org": option("org"),
"region": region,
"vcpu": vcpu,
"memory": f"{memory}G" if memory is not None else None,
"from_snapshot": option("from_snapshot"),
"auto_suspend_timeout": self._optional_int("auto_suspend_timeout", option("auto_suspend_timeout"), minimum=0),
"ready_timeout": ready_timeout,
"labels": dict(labels),
"name_prefix": name_prefix,
"environment": environment,
"skills_container_path": _validate_skills_root(app_config.skills.container_path),
"replicas": replicas if replicas is not None else self.DEFAULT_REPLICAS,
"idle_timeout": idle_timeout if idle_timeout is not None else self.DEFAULT_IDLE_TIMEOUT,
}
def _get_client(self) -> Boxd:
with self._lock:
if self._client is not None:
return self._client
client_cls = _import_client()
if self._config["api_key"] is None and not os.environ.get("BOXD_API_KEY"):
raise ValueError("BoxdSandboxProvider needs an API key: set sandbox.boxd_api_key in config.yaml or the BOXD_API_KEY environment variable")
client = client_cls(api_key=self._config["api_key"], region=self._config["region"], timeout=_CLIENT_TIMEOUT)
with self._lock:
if self._client is None:
self._client = client
return self._client
# ── Identity ─────────────────────────────────────────────────────────
@staticmethod
def _sandbox_id(thread_id: str, user_id: str) -> str:
return derive_sandbox_scope_token(user_id=user_id, thread_id=thread_id)
@staticmethod
def _thread_key(thread_id: str, user_id: str | None) -> tuple[str, str]:
return (user_id or "", thread_id)
def _machine_name(self, sandbox_id: str) -> str:
# Names are unique per owner and are not free again the moment a
# machine is deleted, so each generation gets a random suffix.
return f"{self._config['name_prefix']}{sandbox_id}-{uuid.uuid4().hex[:6]}"
# ── Warm pool plumbing ───────────────────────────────────────────────
def _start_idle_checker(self) -> None:
if self._config["idle_timeout"] <= 0:
return
super()._start_idle_checker()
def _active_count_locked(self) -> int:
return len(self._sandboxes) + len(self._pending_closes)
def _cleanup_idle_resources(self, idle_timeout: float) -> None:
self._retry_pending_closes()
super()._cleanup_idle_resources(idle_timeout)
def _destroy_warm_entry(self, sandbox_id: str, entry: BoxdSandbox, *, reason: str) -> None:
self._close_quietly(entry, context=f"warm pool, reason={reason}")
def _close_quietly(self, sandbox: BoxdSandbox, *, context: str) -> None:
"""Own deletion until it is accepted; keep failures for a later retry."""
token = id(sandbox)
with self._lock:
self._pending_closes[token] = sandbox
if token in self._closing:
return
self._closing.add(token)
closed = False
try:
sandbox.close()
except Exception as e:
logger.warning("Error deleting boxd sandbox %s (%s): %s", sandbox.id, context, e)
else:
closed = True
finally:
with self._lock:
if closed:
self._pending_closes.pop(token, None)
self._closing.discard(token)
def _retry_pending_closes(self) -> None:
with self._lock:
pending = list(self._pending_closes.values())
for sandbox in pending:
self._close_quietly(sandbox, context="retry deletion")
def _invalidate_sandbox(self, sandbox_id: str, reason: str, *, expected_sandbox: BoxdSandbox | None = None) -> None:
"""Deregister a sandbox whose machine is gone, and finish its cleanup."""
with self._lock:
active = self._sandboxes.get(sandbox_id)
warm_entry = self._warm_pool.get(sandbox_id)
current = active or (warm_entry[0] if warm_entry is not None else None)
if expected_sandbox is not None and current is not expected_sandbox:
logger.debug("Ignoring late terminal failure from replaced boxd sandbox %s: %s", sandbox_id, reason)
return
self._sandboxes.pop(sandbox_id, None)
self._claims.pop(sandbox_id, None)
self._warm_pool.pop(sandbox_id, None)
for key in [k for k, sid in self._thread_sandboxes.items() if sid == sandbox_id]:
self._thread_sandboxes.pop(key, None)
if current is None:
return
self._pending_closes[id(current)] = current
logger.warning("Invalidating boxd sandbox %s: %s", sandbox_id, reason)
self._close_quietly(current, context="terminal failure")
# ── Acquire ──────────────────────────────────────────────────────────
@contextmanager
def _acquiring(self) -> Iterator[None]:
with self._lock:
if not self._accepting:
raise RuntimeError("boxd provider is retired or shutting down")
self._acquires += 1
try:
yield
finally:
with self._lock:
self._acquires -= 1
def acquire(self, thread_id: str | None = None, *, user_id: str | None = None) -> str:
return self._acquire(thread_id, user_id)
def _acquire(self, thread_id: str | None, user_id: str | None, ticket: _AcquireTicket | None = None) -> str:
with self._acquiring():
if thread_id is None:
sandbox_id = str(uuid.uuid4())[:8]
sandbox_id = self._publish_sandbox(self._create_sandbox(sandbox_id, thread_id=None, user_id=user_id))
else:
key = self._thread_key(thread_id, user_id)
scope_id = self._sandbox_id(thread_id, user_id or "")
with self._acquire_serializer.hold(scope_id):
sandbox_id = self._acquire_scope_locked(key, scope_id, thread_id=thread_id, user_id=user_id)
if ticket is not None:
with ticket.lock:
ticket.sandbox_id = sandbox_id
abandoned = ticket.abandoned
if abandoned:
self._drop_claim(sandbox_id)
return sandbox_id
async def acquire_async(self, thread_id: str | None = None, *, user_id: str | None = None) -> str:
"""Acquire without blocking the event loop.
The whole synchronous acquire (serializer wait and body) runs on the
serializer's dedicated executor. A cancelled caller never receives the
sandbox id, so it gives back the claim that the worker took for it,
either right away or when the worker finishes. A sandbox whose last
claim is given back is released to the warm pool, where the idle
reaper deletes it. Neither path depends on the event loop.
"""
ticket = _AcquireTicket()
try:
return await self._acquire_serializer.run_on_executor(partial(self._acquire, thread_id, user_id, ticket))
except asyncio.CancelledError:
with ticket.lock:
ticket.abandoned = True
sandbox_id = ticket.sandbox_id
if sandbox_id is not None:
# release() can transfer files and wait on the scope lock.
threading.Thread(target=self._drop_claim, args=(sandbox_id,), name="boxd-abandoned-release", daemon=True).start()
raise
def _drop_claim(self, sandbox_id: str) -> None:
"""Give back the claim of a caller that never received ``sandbox_id``."""
try:
with self._acquire_serializer.hold(sandbox_id):
with self._lock:
if sandbox_id not in self._sandboxes:
return
remaining = self._claims.get(sandbox_id, 1) - 1
self._claims[sandbox_id] = remaining
if remaining > 0:
return
logger.info("Releasing boxd sandbox %s: every caller that acquired it was cancelled", sandbox_id)
self._release_scope_locked(sandbox_id)
except AcquireSerializerClosed:
pass # reset/shutdown owns the sandbox now
except Exception:
logger.warning("Could not release abandoned boxd sandbox %s", sandbox_id, exc_info=True)
def _publish_sandbox(self, sandbox: BoxdSandbox, key: tuple[str, str] | None = None) -> str:
with self._lock:
if self._accepting:
self._sandboxes[sandbox.id] = sandbox
self._claims[sandbox.id] = 1
if key is not None:
self._thread_sandboxes[key] = sandbox.id
return sandbox.id
# The worker that created the machine owns its cleanup, even when the
# caller abandoned the await or shutdown closed admission meanwhile.
self._close_quietly(sandbox, context="acquired after provider retirement")
raise RuntimeError("boxd provider retired or shut down during acquire")
def _acquire_scope_locked(self, key: tuple[str, str], sandbox_id: str, *, thread_id: str, user_id: str | None) -> str:
with self._lock:
if not self._accepting:
raise RuntimeError("boxd provider is retired or shutting down")
existing = self._thread_sandboxes.get(key)
if existing is not None and existing in self._sandboxes:
self._claims[existing] = self._claims.get(existing, 0) + 1
return existing
reclaimed = self._reclaim_warm_pool(sandbox_id)
if reclaimed is not None:
with self._lock:
if not self._accepting:
raise RuntimeError("boxd provider retired or shut down during acquire")
self._thread_sandboxes[key] = reclaimed
return reclaimed
sandbox = self._create_sandbox(sandbox_id, thread_id=thread_id, user_id=user_id)
return self._publish_sandbox(sandbox, key)
def _create_sandbox(self, sandbox_id: str, *, thread_id: str | None, user_id: str | None) -> BoxdSandbox:
replicas, total = self._replica_count()
if total >= replicas:
evicted = self._evict_oldest_warm()
self._log_replicas_soft_cap(replicas, sandbox_id, evicted)
client = self._get_client()
generation = uuid.uuid4().hex
labels = {**self._config["labels"], _LABEL_APP: "sandbox", _LABEL_SCOPE: sandbox_id, _LABEL_GENERATION: generation}
create_kwargs: dict[str, Any] = {
"org": self._config["org"],
# The agent runs untrusted code: an isolated machine joins no
# private network and holds no boxd credentials.
"isolated": True,
"labels": labels,
}
for name in ("region", "vcpu", "memory", "from_snapshot", "auto_suspend_timeout"):
if self._config[name] is not None:
create_kwargs[name] = self._config[name]
try:
machine = client.machines.create(self._machine_name(sandbox_id), **create_kwargs)
except BaseException:
self._delete_generation(client, generation, sandbox_id)
raise
sandbox = BoxdSandbox(
sandbox_id,
client,
machine.id,
region=self._config["region"],
default_env=self._config["environment"],
default_command_timeout=self._config["command_timeout"],
)
try:
client.machines.wait_until_ready(machine.id, timeout=self._config["ready_timeout"], region=self._config["region"])
self._bootstrap(sandbox)
if thread_id is not None and user_id:
self._restore_thread_files(sandbox, thread_id=thread_id, user_id=user_id)
self._upload_shared_skills(sandbox, thread_id=thread_id, user_id=user_id)
except BaseException:
# Never hand out a half-prepared machine, and never leave one running.
self._close_quietly(sandbox, context="preparation failure")
raise
sandbox._on_terminal_failure = lambda failed_id, reason: self._invalidate_sandbox(failed_id, reason, expected_sandbox=sandbox)
logger.info("Created boxd sandbox %s (machine %s)", sandbox_id, machine.id)
return sandbox
def _delete_generation(self, client: Boxd, generation: str, sandbox_id: str) -> None:
"""Delete a machine whose create raised after the server made it.
The generation label is unique to one create call, so a match can
only be the machine that this call asked for.
"""
try:
found = client.machines.list(org=self._config["org"], labels={_LABEL_GENERATION: generation}, region=self._config["region"])
except Exception:
logger.warning("Could not look for a boxd machine left by a failed create", exc_info=True)
return
for machine in found:
if (getattr(machine, "labels", None) or {}).get(_LABEL_GENERATION) != generation:
continue
sandbox = BoxdSandbox(sandbox_id, client, machine.id, region=self._config["region"])
self._close_quietly(sandbox, context="failed create")
def _bootstrap(self, sandbox: BoxdSandbox) -> None:
skills_root = shlex.quote(self._config["skills_container_path"])
dirs = " ".join(shlex.quote(f"{USER_DATA_ROOT}/{name}") for name in ("workspace", "uploads", "outputs"))
script = f'set -e; sudo -n mkdir -p -- {dirs} {skills_root}; sudo -n chown "$(id -u):$(id -g)" -- {shlex.quote(USER_DATA_ROOT)} {dirs} {skills_root}; echo BOOTSTRAP_OK'
result = sandbox.run_shell(script, timeout=_BOOTSTRAP_TIMEOUT)
stdout = result.stdout if isinstance(result.stdout, str) else bytes(result.stdout).decode("utf-8", "replace")
if result.exit_code != 0 or "BOOTSTRAP_OK" not in stdout:
raise RuntimeError(f"boxd sandbox bootstrap failed with exit code {result.exit_code}")
# ── Host to machine transfer ─────────────────────────────────────────
def _add_tree(self, archive: tarfile.TarFile, source: Path, arcname: str, budget: _UploadBudget) -> bool:
"""Add regular files below ``source``; skip links and special files.
Returns False when the budget stopped the walk early.
"""
def scan_error(error: OSError) -> None:
raise error
for directory, dirnames, filenames in os.walk(source, followlinks=False, onerror=scan_error):
parent = Path(directory)
dirnames[:] = sorted(name for name in dirnames if not (parent / name).is_symlink())
for name in sorted(filenames):
path = parent / name
st = path.lstat()
if not stat.S_ISREG(st.st_mode):
continue
if not budget.admit(st.st_size):
return False
rel = path.relative_to(source).as_posix()
info = tarfile.TarInfo(f"{arcname}/{rel}")
info.size = st.st_size
info.mode = st.st_mode & 0o755
info.mtime = int(st.st_mtime)
with path.open("rb") as fh:
archive.addfile(info, fh)
return True
def _upload_trees(self, sandbox: BoxdSandbox, sources: list[tuple[Path, str]], *, dest_root: str, replace: tuple[str, ...] = ()) -> bool:
"""Upload host directories into ``dest_root`` as one tar archive.
``sources`` pairs a host directory with its sub-path below
``dest_root``. ``replace`` lists sub-paths removed before extraction.
Returns False when the upload stopped at its size limit.
"""
budget = _UploadBudget(self._MAX_UPLOAD_FILES, self._MAX_UPLOAD_BYTES)
complete = True
with tempfile.TemporaryFile() as spool:
with tarfile.open(fileobj=spool, mode="w:gz") as archive:
for source, arcname in sources:
for part in (source, *source.parents):
if part.is_symlink():
raise ValueError(f"Refusing symlinked upload root for {arcname}")
if not source.is_dir():
continue
if not self._add_tree(archive, source, arcname, budget):
complete = False
break
if budget.files == 0 and not replace:
return complete
remote_tar = f"/tmp/deerflow-upload-{uuid.uuid4().hex}.tar.gz"
def chunks() -> Iterator[bytes]:
spool.seek(0)
while chunk := spool.read(self._UPLOAD_CHUNK_BYTES):
yield chunk
sandbox._call(lambda: sandbox._client.machines.fs.write(sandbox.machine_id, remote_tar, chunks(), region=sandbox._region))
root = shlex.quote(dest_root)
removals = " ".join(shlex.quote(f"{dest_root}/{name}") for name in replace)
script = (
f"set -e; trap 'rm -f {shlex.quote(remote_tar)}' EXIT; "
f"if [ -L {root} ]; then echo 'refusing symlinked destination' >&2; exit 2; fi; "
+ (f"rm -rf -- {removals}; " if removals else "")
+ f"mkdir -p -- {root}; tar -xzf {shlex.quote(remote_tar)} -C {root} --no-same-owner; echo UPLOAD_OK"
)
result = sandbox.run_shell(script, timeout=_UPLOAD_EXTRACT_TIMEOUT)
stdout = result.stdout if isinstance(result.stdout, str) else bytes(result.stdout).decode("utf-8", "replace")
if result.exit_code != 0 or "UPLOAD_OK" not in stdout:
raise RuntimeError(f"boxd upload into {dest_root} failed with exit code {result.exit_code}")
return complete
def _restore_thread_files(self, sandbox: BoxdSandbox, *, thread_id: str, user_id: str) -> None:
"""Give a fresh machine the thread's Gateway-side files.
Uploads go first, then the workspace, then outputs (which the Gateway
already serves). Files past the size limit are skipped with a warning,
so a large thread still gets a sandbox. A transfer failure raises: the
caller discards the machine instead of handing out a half-restored one.
"""
from deerflow.config.paths import get_paths
paths = get_paths()
sources = [
(paths.sandbox_uploads_dir(thread_id, user_id=user_id), "uploads"),
(paths.sandbox_work_dir(thread_id, user_id=user_id), "workspace"),
(paths.sandbox_outputs_dir(thread_id, user_id=user_id), "outputs"),
]
if not self._upload_trees(sandbox, sources, dest_root=USER_DATA_ROOT):
logger.warning(
"Thread files for boxd sandbox %s exceed %d files or %d bytes; the rest were not copied into the new machine",
sandbox.id,
self._MAX_UPLOAD_FILES,
self._MAX_UPLOAD_BYTES,
)
def _upload_shared_skills(self, sandbox: BoxdSandbox, *, thread_id: str, user_id: str) -> None:
"""Best effort: upload the user's shared skills projection.
A thread with an Agent skill policy view gets its projection from the
sandbox middleware through :meth:`sync_agent_skills`, so it never sees
the shared view first.
"""
from deerflow.config.paths import get_paths
from deerflow.skills.projection import ensure_skill_projections
from deerflow.skills.storage import get_or_new_user_skill_storage
try:
if get_paths().thread_skills_view_dir(thread_id, user_id=user_id).exists():
return
storage = get_or_new_user_skill_storage(user_id, app_config=get_app_config())
projection = ensure_skill_projections(storage)
self._upload_skills(sandbox, projection)
except Exception as exc:
logger.warning("Could not upload the skills projection for boxd sandbox %s: %s", sandbox.id, exc)
def _upload_skills(self, sandbox: BoxdSandbox, projection: SkillProjectionPaths) -> None:
sources = [(Path(getattr(projection, category)), category) for category in SKILL_CATEGORIES]
if not self._upload_trees(sandbox, sources, dest_root=self._config["skills_container_path"], replace=SKILL_CATEGORIES):
raise RuntimeError("Skills projection exceeds the boxd upload limit")
def sync_agent_skills(
self,
sandbox_id: str,
*,
thread_id: str,
user_id: str,
projection: SkillProjectionPaths,
) -> None:
"""Replace the machine's skill categories with this thread's projection.
Holds the same scope serializer as acquire and release, so no other
lifecycle step sees a half-written skills tree.
"""
with self._acquire_serializer.hold(sandbox_id):
with self._lock:
sandbox = self._sandboxes.get(sandbox_id)
if sandbox is None:
raise RuntimeError(f"boxd sandbox {sandbox_id} is not available for skill synchronization")
self._upload_skills(sandbox, projection)
# ── Reuse ────────────────────────────────────────────────────────────
def _reclaim_warm_pool(self, sandbox_id: str) -> str | None:
with self._lock:
warm_entry = self._warm_pool.get(sandbox_id)
if warm_entry is None:
return None
sandbox = warm_entry[0]
if not sandbox.ping():
with self._lock:
removed = self._warm_pool.pop(sandbox_id, None)
if removed is not None:
self._destroy_warm_entry(sandbox_id, removed[0], reason="health_check_failed")
return None
with self._lock:
removed = self._warm_pool.pop(sandbox_id, None)
if removed is None:
return None
admitted = self._accepting
if admitted:
self._sandboxes[sandbox_id] = removed[0]
self._claims[sandbox_id] = 1
if not admitted:
self._close_quietly(removed[0], context="reclaimed after provider retirement")
raise RuntimeError("boxd provider retired or shut down during acquire")
logger.info("Reclaimed warm boxd sandbox %s", sandbox_id)
return sandbox_id
def get(self, sandbox_id: str) -> Sandbox | None:
with self._lock:
return self._sandboxes.get(sandbox_id)
# ── Release and output mirroring ─────────────────────────────────────
def release(self, sandbox_id: str) -> None:
"""Mirror outputs to the Gateway, then park the machine in the warm pool.
Holds the acquire serializer, so the next turn cannot reuse the
machine while its outputs are still being copied, and keeps the active
entry during the transfer so the reaper cannot delete it.
"""
with self._lock:
if sandbox_id not in self._sandboxes:
return
try:
with self._acquire_serializer.hold(sandbox_id):
self._release_scope_locked(sandbox_id)
except AcquireSerializerClosed:
# reset/shutdown closed the serializer after the check above.
with self._lock:
untracked = sandbox_id not in self._sandboxes
if not untracked:
raise
def _release_scope_locked(self, sandbox_id: str) -> None:
with self._lock:
sandbox = self._sandboxes.get(sandbox_id)
scope = next((key for key, value in self._thread_sandboxes.items() if value == sandbox_id), None)
if sandbox is None:
return
if scope is not None and scope[0]:
try:
self._sync_outputs_to_host(sandbox, user_id=scope[0], thread_id=scope[1])
except Exception:
logger.warning("Failed to synchronize boxd outputs for %s", sandbox_id, exc_info=True)
with self._lock:
# A terminal failure or shutdown may have untracked the sandbox
# during the transfer. Never resurrect it in the warm pool.
if self._sandboxes.get(sandbox_id) is not sandbox:
return
self._sandboxes.pop(sandbox_id)
self._claims.pop(sandbox_id, None)
for key in [key for key, value in self._thread_sandboxes.items() if value == sandbox_id]:
self._thread_sandboxes.pop(key, None)
close_now = self._shutdown_called
if close_now:
self._pending_closes[id(sandbox)] = sandbox
else:
self._warm_pool[sandbox_id] = (sandbox, time.monotonic())
if close_now:
self._close_quietly(sandbox, context="released during shutdown")
else:
logger.info("Released boxd sandbox %s to the warm pool", sandbox_id)
@staticmethod
def _output_parts(remote_path: str) -> tuple[str, ...]:
prefix = f"{OUTPUTS_ROOT}/"
if not remote_path.startswith(prefix) or "\\" in remote_path:
raise ValueError("Output path is outside the outputs directory")
parts = remote_path[len(prefix) :].split("/")
if any(part in {"", ".", ".."} or ":" in part or part.partition(".")[0].rstrip(" ").upper() in _WINDOWS_RESERVED_NAMES or part.endswith((" ", ".")) for part in parts):
raise ValueError("Unsafe output path")
return tuple(parts)
@staticmethod
def _output_name_key(name: str) -> str:
return unicodedata.normalize("NFC", name.casefold())
@classmethod
def _ambiguous_output_paths(cls, remote_paths: list[str | None]) -> set[str]:
# Names that differ only by case or Unicode normalization collide on
# some hosts. A directory alias rejects all of its children.
components: dict[tuple[int, str], tuple[str, int]] = {}
ambiguous: set[int] = set()
path_nodes: dict[str, list[int]] = {}
for remote_path in remote_paths:
if remote_path is None:
continue
try:
parts = cls._output_parts(remote_path)
except ValueError:
continue
parent = 0
nodes = path_nodes.setdefault(remote_path, [])
for part in parts:
key = (parent, cls._output_name_key(part))
spelling, node = components.setdefault(key, (part, len(components) + 1))
if spelling != part:
ambiguous.add(node)
nodes.append(node)
parent = node
return {path for path, nodes in path_nodes.items() if ambiguous.intersection(nodes)}
@classmethod
def _output_destination(cls, root: Path, remote_path: str) -> Path:
parts = cls._output_parts(remote_path)
target = root.joinpath(*PurePosixPath(*parts).parts)
# Reject links in existing parents, including the owner/thread bucket.
for component in reversed((target, *target.parents)):
if component.is_symlink() or component.is_junction():
raise ValueError("Symlinked output destination")
parent = root
for part in parts:
if parent.is_dir():
key = cls._output_name_key(part)
with os.scandir(parent) as entries:
if any(entry.name != part and cls._output_name_key(entry.name) == key for entry in entries):
raise ValueError("Output destination aliases an existing path")
parent = parent / part
target.parent.mkdir(parents=True, exist_ok=True)
if target.exists() and not stat.S_ISREG(target.stat().st_mode):
raise ValueError("Output destination is not a regular file")
return target
def _list_remote_outputs(self, sandbox: BoxdSandbox, *, deadline: float) -> list[str | None] | None:
"""NUL-framed listing of output files; None when it is over the limit."""
root = shlex.quote(OUTPUTS_ROOT)
script = f"set -o pipefail; [ -d {root} ] && [ ! -L {root} ] || exit 0; find {root} -type f -print0 2>/dev/null | head -c {self._MAX_SYNC_LISTING_BYTES + 1}"
result = sandbox.run_shell(script, timeout=self._SYNC_DEADLINE_SECONDS, binary=True, deadline=deadline)
raw = result.stdout if isinstance(result.stdout, bytes) else str(result.stdout).encode("utf-8")
if len(raw) > self._MAX_SYNC_LISTING_BYTES:
logger.warning("boxd output listing exceeded %d bytes; skipping synchronization", self._MAX_SYNC_LISTING_BYTES)
return None
if result.exit_code != 0:
# find hit an unreadable directory: keep the complete records.
logger.warning("boxd output listing had traversal errors; synchronizing complete records")
if raw and not raw.endswith(b"\0"):
raw = raw[: raw.rfind(b"\0") + 1]
paths: list[str | None] = []
for record in filter(None, raw.split(b"\0")):
try:
paths.append(record.decode("utf-8"))
except UnicodeDecodeError:
logger.warning("Skipping non-UTF-8 boxd output %r", record)
paths.append(None)
return paths
def _sync_outputs_to_host(self, sandbox: BoxdSandbox, *, thread_id: str, user_id: str) -> None:
"""Best-effort bounded mirror of the outputs directory to the Gateway."""
from deerflow.config.paths import get_paths
root = get_paths().sandbox_outputs_dir(thread_id, user_id=user_id)
deadline = time.monotonic() + self._SYNC_DEADLINE_SECONDS
remote_paths = self._list_remote_outputs(sandbox, deadline=deadline)
if not remote_paths:
return
# Include records beyond the transfer cap: publishing the first of two
# aliases would otherwise expose its bytes through both artifact URLs.
ambiguous_paths = self._ambiguous_output_paths(remote_paths)
total = 0
for index, remote_path in enumerate(remote_paths):
if index >= self._MAX_SYNC_FILES or total >= self._MAX_SYNC_TOTAL_BYTES or time.monotonic() >= deadline:
logger.warning("boxd output synchronization stopped at its file, byte or time limit")
break
if remote_path is None:
continue
if remote_path in ambiguous_paths:
logger.warning("Skipping ambiguous boxd output %r", remote_path)
continue
staged: Path | None = None
try:
target = self._output_destination(root, remote_path)
data = sandbox.download_file(remote_path, max_bytes=min(DEFAULT_DOWNLOAD_LIMIT, self._MAX_SYNC_TOTAL_BYTES - total), deadline=deadline)
total += len(data)
with tempfile.NamedTemporaryFile(prefix=".boxd-sync-", dir=target.parent, delete=False) as output:
staged = Path(output.name)
output.write(data)
# Check confinement again after the network call, keeping any
# previous complete version on error.
self._output_destination(root, remote_path)
staged.replace(target)
except (OSError, ValueError):
logger.warning("Could not synchronize boxd output %r", remote_path, exc_info=True)
finally:
if staged is not None:
staged.unlink(missing_ok=True)
# ── Teardown ─────────────────────────────────────────────────────────
def reset(self) -> None:
"""Park tracked sandboxes in this instance's warm pool.
``reset_sandbox_provider()`` drops the singleton and calls this so a
config change applies on the next construction. The machines stay
visible to this instance's idle reaper and atexit shutdown.
"""
with self._lock:
self._accepting = False
now = time.monotonic()
for sandbox_id, sandbox in self._sandboxes.items():
self._warm_pool.setdefault(sandbox_id, (sandbox, now))
self._sandboxes.clear()
self._thread_sandboxes.clear()
self._acquire_serializer.close()
def _shutdown_at_exit(self) -> None:
try:
self.shutdown()
except _ShutdownIncompleteError as e:
logger.warning("%s", e)
def shutdown(self) -> None:
# Shutdown retries cleanup but never reopens admission. An SDK call in
# flight cannot be cancelled: its worker keeps ownership and deletes a
# late machine itself. Report unfinished work instead of hiding it.
with self._shutdown_lock:
with self._lock:
if self._shutdown_complete:
return
self._shutdown_called = True
self._accepting = False
self._acquire_serializer.close()
reaper_stopped = True
try:
self._stop_idle_checker()
except RuntimeError:
# The reaper may be inside a slow delete. Delete everything
# else now; _closing keeps the two from deleting one machine twice.
reaper_stopped = False
with self._lock:
active = list(self._sandboxes.values())
warm = [sandbox for sandbox, _ in self._warm_pool.values()]
self._sandboxes.clear()
self._claims.clear()
self._warm_pool.clear()
self._thread_sandboxes.clear()
previous_pending = list(self._pending_closes.values())
for sandbox in {id(box): box for box in active + warm + previous_pending}.values():
self._close_quietly(sandbox, context="shutdown")
with self._lock:
if self._acquires or self._pending_closes or not reaper_stopped:
raise _ShutdownIncompleteError(f"boxd shutdown incomplete: {self._acquires} acquires and {len(self._pending_closes)} deletions pending, reaper stopped: {reaper_stopped}; retry shutdown")
self._shutdown_complete = True
__all__ = ["BoxdSandboxProvider"]

View File

@ -38,6 +38,7 @@ URL_USERINFO_ERROR = "base_url must not contain username or password information
ENV_TEMPLATE_PLACEHOLDERS = frozenset(
{
"your-anthropic-auth-token",
"your-boxd-api-key",
"your-claude-code-oauth-token",
"your-deepseek-api-key",
"your-dingtalk-client-id",

View File

@ -234,6 +234,12 @@ class SandboxConfig(BaseModel):
ready_timeout: Create/readiness deadline in seconds (default: 30).
sandbox_timeout: Remote lifetime in seconds (default: 14400); 0 requires
explicit provider cleanup.
BoxdSandboxProvider specific options:
boxd_api_key, org, region: boxd credentials and placement
(``BOXD_API_KEY`` is the fallback for ``boxd_api_key``).
vcpu, memory, from_snapshot, auto_suspend_timeout, ready_timeout,
name_prefix, labels: machine shape, readiness deadline and naming.
"""
use: str = Field(
@ -365,7 +371,7 @@ class SandboxConfig(BaseModel):
"`hard_timeout`; the frozen legacy `all-in-one-sandbox:latest` image "
"only gets the bounded host-side request. `bash_command_timeout` is "
"forwarded to each provider's command API. LocalSandbox, AioSandbox, "
"OpenSandbox, E2B, BoxLite, and Tenki apply it to bash commands; "
"OpenSandbox, E2B, BoxLite, Tenki, and boxd apply it to bash commands; "
"providers without per-command timeout support retain their own "
"defaults."
),

View File

@ -98,6 +98,9 @@ tenki = ["tenki>=1.4.0"]
# OpenSandbox remote sandbox provider (deerflow.community.opensandbox). The
# sync SDK is loaded only when this provider is selected.
opensandbox = ["opensandbox>=0.1.15,<0.2.0"]
# boxd machine sandbox provider (deerflow.community.boxd_sandbox). The SDK is
# loaded only when this provider is selected.
boxd = ["boxd>=0.3.2"]
# Agent observability (Monocle). Optional so a default install stays free of the
# OpenTelemetry stack; only pulled in when MONOCLE_TRACING is used.
monocle = ["monocle_apptrace>=0.8.8"]

View File

@ -35,6 +35,7 @@ EXPECTED_GUIDANCE_PATHS = {
"backend/packages/harness/deerflow/community/boxlite/AGENTS.md",
"backend/packages/harness/deerflow/community/browser_automation/AGENTS.md",
"backend/packages/harness/deerflow/community/opensandbox/AGENTS.md",
"backend/packages/harness/deerflow/community/boxd_sandbox/AGENTS.md",
"backend/packages/harness/deerflow/config/AGENTS.md",
"backend/packages/harness/deerflow/extensions/AGENTS.md",
"backend/packages/harness/deerflow/runtime/AGENTS.md",

View File

@ -0,0 +1,196 @@
"""Live tests for the boxd sandbox provider against real boxd machines.
Skipped unless explicitly enabled; they create and delete real machines:
DEER_FLOW_RUN_LIVE_TESTS=1 BOXD_API_KEY=... PYTHONPATH=. uv run pytest tests/test_boxd_sandbox_live.py -v -s
Optional: ``BOXD_ORG``, ``BOXD_TEST_REGION`` (fra or nyc), ``BOXD_TEST_NAME_PREFIX``
(default ``deer-flow-test-``) and ``BOXD_TEST_LABELS`` (``key=value,key=value``).
The module shares one provider; its shutdown deletes every machine the tests
created and checks that each one is gone.
"""
from __future__ import annotations
import hashlib
import os
import time
import types
from pathlib import Path
import pytest
pytestmark = pytest.mark.live
_skip_reason = None
if os.environ.get("CI"):
_skip_reason = "Live tests skipped in CI"
elif os.environ.get("DEER_FLOW_RUN_LIVE_TESTS") != "1":
_skip_reason = "Set DEER_FLOW_RUN_LIVE_TESTS=1 to run live tests with real external APIs"
elif not os.environ.get("BOXD_API_KEY"):
_skip_reason = "Set BOXD_API_KEY to run the boxd live tests"
if _skip_reason:
pytest.skip(_skip_reason, allow_module_level=True)
pytest.importorskip("boxd")
from deerflow.community.boxd_sandbox import BoxdSandboxProvider # noqa: E402
from deerflow.config.paths import Paths # noqa: E402
THREAD = "live-thread"
USER = "live-user"
def _labels() -> dict[str, str]:
raw = os.environ.get("BOXD_TEST_LABELS", "")
return dict(item.split("=", 1) for item in raw.split(",") if "=" in item)
@pytest.fixture(scope="module")
def live(tmp_path_factory: pytest.TempPathFactory):
paths = Paths(base_dir=tmp_path_factory.mktemp("deer-flow-home"))
sandbox_options = {
"use": "deerflow.community.boxd_sandbox:BoxdSandboxProvider",
"org": os.environ.get("BOXD_ORG"),
"region": os.environ.get("BOXD_TEST_REGION", "fra"),
"vcpu": 1,
"name_prefix": os.environ.get("BOXD_TEST_NAME_PREFIX", "deer-flow-test-"),
"labels": _labels(),
"idle_timeout": 0,
"environment": {"DEERFLOW_LIVE": "static"},
}
config = types.SimpleNamespace(sandbox=types.SimpleNamespace(**sandbox_options), skills=types.SimpleNamespace(container_path="/mnt/skills"))
patch = pytest.MonkeyPatch()
patch.setattr("deerflow.community.boxd_sandbox.boxd_sandbox_provider.get_app_config", lambda: config)
patch.setattr("deerflow.config.paths.get_paths", lambda: paths)
def no_shared_skills(*args, **kwargs):
raise RuntimeError("shared skills are not part of this test")
patch.setattr("deerflow.skills.storage.get_or_new_user_skill_storage", no_shared_skills)
uploads = paths.sandbox_uploads_dir(THREAD, user_id=USER)
uploads.mkdir(parents=True)
(uploads / "input.csv").write_text("a,b\n1,2\n")
provider = BoxdSandboxProvider()
started = time.monotonic()
sandbox_id = provider.acquire(THREAD, user_id=USER)
sandbox = provider.get(sandbox_id)
print(f"\nacquire (create, ready, bootstrap, restore): {time.monotonic() - started:.1f}s, machine {sandbox.machine_id}")
machine_ids = {sandbox.machine_id}
yield types.SimpleNamespace(provider=provider, paths=paths, sandbox_id=sandbox_id, machine_ids=machine_ids)
client = provider._get_client()
provider.shutdown()
for machine_id in machine_ids:
with pytest.raises(Exception) as excinfo:
client.machines.get(machine_id, region=sandbox_options["region"])
assert type(excinfo.value).__name__ == "NotFoundError"
print(f"machine {machine_id} deleted by shutdown")
patch.undo()
def _sandbox(live):
return live.provider.get(live.sandbox_id)
def test_machine_is_isolated(live) -> None:
client = live.provider._get_client()
machine = client.machines.get(_sandbox(live).machine_id)
assert machine.networking.isolated is True
assert machine.labels["deer-flow-scope"] == live.sandbox_id
def test_restored_upload_is_visible(live) -> None:
assert _sandbox(live).read_file("/mnt/user-data/uploads/input.csv") == "a,b\n1,2\n"
def test_exec_env_streams_and_exit_code(live) -> None:
sandbox = _sandbox(live)
started = time.monotonic()
output = sandbox.execute_command('echo "$DEERFLOW_LIVE $REQUEST_TOKEN"; echo warn >&2; exit 3', env={"REQUEST_TOKEN": "per-call"})
print(f"exec: {time.monotonic() - started:.2f}s")
assert output == "static per-call\n\nwarn\n\nExit Code: 3"
assert sandbox.execute_command("cat") == "(no output)" # stdin is closed
def test_timeout_keeps_partial_output(live) -> None:
output = _sandbox(live).execute_command("echo before; sleep 30; echo after", timeout=2)
assert output.startswith("before") and "after" not in output
assert output.endswith("Exit Code: 124")
def test_background_process_survives_between_calls(live) -> None:
sandbox = _sandbox(live)
sandbox.execute_command("nohup sleep 300 >/dev/null 2>&1 & echo $! > /mnt/user-data/workspace/bg.pid")
output = sandbox.execute_command("kill -0 $(cat /mnt/user-data/workspace/bg.pid) && echo alive")
assert output.strip() == "alive"
def test_five_mib_round_trip(live) -> None:
sandbox = _sandbox(live)
data = os.urandom(5 * 1024 * 1024)
started = time.monotonic()
sandbox.update_file("/mnt/user-data/workspace/blob.bin", data)
got = sandbox.download_file("/mnt/user-data/workspace/blob.bin")
print(f"5 MiB up + down: {time.monotonic() - started:.2f}s")
assert hashlib.sha256(got).hexdigest() == hashlib.sha256(data).hexdigest()
with pytest.raises(OSError):
sandbox.download_file("/mnt/user-data/workspace/blob.bin", max_bytes=1024)
def test_write_append_and_read(live) -> None:
sandbox = _sandbox(live)
sandbox.write_file("/mnt/user-data/workspace/new/appended.txt", "first\n", append=True)
sandbox.write_file("/mnt/user-data/workspace/new/appended.txt", "second\n", append=True)
assert sandbox.read_file("/mnt/user-data/workspace/new/appended.txt") == "first\nsecond\n"
assert sandbox.read_file("/mnt/user-data/workspace/new/appended.txt", start_line=2, end_line=2) == "second"
assert sandbox.read_file("/mnt/user-data/workspace/missing.txt").startswith("Error:")
def test_search_tools(live) -> None:
sandbox = _sandbox(live)
sandbox.write_file("/mnt/user-data/workspace/src/report:2026.txt", "alpha\nneedle here\n")
sandbox.write_file("/mnt/user-data/workspace/src/main.py", "needle = 1\n")
matches, _ = sandbox.grep("/mnt/user-data/workspace/src", "needle")
assert sorted((Path(m.path).name, m.line_number) for m in matches) == [("main.py", 1), ("report:2026.txt", 2)]
found, _ = sandbox.glob("/mnt/user-data/workspace", "**/*.py")
assert found == ["/mnt/user-data/workspace/src/main.py"]
assert "/mnt/user-data/workspace/src/main.py" in sandbox.list_dir("/mnt/user-data/workspace")
def test_release_mirrors_outputs_and_reclaims_same_machine(live) -> None:
sandbox = _sandbox(live)
machine_id = sandbox.machine_id
sandbox.execute_command("mkdir -p /mnt/user-data/outputs/charts && echo '# Report' > /mnt/user-data/outputs/report.md && printf png > /mnt/user-data/outputs/charts/a.png")
live.provider.release(live.sandbox_id)
outputs = live.paths.sandbox_outputs_dir(THREAD, user_id=USER)
assert (outputs / "report.md").read_text() == "# Report\n"
assert (outputs / "charts" / "a.png").read_bytes() == b"png"
started = time.monotonic()
assert live.provider.acquire(THREAD, user_id=USER) == live.sandbox_id
print(f"warm reclaim: {time.monotonic() - started:.2f}s")
assert _sandbox(live).machine_id == machine_id
def test_stopped_machine_is_started_on_next_call(live) -> None:
sandbox = _sandbox(live)
client = live.provider._get_client()
client.machines.stop(sandbox.machine_id)
started = time.monotonic()
# A command that meets a stopped machine starts it but is not repeated.
assert "run it again" in sandbox.execute_command("echo once >> /mnt/user-data/workspace/ran.txt")
print(f"start after stop: {time.monotonic() - started:.1f}s")
assert sandbox.execute_command("cat /mnt/user-data/workspace/new/appended.txt") == "first\nsecond\n"
assert sandbox.read_file("/mnt/user-data/workspace/ran.txt").startswith("Error:")
client.machines.stop(sandbox.machine_id)
# A file read is repeated once the machine runs again.
assert sandbox.read_file("/mnt/user-data/workspace/new/appended.txt") == "first\nsecond\n"
def test_wrong_api_key_fails_clearly(live) -> None:
from boxd import Boxd
with pytest.raises(Exception) as excinfo:
Boxd(api_key="not-a-real-key", region=os.environ.get("BOXD_TEST_REGION", "fra")).machines.get(_sandbox(live).machine_id)
assert type(excinfo.value).__name__ == "AuthenticationError"

File diff suppressed because it is too large Load Diff

View File

@ -26,3 +26,15 @@ def test_opensandbox_is_optional_harness_dependency() -> None:
assert not any(dep.startswith("opensandbox") for dep in core_dependencies)
assert optional_dependencies["opensandbox"] == ["opensandbox>=0.1.15,<0.2.0"]
def test_boxd_is_optional_harness_dependency() -> None:
"""The boxd SDK should be installed only when its provider is selected."""
pyproject_path = Path(__file__).resolve().parents[1] / "packages" / "harness" / "pyproject.toml"
pyproject = tomllib.loads(pyproject_path.read_text(encoding="utf-8"))
core_dependencies = pyproject["project"]["dependencies"]
optional_dependencies = pyproject["project"]["optional-dependencies"]
assert not any(dep.startswith("boxd") for dep in core_dependencies)
assert optional_dependencies["boxd"] == ["boxd>=0.3.2"]

22
backend/uv.lock generated
View File

@ -529,6 +529,22 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/95/c1/84fc6811122f54b20de2e5afb312ee07a3a47a328755587d1e505475239b/blockbuster-1.5.26-py3-none-any.whl", hash = "sha256:f8e53fb2dd4b6c6ec2f04907ddbd063ca7cd1ef587d24448ef4e50e81e3a79bb", size = 13226, upload-time = "2025-12-05T10:43:48.778Z" },
]
[[package]]
name = "boxd"
version = "0.3.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "grpcio" },
{ name = "httpx" },
{ name = "protobuf" },
{ name = "pydantic" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/1d/37/ad184af822d5b1c8171330986aef06e521d0e3dd0c0c47b3c37f5a1a85b7/boxd-0.3.2.tar.gz", hash = "sha256:03aa9471d1ddd663a379655b19bc7a1f20b6199a5aadfb51238889552450bd75", size = 308049, upload-time = "2026-11-13T01:14:49.059Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/7e/f0/badded7811ce0c26841e0a6145c98413189a138dc2edca2eb61f9abf5787/boxd-0.3.2-py3-none-any.whl", hash = "sha256:b78c9609ea8bbad942a27447718902744ce8904bed2185c79283eea2bfd72c42", size = 199336, upload-time = "2026-11-13T01:14:47.777Z" },
]
[[package]]
name = "boxlite"
version = "0.13.1"
@ -1030,6 +1046,9 @@ dependencies = [
]
[package.optional-dependencies]
boxd = [
{ name = "boxd" },
]
boxlite = [
{ name = "boxlite" },
]
@ -1075,6 +1094,7 @@ requires-dist = [
{ name = "alembic", specifier = ">=1.13" },
{ name = "asyncpg", marker = "extra == 'postgres'", specifier = ">=0.29" },
{ name = "beautifulsoup4", specifier = ">=4.12" },
{ name = "boxd", marker = "extra == 'boxd'", specifier = ">=0.3.2" },
{ name = "boxlite", marker = "extra == 'boxlite'", specifier = ">=0.9.7" },
{ name = "croniter", specifier = ">=6.0.0" },
{ name = "cryptography", specifier = ">=50.0.0" },
@ -1128,7 +1148,7 @@ requires-dist = [
{ name = "tiktoken", specifier = ">=0.8.0" },
{ name = "tzdata", specifier = ">=2025.1" },
]
provides-extras = ["tui", "groundroute", "ollama", "postgres", "redis", "pymupdf", "boxlite", "tenki", "opensandbox", "monocle", "browser", "memory-zh"]
provides-extras = ["tui", "groundroute", "ollama", "postgres", "redis", "pymupdf", "boxlite", "tenki", "opensandbox", "boxd", "monocle", "browser", "memory-zh"]
[[package]]
name = "defusedxml"

View File

@ -2034,6 +2034,25 @@ sandbox:
# # environment: # create-time and per-command defaults
# # PYTHONUNBUFFERED: "1"
# Option 7: boxd machine Sandbox
# Runs each sandbox as an isolated boxd machine (an Ubuntu microVM). Released
# machines stay in an in-process warm pool and are reclaimed only by the same
# user/thread. Requires the optional SDK: pip install "deerflow-harness[boxd]".
# sandbox:
# use: deerflow.community.boxd_sandbox:BoxdSandboxProvider
# # boxd_api_key: $BOXD_API_KEY # falls back to the BOXD_API_KEY env var
# # org: my-org # optional; the key's own org when omitted
# # region: fra # optional; fra or nyc
# # vcpu: 2 # optional; 1, 2, 4 or 8 (memory 4G, 8G, 16G or 32G follows)
# # from_snapshot: my-snapshot # optional; start machines from a prepared snapshot
# # auto_suspend_timeout: 300 # optional; idle seconds before a machine goes to standby
# # ready_timeout: 120 # seconds to wait for a new machine to boot
# # name_prefix: deer-flow- # machine names are <prefix><scope>-<random suffix>
# # replicas: 3 # active + warm cap per gateway process
# # idle_timeout: 600 # warm seconds before the machine is deleted; 0 disables
# # environment: # merged into every command's environment
# # PYTHONUNBUFFERED: "1"
# ============================================================================
# Subagents Configuration
# ============================================================================

View File

@ -84,6 +84,12 @@ Outputs with case- or Unicode-equivalent file or directory names are skipped
before copying, and aliases of existing local paths are rejected, preserving
previous complete copies.
The optional [boxd provider](../../backend/packages/harness/deerflow/community/boxd_sandbox/README.md)
runs each user/thread scope in an isolated boxd machine. A new machine receives
the thread's files and skills before it is handed out, and release copies
generated outputs back to thread storage with the same alias and path checks.
See its [configuration](../../backend/docs/CONFIGURATION.md#sandbox).
The [sandbox benchmark](../../backend/scripts/benchmark/sandbox/README.md#opensandbox)
also supports OpenSandbox cold acquisition, warm reuse and concurrent scopes,
using the configured service connection and reporting unconfirmed cleanup as a