mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-26 07:57:57 +00:00
feat(sandbox): warm pool for boxlite (#3951)
* feat: add boxlite SDK as harness dependency * test: add fake SimpleBox fixtures for BoxLite warm pool tests * feat: add deterministic sandbox_id and warm pool fields to BoxliteProvider * feat: pass deterministic sandbox_id to SimpleBox name * feat: warm pool lifecycle — park on release, reclaim on acquire - release(): parks VMs in _warm_pool with timestamp instead of closing - _reclaim_warm_pool(): health checks warm boxes via echo ok - acquire(): tries warm pool reclaim before creating new boxes - Deterministic sandbox_id ensures thread isolation * feat(boxlite): idle reaper, replica enforcement, warm-pool shutdown/reset - Task 6: idle reaper daemon thread destroys expired warm-pool boxes - Task 7: replica enforcement evicts oldest warm-pool box when at capacity - Task 8: shutdown() stops idle checker first, destroys all boxes (active+warm); reset() clears warm pool Tests: 17 passed (4 new: idle reaper, replica enforcement, shutdown, reset) * fix(boxlite): harden warm pool lifecycle races * docs(boxlite): document warm pool configuration * fix(sandbox): log warning when evicting oldest warm box is failed Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(boxlite): stop provider lifecycle on reset * fix(boxlite): make runtime an optional dependency * test(boxlite): remove unused warm pool lookup * docs(boxlite): clarify optional runtime support * refactor: extract shared WarmPoolLifecycleMixin for sandbox warm-pool lifecycle Introduce deerflow.community.warm_pool_lifecycle.WarmPoolLifecycleMixin owning idle-checker loop, warm-pool expiry, oldest-warm eviction, replica counting, and soft-cap logging. Move AioSandboxProvider and BoxliteProvider onto the mixin; keep AIO active-idle cleanup local and delegate only warm-pool expiry to the shared helper. BoxliteProvider also gains: - Prefixed box names (deer-flow-boxlite-*) for startup orphan reconciliation - _reconcile_orphans() adopting surviving boxes from a prior process - Pinned timeout forwarding: command timeout now bounds both BoxLite SDK exec(timeout=...) and the loop bridge .result(timeout) - reset() reworked as lightweight registry clear (boxes -> warm pool, no close, no idle-reaper stop, no loop close) so reset_sandbox_provider() config switches are safe; shutdown() remains the teardown path Backward-compatible: AIO DEFAULT_IDLE_TIMEOUT/DEFAULT_REPLICAS/ IDLE_CHECK_INTERVAL stay importable; Boxlite IDLE_CHECK_INTERVAL stays monkeypatchable. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Vanzeren
GitHub
parent
7a985f441c
commit
67c8ade375
@ -343,6 +343,12 @@ Proxied through nginx: `/api/langgraph/*` → Gateway LangGraph-compatible runti
|
||||
**Implementations**:
|
||||
- `LocalSandboxProvider` - Local filesystem execution. `acquire(thread_id)` returns a per-thread `LocalSandbox` (id `local:{thread_id}`) whose `path_mappings` resolve `/mnt/user-data/{workspace,uploads,outputs}` and `/mnt/acp-workspace` to that thread's host directories, so the public `Sandbox` API honours the `/mnt/user-data` contract uniformly with AIO. `acquire()` / `acquire(None)` keeps the legacy generic singleton (id `local`) for callers without a thread context. Per-thread sandboxes are held in an LRU cache (default 256 entries) guarded by a `threading.Lock`.
|
||||
- `AioSandboxProvider` (`packages/harness/deerflow/community/`) - Docker-based isolation. Active-cache and warm-pool entries are checked with the backend during acquire/reuse; definitively dead containers are dropped from all in-process maps so the thread can discover or create a fresh sandbox instead of reusing a stale client. Backend health-check failures are treated as unknown, not dead; local discovery likewise treats an unverifiable container as not adoptable and falls through to create rather than failing acquire. `get()` remains an in-memory lookup for event-loop-safe tool paths.
|
||||
- `BoxliteProvider` (`packages/harness/deerflow/community/boxlite/`) - BoxLite micro-VM isolation. The `boxlite` runtime is optional (`deerflow-harness[boxlite]`) and lazy-imported only when this provider is selected. The provider owns one private asyncio event loop on a daemon thread because BoxLite handles are loop-affine; sync `Sandbox` calls marshal onto that loop with `run_coroutine_threadsafe`.
|
||||
Boxes are named deterministically from `user_id:thread_id`, released into an in-process warm pool after each agent turn, and reclaimed only by the same user/thread. Warm-pool health checks use a short explicit timeout and forward that timeout through both BoxLite `exec(timeout=...)` and the private-loop `.result(timeout)` bridge so a hung VM cannot pin the per-thread acquire lock indefinitely.
|
||||
`sandbox.replicas` caps active + warm VMs per gateway process; if capacity is exhausted, only warm-pool VMs are evicted. `sandbox.idle_timeout` stops idle warm VMs after the configured seconds. `reset()` is intentionally a lightweight registry clear for `reset_sandbox_provider()` and does not close boxes, stop the idle reaper, or close the private loop; full teardown remains `shutdown()`.
|
||||
|
||||
|
||||
**Shared warm-pool lifecycle:** community sandbox providers that keep released sandboxes alive for fast reuse share `deerflow.community.warm_pool_lifecycle.WarmPoolLifecycleMixin`. The mixin owns the common `DEFAULT_IDLE_TIMEOUT=600`, `IDLE_CHECK_INTERVAL=60`, `DEFAULT_REPLICAS=3`, idle-checker loop, warm-pool expiry, oldest-warm eviction, replica counting, and soft-cap logging. Providers remain responsible for their own active registries, creation/discovery, health checks, and destroy hook (`_destroy_warm_entry`): AIO destroys `SandboxInfo` through its backend; Boxlite closes loop-affine `BoxliteBox` handles. AIO keeps active-idle cleanup outside the mixin and delegates only warm-pool expiry to the shared helper.
|
||||
|
||||
**Virtual Path System**:
|
||||
- Agent sees: `/mnt/user-data/{workspace,uploads,outputs}`, `/mnt/skills`
|
||||
|
||||
@ -342,6 +342,32 @@ sandbox:
|
||||
use: deerflow.community.aio_sandbox:AioSandboxProvider # Docker-based sandbox
|
||||
```
|
||||
|
||||
**BoxLite micro-VM Sandbox** (runs sandbox code in daemonless OCI micro-VMs):
|
||||
```yaml
|
||||
sandbox:
|
||||
use: deerflow.community.boxlite:BoxliteProvider
|
||||
image: python:3.12-slim
|
||||
memory_mib: 1024 # optional per-box memory cap
|
||||
cpus: 2 # optional per-box vCPUs
|
||||
replicas: 3 # max active + warm VMs per gateway process
|
||||
idle_timeout: 600 # warm VM idle seconds before stop; 0 disables idle reaping
|
||||
environment:
|
||||
PYTHONUNBUFFERED: "1"
|
||||
```
|
||||
|
||||
Install the optional runtime before selecting this provider:
|
||||
|
||||
```bash
|
||||
pip install "deerflow-harness[boxlite]"
|
||||
```
|
||||
|
||||
BoxLite boxes are named from the effective `(user_id, thread_id)` scope and are
|
||||
released into an in-process warm pool after each turn. The same user/thread can
|
||||
reclaim its warm VM on the next acquire; different threads cannot share a VM.
|
||||
`replicas` caps active plus warm VMs. When the cap is reached only warm VMs are
|
||||
evicted; active VMs continue and the provider may temporarily exceed the cap if
|
||||
all boxes are active.
|
||||
|
||||
**Docker Execution with Kubernetes** (runs sandbox code in Kubernetes pods via provisioner service):
|
||||
|
||||
This mode runs each sandbox in an isolated Kubernetes Pod on your **host machine's cluster**. Requires Docker Desktop K8s, OrbStack, or similar local K8s setup.
|
||||
|
||||
@ -27,6 +27,14 @@ except ImportError: # pragma: no cover - Windows fallback
|
||||
fcntl = None # type: ignore[assignment]
|
||||
import msvcrt
|
||||
|
||||
from deerflow.community.warm_pool_lifecycle import (
|
||||
DEFAULT_IDLE_TIMEOUT,
|
||||
DEFAULT_REPLICAS,
|
||||
WarmPoolLifecycleMixin,
|
||||
)
|
||||
from deerflow.community.warm_pool_lifecycle import (
|
||||
IDLE_CHECK_INTERVAL as _SHARED_IDLE_CHECK_INTERVAL,
|
||||
)
|
||||
from deerflow.config import get_app_config
|
||||
from deerflow.config.paths import VIRTUAL_PATH_PREFIX, get_paths
|
||||
from deerflow.runtime.user_context import get_effective_user_id
|
||||
@ -45,9 +53,7 @@ logger = logging.getLogger(__name__)
|
||||
DEFAULT_IMAGE = "enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest"
|
||||
DEFAULT_PORT = 8080
|
||||
DEFAULT_CONTAINER_PREFIX = "deer-flow-sandbox"
|
||||
DEFAULT_IDLE_TIMEOUT = 600 # 10 minutes in seconds
|
||||
DEFAULT_REPLICAS = 3 # Maximum concurrent sandbox containers
|
||||
IDLE_CHECK_INTERVAL = 60 # Check every 60 seconds
|
||||
IDLE_CHECK_INTERVAL = _SHARED_IDLE_CHECK_INTERVAL
|
||||
THREAD_LOCK_EXECUTOR_WORKERS = min(32, (os.cpu_count() or 1) + 4)
|
||||
_THREAD_LOCK_EXECUTOR = ThreadPoolExecutor(max_workers=THREAD_LOCK_EXECUTOR_WORKERS, thread_name_prefix="sandbox-lock-wait")
|
||||
atexit.register(_THREAD_LOCK_EXECUTOR.shutdown, wait=False, cancel_futures=True)
|
||||
@ -105,7 +111,7 @@ def _release_cancelled_lock_acquire(lock: threading.Lock, task: asyncio.Future[b
|
||||
lock.release()
|
||||
|
||||
|
||||
class AioSandboxProvider(SandboxProvider):
|
||||
class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
|
||||
"""Sandbox provider that manages containers running the AIO sandbox.
|
||||
|
||||
Architecture:
|
||||
@ -352,28 +358,13 @@ class AioSandboxProvider(SandboxProvider):
|
||||
|
||||
# ── Idle timeout management ──────────────────────────────────────────
|
||||
|
||||
def _start_idle_checker(self) -> None:
|
||||
"""Start the background thread that checks for idle sandboxes."""
|
||||
self._idle_checker_thread = threading.Thread(
|
||||
target=self._idle_checker_loop,
|
||||
name="sandbox-idle-checker",
|
||||
daemon=True,
|
||||
)
|
||||
self._idle_checker_thread.start()
|
||||
logger.info(f"Started idle checker thread (timeout: {self._config.get('idle_timeout', DEFAULT_IDLE_TIMEOUT)}s)")
|
||||
|
||||
def _idle_checker_loop(self) -> None:
|
||||
idle_timeout = self._config.get("idle_timeout", DEFAULT_IDLE_TIMEOUT)
|
||||
while not self._idle_checker_stop.wait(timeout=IDLE_CHECK_INTERVAL):
|
||||
try:
|
||||
self._cleanup_idle_sandboxes(idle_timeout)
|
||||
except Exception as e:
|
||||
logger.error(f"Error in idle checker loop: {e}")
|
||||
def _cleanup_idle_resources(self, idle_timeout: float) -> None:
|
||||
"""Clean AIO resources idle longer than ``idle_timeout`` seconds."""
|
||||
self._cleanup_idle_sandboxes(idle_timeout)
|
||||
|
||||
def _cleanup_idle_sandboxes(self, idle_timeout: float) -> None:
|
||||
current_time = time.time()
|
||||
active_to_destroy = []
|
||||
warm_to_destroy: list[tuple[str, SandboxInfo]] = []
|
||||
|
||||
with self._lock:
|
||||
# Active sandboxes: tracked via _last_activity
|
||||
@ -383,14 +374,6 @@ class AioSandboxProvider(SandboxProvider):
|
||||
active_to_destroy.append(sandbox_id)
|
||||
logger.info(f"Sandbox {sandbox_id} idle for {idle_duration:.1f}s, marking for destroy")
|
||||
|
||||
# Warm pool: tracked via release_timestamp stored in _warm_pool
|
||||
for sandbox_id, (info, release_ts) in list(self._warm_pool.items()):
|
||||
warm_duration = current_time - release_ts
|
||||
if warm_duration > idle_timeout:
|
||||
warm_to_destroy.append((sandbox_id, info))
|
||||
del self._warm_pool[sandbox_id]
|
||||
logger.info(f"Warm-pool sandbox {sandbox_id} idle for {warm_duration:.1f}s, marking for destroy")
|
||||
|
||||
# Destroy active sandboxes (re-verify still idle before acting)
|
||||
for sandbox_id in active_to_destroy:
|
||||
try:
|
||||
@ -412,13 +395,7 @@ class AioSandboxProvider(SandboxProvider):
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to destroy idle sandbox {sandbox_id}: {e}")
|
||||
|
||||
# Destroy warm-pool sandboxes (already removed from _warm_pool under lock above)
|
||||
for sandbox_id, info in warm_to_destroy:
|
||||
try:
|
||||
self._backend.destroy(info)
|
||||
logger.info(f"Destroyed idle warm-pool sandbox {sandbox_id}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to destroy idle warm-pool sandbox {sandbox_id}: {e}")
|
||||
self._reap_expired_warm(idle_timeout)
|
||||
|
||||
# ── Signal handling ──────────────────────────────────────────────────
|
||||
|
||||
@ -650,23 +627,29 @@ class AioSandboxProvider(SandboxProvider):
|
||||
|
||||
logger.warning(f"Dropped unhealthy sandbox {sandbox_id}: {reason}")
|
||||
|
||||
def _replica_count(self) -> tuple[int, int]:
|
||||
"""Return configured replicas and currently tracked sandbox count."""
|
||||
replicas = self._config.get("replicas", DEFAULT_REPLICAS)
|
||||
with self._lock:
|
||||
total = len(self._sandboxes) + len(self._warm_pool)
|
||||
return replicas, total
|
||||
def _active_count_locked(self) -> int:
|
||||
"""Return active AIO sandbox count while ``_lock`` is held."""
|
||||
return len(self._sandboxes)
|
||||
|
||||
def _log_replicas_soft_cap(self, replicas: int, sandbox_id: str, evicted: str | None) -> None:
|
||||
"""Log the result of enforcing the warm-pool replica budget."""
|
||||
if evicted:
|
||||
logger.info(f"Evicted warm-pool sandbox {evicted} to stay within replicas={replicas}")
|
||||
def _destroy_warm_entry(self, sandbox_id: str, entry: SandboxInfo, *, reason: str) -> None:
|
||||
"""Destroy a warm-pool sandbox using AIO-specific backend logging."""
|
||||
try:
|
||||
self._backend.destroy(entry)
|
||||
except Exception as e:
|
||||
if reason == "idle_timeout":
|
||||
logger.error(f"Failed to destroy idle warm-pool sandbox {sandbox_id}: {e}")
|
||||
elif reason == "replica_enforcement":
|
||||
logger.error(f"Failed to destroy warm-pool sandbox {sandbox_id}: {e}")
|
||||
else:
|
||||
logger.error(f"Failed to destroy warm-pool sandbox {sandbox_id} for {reason}: {e}")
|
||||
return
|
||||
|
||||
# All slots are occupied by active sandboxes — proceed anyway and log.
|
||||
# The replicas limit is a soft cap; we never forcibly stop a container
|
||||
# that is actively serving a thread.
|
||||
logger.warning(f"All {replicas} replica slots are in active use; creating sandbox {sandbox_id} beyond the soft limit")
|
||||
if reason == "idle_timeout":
|
||||
logger.info(f"Destroyed idle warm-pool sandbox {sandbox_id}")
|
||||
elif reason == "replica_enforcement":
|
||||
logger.info(f"Destroyed warm-pool sandbox {sandbox_id}")
|
||||
else:
|
||||
logger.info(f"Destroyed warm-pool sandbox {sandbox_id} for {reason}")
|
||||
|
||||
# ── Core: acquire / get / release / shutdown ─────────────────────────
|
||||
|
||||
@ -822,26 +805,6 @@ class AioSandboxProvider(SandboxProvider):
|
||||
await asyncio.to_thread(_unlock_file, lock_file)
|
||||
await asyncio.to_thread(lock_file.close)
|
||||
|
||||
def _evict_oldest_warm(self) -> str | None:
|
||||
"""Destroy the oldest container in the warm pool to free capacity.
|
||||
|
||||
Returns:
|
||||
The evicted sandbox_id, or None if warm pool is empty.
|
||||
"""
|
||||
with self._lock:
|
||||
if not self._warm_pool:
|
||||
return None
|
||||
oldest_id = min(self._warm_pool, key=lambda sid: self._warm_pool[sid][1])
|
||||
info, _ = self._warm_pool.pop(oldest_id)
|
||||
|
||||
try:
|
||||
self._backend.destroy(info)
|
||||
logger.info(f"Destroyed warm-pool sandbox {oldest_id}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to destroy warm-pool sandbox {oldest_id}: {e}")
|
||||
return None
|
||||
return oldest_id
|
||||
|
||||
def _create_sandbox(self, thread_id: str | None, sandbox_id: str, *, user_id: str | None = None) -> str:
|
||||
"""Create a new sandbox via the backend.
|
||||
|
||||
@ -989,11 +952,7 @@ class AioSandboxProvider(SandboxProvider):
|
||||
warm_items = list(self._warm_pool.items())
|
||||
self._warm_pool.clear()
|
||||
|
||||
# Stop idle checker
|
||||
self._idle_checker_stop.set()
|
||||
if self._idle_checker_thread is not None and self._idle_checker_thread.is_alive():
|
||||
self._idle_checker_thread.join(timeout=5)
|
||||
logger.info("Stopped idle checker thread")
|
||||
self._stop_idle_checker()
|
||||
|
||||
logger.info(f"Shutting down {len(sandbox_ids)} active + {len(warm_items)} warm-pool sandbox(es)")
|
||||
|
||||
|
||||
@ -16,14 +16,24 @@ sandbox:
|
||||
image: python:3.12-slim # any OCI image, run unchanged (default: python:3.12-slim)
|
||||
memory_mib: 1024 # per-box memory cap (optional)
|
||||
cpus: 2 # per-box vCPUs (optional)
|
||||
replicas: 3 # active + warm VM cap per gateway process (default: 3)
|
||||
idle_timeout: 600 # warm VM idle seconds before stop; 0 disables reaping
|
||||
environment: # injected into every command
|
||||
PYTHONUNBUFFERED: "1"
|
||||
```
|
||||
|
||||
Install the optional runtime before selecting this provider:
|
||||
|
||||
```bash
|
||||
pip install boxlite # an optional `[boxlite]` extra + uv.lock update will follow once the approach lands
|
||||
pip install "deerflow-harness[boxlite]"
|
||||
```
|
||||
|
||||
The `boxlite` package is an optional DeerFlow harness extra, not part of the
|
||||
default install. It is also limited to the host platforms and architectures
|
||||
where BoxLite publishes wheels and can boot micro-VMs. Unsupported development
|
||||
hosts, such as Windows, should use another sandbox provider or run DeerFlow from
|
||||
a supported Linux/macOS environment.
|
||||
|
||||
**Host requirement:** BoxLite boots micro-VMs, so a Linux host needs KVM — i.e.
|
||||
nested virtualization when DeerFlow runs inside a cloud VM. macOS uses
|
||||
Hypervisor.framework. This is the main deployment constraint to weigh vs. the
|
||||
@ -34,10 +44,9 @@ container-based providers.
|
||||
DeerFlow's `Sandbox` contract is synchronous; BoxLite's SDK is async-native and
|
||||
its box handles are event-loop-affine. The provider owns **one** private asyncio
|
||||
loop on a daemon thread and marshals every coroutine onto it via
|
||||
`run_coroutine_threadsafe`. This keeps all operations on the loop the box was
|
||||
started on and is safe under DeerFlow's `asyncio.to_thread` worker pool — without
|
||||
using BoxLite's greenlet sync facade, which refuses to run inside an async
|
||||
context and is thread-affine.
|
||||
`run_coroutine_threadsafe`. BoxLite boxes are named deterministically from
|
||||
`user_id:thread_id`, released into an in-process warm pool after each agent turn,
|
||||
and reclaimed by the same thread on the next acquire.
|
||||
|
||||
| File | Role |
|
||||
| --- | --- |
|
||||
@ -57,8 +66,9 @@ inside the box and reuse `deerflow.sandbox.search`, mirroring `e2b_sandbox`:
|
||||
The provider creates `/mnt/user-data/{workspace,uploads,outputs}` and
|
||||
`/mnt/skills` on box start so those virtual paths resolve natively.
|
||||
|
||||
**Out of scope for this pass** (follow-ups): warm pooling, idle reaping, mount
|
||||
syncing, and remote/provisioner modes.
|
||||
Warm-pool capacity is governed by `sandbox.replicas` across active + warm VMs.
|
||||
`sandbox.idle_timeout` controls how long released warm VMs stay running; `0`
|
||||
disables idle reaping. Active boxes are never evicted to satisfy the cap.
|
||||
|
||||
## Status
|
||||
|
||||
|
||||
@ -17,13 +17,14 @@ Configuration example (``config.yaml``)::
|
||||
image: python:3.12-slim # any OCI image; runs unchanged
|
||||
memory_mib: 1024 # per-box memory cap (optional)
|
||||
cpus: 2 # per-box vCPUs (optional)
|
||||
replicas: 3 # active + warm VM cap per gateway process
|
||||
idle_timeout: 600 # warm VM idle seconds before stop; 0 disables
|
||||
environment: # injected into every command
|
||||
PYTHONUNBUFFERED: "1"
|
||||
|
||||
Install the runtime (an optional ``[boxlite]`` extra + lockfile update will
|
||||
follow once the approach lands)::
|
||||
Install the optional runtime before selecting this provider::
|
||||
|
||||
pip install boxlite
|
||||
pip install "deerflow-harness[boxlite]"
|
||||
|
||||
Host requirement: BoxLite boots micro-VMs, so a Linux host needs KVM (nested
|
||||
virtualization when DeerFlow itself runs inside a cloud VM); macOS uses
|
||||
|
||||
@ -29,7 +29,7 @@ from deerflow.sandbox.sandbox import Sandbox, _validate_extra_env
|
||||
from deerflow.sandbox.search import GrepMatch, path_matches, should_ignore_path, truncate_line
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Awaitable, Callable
|
||||
from collections.abc import Callable
|
||||
|
||||
from boxlite import SimpleBox
|
||||
|
||||
@ -61,7 +61,7 @@ class BoxliteBox(Sandbox):
|
||||
self,
|
||||
id: str,
|
||||
box: SimpleBox,
|
||||
run: Callable[[Awaitable[T]], T],
|
||||
run: Callable[..., T],
|
||||
*,
|
||||
default_env: dict[str, str] | None = None,
|
||||
) -> None:
|
||||
@ -124,6 +124,11 @@ class BoxliteBox(Sandbox):
|
||||
DeerFlow passes a bash command *string*; BoxLite's ``exec`` takes argv, so
|
||||
it runs through ``sh -lc``. Per-call ``env`` is layered over the static
|
||||
config environment and scoped to this command only.
|
||||
|
||||
*timeout* bounds both layers: BoxLite's SDK ``exec(timeout=...)`` handles
|
||||
command timeout inside the VM, and the event-loop bridge receives the
|
||||
same value so ``run_coroutine_threadsafe(...).result(timeout)`` cannot
|
||||
block the caller forever if the SDK future itself never resolves.
|
||||
"""
|
||||
_validate_extra_env(env) # POSIX env-var key rule; raises ValueError on a bad key
|
||||
merged_env = {**self._default_env, **(env or {})} or None
|
||||
@ -132,7 +137,7 @@ class BoxliteBox(Sandbox):
|
||||
return "Error: sandbox has been closed"
|
||||
box = self._box
|
||||
try:
|
||||
result = self._run(box.exec("sh", "-lc", command, env=merged_env, timeout=timeout))
|
||||
result = self._run(box.exec("sh", "-lc", command, env=merged_env, timeout=timeout), timeout=timeout)
|
||||
except Exception as e:
|
||||
logger.error("Failed to execute command in BoxLite box %s: %s", self.id, e)
|
||||
return f"Error: {e}"
|
||||
|
||||
@ -8,15 +8,18 @@ Config is read off :class:`SandboxConfig` (``extra="allow"``), so BoxLite keys
|
||||
may appear under ``sandbox:`` in ``config.yaml`` even though they are not declared
|
||||
on the model — see this package's ``__init__`` docstring for the full set. The
|
||||
provider creates one micro-VM per ``(user, thread)`` and reuses it within the
|
||||
process; warm pooling, idle reaping and remote modes are out of scope for now.
|
||||
process.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import atexit
|
||||
import hashlib
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from collections.abc import Awaitable
|
||||
from typing import TYPE_CHECKING, Any, TypeVar
|
||||
|
||||
@ -26,6 +29,7 @@ from deerflow.constants import DEFAULT_SKILLS_CONTAINER_PATH
|
||||
from deerflow.sandbox.sandbox import Sandbox
|
||||
from deerflow.sandbox.sandbox_provider import SandboxProvider
|
||||
|
||||
from ..warm_pool_lifecycle import WarmPoolLifecycleMixin
|
||||
from .box import BoxliteBox
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@ -36,6 +40,7 @@ logger = logging.getLogger(__name__)
|
||||
T = TypeVar("T")
|
||||
|
||||
DEFAULT_IMAGE = "python:3.12-slim"
|
||||
_BOX_NAME_PREFIX = "deer-flow-boxlite-"
|
||||
# DeerFlow's virtual prefixes, materialised on the box rootfs at start so the
|
||||
# Sandbox file APIs (which address /mnt/user-data/...) resolve natively.
|
||||
_VIRTUAL_DIRS = (
|
||||
@ -55,10 +60,19 @@ def _import_simplebox() -> type[SimpleBox]:
|
||||
try:
|
||||
from boxlite import SimpleBox
|
||||
except ImportError as e: # pragma: no cover - depends on the optional dependency
|
||||
raise ImportError("BoxliteProvider requires the 'boxlite' package. Install it with: pip install boxlite.") from e
|
||||
raise ImportError("BoxliteProvider requires the optional 'boxlite' dependency. Install it with: pip install 'deerflow-harness[boxlite]' or pip install boxlite.") from e
|
||||
return SimpleBox
|
||||
|
||||
|
||||
def _import_sync_boxlite_runtime():
|
||||
"""Import BoxLite's sync runtime lazily for startup reconciliation."""
|
||||
try:
|
||||
from boxlite import SyncBoxlite
|
||||
except ImportError as e: # pragma: no cover - depends on the optional dependency
|
||||
raise ImportError("BoxliteProvider requires the optional 'boxlite' dependency. Install it with: pip install 'deerflow-harness[boxlite]' or pip install boxlite.") from e
|
||||
return SyncBoxlite
|
||||
|
||||
|
||||
class _EventLoopThread:
|
||||
"""A private asyncio event loop running on a dedicated daemon thread.
|
||||
|
||||
@ -71,34 +85,106 @@ class _EventLoopThread:
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._loop = asyncio.new_event_loop()
|
||||
self._thread = threading.Thread(target=self._loop.run_forever, name="boxlite-loop", daemon=True)
|
||||
self._loop: asyncio.AbstractEventLoop | None = None
|
||||
self._ready = threading.Event()
|
||||
self._thread = threading.Thread(target=self._run_forever, name="boxlite-loop", daemon=True)
|
||||
self._thread.start()
|
||||
self._ready.wait(timeout=5)
|
||||
|
||||
def _run_forever(self) -> None:
|
||||
self._loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(self._loop)
|
||||
self._loop.call_soon(self._ready.set)
|
||||
self._loop.run_forever()
|
||||
|
||||
def run(self, coro: Awaitable[T], *, timeout: float | None = None) -> T:
|
||||
if self._loop is None:
|
||||
raise RuntimeError("BoxLite event loop is not ready")
|
||||
return asyncio.run_coroutine_threadsafe(coro, self._loop).result(timeout)
|
||||
|
||||
def close(self) -> None:
|
||||
if self._loop is None:
|
||||
return
|
||||
self._loop.call_soon_threadsafe(self._loop.stop)
|
||||
wake = getattr(self._loop, "_write_to_self", None)
|
||||
if wake is not None:
|
||||
wake()
|
||||
self._thread.join(timeout=5)
|
||||
if not self._loop.is_running():
|
||||
self._loop.close()
|
||||
|
||||
|
||||
class BoxliteProvider(SandboxProvider):
|
||||
class _SyncBoxAdapter:
|
||||
"""Adapt a sync BoxLite ``Box`` handle to the async ``SimpleBox`` methods we use."""
|
||||
|
||||
def __init__(self, runtime: Any, box: Any) -> None:
|
||||
self._runtime = runtime
|
||||
self._box = box
|
||||
|
||||
async def exec(
|
||||
self,
|
||||
cmd: str,
|
||||
*args: str,
|
||||
env: dict[str, str] | None = None,
|
||||
user: str | None = None,
|
||||
timeout: float | None = None,
|
||||
cwd: str | None = None,
|
||||
) -> Any:
|
||||
return self._box.exec(
|
||||
cmd,
|
||||
*args,
|
||||
env=env,
|
||||
user=user,
|
||||
timeout=timeout,
|
||||
cwd=cwd,
|
||||
)
|
||||
|
||||
async def stop(self) -> None:
|
||||
try:
|
||||
self._box.stop()
|
||||
finally:
|
||||
self._runtime.stop()
|
||||
|
||||
|
||||
def _run_sync_adapter[T](coro: Awaitable[T], *, timeout: float | None = None) -> T:
|
||||
"""Run sync-adapter coroutines without using the BoxLite async loop."""
|
||||
if timeout is None:
|
||||
return asyncio.run(coro)
|
||||
return asyncio.run(asyncio.wait_for(coro, timeout=timeout))
|
||||
|
||||
|
||||
class BoxliteProvider(WarmPoolLifecycleMixin[BoxliteBox], SandboxProvider):
|
||||
"""Run each DeerFlow sandbox as a BoxLite micro-VM."""
|
||||
|
||||
uses_thread_data_mounts = False
|
||||
needs_upload_permission_adjustment = True
|
||||
_idle_checker_thread_name = "boxlite-idle-reaper"
|
||||
|
||||
@staticmethod
|
||||
def _sandbox_id(thread_id: str, user_id: str) -> str:
|
||||
"""Deterministic sandbox ID from user/thread scope.
|
||||
|
||||
Includes user_id so a box created for one user's bucket cannot be
|
||||
reclaimed by another user's thread with the same thread_id.
|
||||
"""
|
||||
return hashlib.sha256(f"{user_id}:{thread_id}".encode()).hexdigest()[:8]
|
||||
|
||||
# ── Provider ────────────────────────────────────────────────────────
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._lock = threading.Lock()
|
||||
self._boxes: dict[str, BoxliteBox] = {}
|
||||
self._thread_boxes: dict[tuple[str, str], str] = {}
|
||||
self._warm_pool: dict[str, tuple[BoxliteBox, float]] = {}
|
||||
self._acquire_locks: dict[str, threading.Lock] = {}
|
||||
self._idle_checker_stop = threading.Event()
|
||||
self._idle_checker_thread: threading.Thread | None = None
|
||||
self._shutdown_called = False
|
||||
self._config = self._load_config()
|
||||
self._loop = _EventLoopThread()
|
||||
atexit.register(self.shutdown)
|
||||
self._reconcile_orphans()
|
||||
self._start_idle_checker()
|
||||
|
||||
def _load_config(self) -> dict[str, Any]:
|
||||
sandbox_config = get_app_config().sandbox
|
||||
@ -108,39 +194,172 @@ class BoxliteProvider(SandboxProvider):
|
||||
|
||||
# $VARS in config.yaml are already resolved by AppConfig.resolve_env_variables
|
||||
# (which raises on a missing var), so the environment dict is used as-is.
|
||||
replicas = _opt("replicas")
|
||||
idle_timeout = _opt("idle_timeout")
|
||||
return {
|
||||
"image": _opt("image") or DEFAULT_IMAGE,
|
||||
"memory_mib": _opt("memory_mib"),
|
||||
"cpus": _opt("cpus"),
|
||||
"environment": dict(_opt("environment") or {}),
|
||||
"replicas": replicas if replicas is not None else self.DEFAULT_REPLICAS,
|
||||
"idle_timeout": idle_timeout if idle_timeout is not None else self.DEFAULT_IDLE_TIMEOUT,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _thread_key(thread_id: str, user_id: str | None) -> tuple[str, str]:
|
||||
return (user_id or "", thread_id)
|
||||
|
||||
@staticmethod
|
||||
def _box_name(sandbox_id: str) -> str:
|
||||
return f"{_BOX_NAME_PREFIX}{sandbox_id}"
|
||||
|
||||
@staticmethod
|
||||
def _sandbox_id_from_box_name(name: str | None) -> str | None:
|
||||
if not name or not name.startswith(_BOX_NAME_PREFIX):
|
||||
return None
|
||||
sandbox_id = name[len(_BOX_NAME_PREFIX) :]
|
||||
return sandbox_id or None
|
||||
|
||||
def _lock_for_sandbox(self, sandbox_id: str) -> threading.Lock:
|
||||
"""Return the per-sandbox acquire lock for a deterministic sandbox id."""
|
||||
with self._lock:
|
||||
lock = self._acquire_locks.get(sandbox_id)
|
||||
if lock is None:
|
||||
lock = threading.Lock()
|
||||
self._acquire_locks[sandbox_id] = lock
|
||||
return lock
|
||||
|
||||
def _start_idle_checker(self) -> None:
|
||||
"""Start idle cleanup when enabled; idle_timeout=0 keeps it disabled."""
|
||||
if self._config["idle_timeout"] <= 0:
|
||||
return
|
||||
super()._start_idle_checker()
|
||||
|
||||
def _active_count_locked(self) -> int:
|
||||
"""Return active BoxLite box count while ``_lock`` is held."""
|
||||
return len(self._boxes)
|
||||
|
||||
def _destroy_warm_entry(self, sandbox_id: str, entry: BoxliteBox, *, reason: str) -> None:
|
||||
"""Close a removed warm-pool entry and log with context."""
|
||||
try:
|
||||
entry.close()
|
||||
if reason == "idle_timeout":
|
||||
logger.info("Idle reaper destroyed expired warm-pool box %s", sandbox_id)
|
||||
elif reason == "replica_enforcement":
|
||||
logger.info("Replica enforcement evicted oldest warm-pool box %s", sandbox_id)
|
||||
else:
|
||||
logger.info("Destroyed warm-pool box %s (reason=%s)", sandbox_id, reason)
|
||||
except Exception as e:
|
||||
if reason == "idle_timeout":
|
||||
logger.warning("Error closing expired BoxLite box %s: %s", sandbox_id, e)
|
||||
elif reason == "replica_enforcement":
|
||||
logger.warning("Error closing evicted BoxLite box %s: %s", sandbox_id, e)
|
||||
else:
|
||||
logger.warning("Error closing BoxLite box %s (reason=%s): %s", sandbox_id, reason, e)
|
||||
|
||||
def _reconcile_orphans(self) -> None:
|
||||
"""Adopt DeerFlow-owned BoxLite boxes left by a previous provider/process.
|
||||
|
||||
BoxLite boxes are discovered by a DeerFlow-specific name prefix. Adopted
|
||||
boxes enter the warm pool so the normal idle reaper can reclaim them.
|
||||
"""
|
||||
try:
|
||||
adopted = self._adopt_existing_boxes()
|
||||
except ImportError:
|
||||
logger.debug("BoxLite is not installed; skipping startup reconciliation")
|
||||
return
|
||||
except Exception as e:
|
||||
logger.warning("Failed to reconcile existing BoxLite boxes: %s", e)
|
||||
return
|
||||
|
||||
if adopted:
|
||||
logger.info("Startup reconciliation adopted %s BoxLite box(es)", adopted)
|
||||
|
||||
def _adopt_existing_boxes(self) -> int:
|
||||
runtime_cls = _import_sync_boxlite_runtime()
|
||||
now = time.time()
|
||||
adopted = 0
|
||||
|
||||
list_runtime = runtime_cls.default().start()
|
||||
try:
|
||||
infos = list_runtime.list_info()
|
||||
finally:
|
||||
list_runtime.stop()
|
||||
|
||||
for info in infos:
|
||||
name = getattr(info, "name", None)
|
||||
sandbox_id = self._sandbox_id_from_box_name(name)
|
||||
if sandbox_id is None:
|
||||
continue
|
||||
with self._lock:
|
||||
if sandbox_id in self._boxes or sandbox_id in self._warm_pool:
|
||||
continue
|
||||
|
||||
box_runtime = runtime_cls.default().start()
|
||||
try:
|
||||
box = box_runtime.get(name)
|
||||
except Exception as e:
|
||||
box_runtime.stop()
|
||||
logger.warning("Failed to retrieve existing BoxLite box %s: %s", name, e)
|
||||
continue
|
||||
if box is None:
|
||||
box_runtime.stop()
|
||||
continue
|
||||
|
||||
wrapped = BoxliteBox(sandbox_id, _SyncBoxAdapter(box_runtime, box), _run_sync_adapter, default_env=self._config["environment"])
|
||||
with self._lock:
|
||||
if sandbox_id in self._boxes or sandbox_id in self._warm_pool:
|
||||
box_runtime.stop()
|
||||
continue
|
||||
self._warm_pool[sandbox_id] = (wrapped, now)
|
||||
adopted += 1
|
||||
logger.info("Adopted existing BoxLite box %s (%s) into warm pool", sandbox_id, name)
|
||||
|
||||
return adopted
|
||||
|
||||
# ── Acquire / release ────────────────────────────────────────────────
|
||||
|
||||
def acquire(self, thread_id: str | None = None, *, user_id: str | None = None) -> str:
|
||||
if thread_id is not None:
|
||||
key = self._thread_key(thread_id, user_id)
|
||||
if thread_id is None:
|
||||
sandbox_id = str(uuid.uuid4())[:8]
|
||||
box = self._create_box(sandbox_id)
|
||||
with self._lock:
|
||||
self._boxes[box.id] = box
|
||||
return box.id
|
||||
|
||||
key = self._thread_key(thread_id, user_id)
|
||||
sandbox_id = self._sandbox_id(thread_id, user_id)
|
||||
acquire_lock = self._lock_for_sandbox(sandbox_id)
|
||||
with acquire_lock:
|
||||
with self._lock:
|
||||
existing = self._thread_boxes.get(key)
|
||||
if existing is not None and existing in self._boxes:
|
||||
return existing
|
||||
|
||||
box = self._create_box()
|
||||
reclaimed = self._reclaim_warm_pool(sandbox_id)
|
||||
if reclaimed is not None:
|
||||
with self._lock:
|
||||
self._thread_boxes[key] = reclaimed
|
||||
return reclaimed
|
||||
|
||||
with self._lock:
|
||||
self._boxes[box.id] = box
|
||||
if thread_id is not None:
|
||||
self._thread_boxes[self._thread_key(thread_id, user_id)] = box.id
|
||||
return box.id
|
||||
box = self._create_box(sandbox_id)
|
||||
with self._lock:
|
||||
self._boxes[box.id] = box
|
||||
self._thread_boxes[key] = box.id
|
||||
return box.id
|
||||
|
||||
def _create_box(self) -> BoxliteBox:
|
||||
def _create_box(self, sandbox_id: str) -> BoxliteBox:
|
||||
# Enforce replica limit: evict oldest warm-pool box if active + warm boxes are at capacity.
|
||||
replicas, total = self._replica_count()
|
||||
if total >= replicas:
|
||||
evicted = self._evict_oldest_warm()
|
||||
self._log_replicas_soft_cap(replicas, sandbox_id, evicted)
|
||||
simplebox_cls = _import_simplebox()
|
||||
mkdir_cmd = "mkdir -p " + " ".join(_VIRTUAL_DIRS)
|
||||
|
||||
async def _make() -> SimpleBox:
|
||||
box = simplebox_cls(
|
||||
name=self._box_name(sandbox_id),
|
||||
image=self._config["image"],
|
||||
memory_mib=self._config["memory_mib"],
|
||||
cpus=self._config["cpus"],
|
||||
@ -151,36 +370,109 @@ class BoxliteProvider(SandboxProvider):
|
||||
return box
|
||||
|
||||
box = self._loop.run(_make())
|
||||
logger.info("Created BoxLite box %s (image=%s)", box.id, self._config["image"])
|
||||
return BoxliteBox(box.id, box, self._loop.run, default_env=self._config["environment"])
|
||||
logger.info("Created BoxLite box %s (name=%s, image=%s)", sandbox_id, self._box_name(sandbox_id), self._config["image"])
|
||||
return BoxliteBox(sandbox_id, box, self._loop.run, default_env=self._config["environment"])
|
||||
|
||||
def get(self, sandbox_id: str) -> Sandbox | None:
|
||||
with self._lock:
|
||||
return self._boxes.get(sandbox_id)
|
||||
|
||||
def release(self, sandbox_id: str) -> None:
|
||||
"""Release a sandbox into the warm pool — VM stays running.
|
||||
|
||||
The box is moved from _boxes to _warm_pool; _thread_boxes entries are
|
||||
cleared so the thread no longer holds an active reference. The VM is
|
||||
NOT stopped unless shutdown has already begun.
|
||||
"""
|
||||
close_box: BoxliteBox | None = None
|
||||
with self._lock:
|
||||
box = self._boxes.pop(sandbox_id, None)
|
||||
for key in [k for k, sid in self._thread_boxes.items() if sid == sandbox_id]:
|
||||
self._thread_boxes.pop(key, None)
|
||||
if box is not None:
|
||||
if box is None:
|
||||
return
|
||||
if self._shutdown_called:
|
||||
close_box = box
|
||||
else:
|
||||
self._warm_pool[sandbox_id] = (box, time.time())
|
||||
|
||||
if close_box is not None:
|
||||
close_box.close()
|
||||
logger.info("Closed released sandbox %s because shutdown is in progress", sandbox_id)
|
||||
else:
|
||||
logger.info("Released sandbox %s to warm pool (VM still running)", sandbox_id)
|
||||
|
||||
def _reclaim_warm_pool(self, sandbox_id: str) -> str | None:
|
||||
"""Try to reclaim a warm-pool box by sandbox_id.
|
||||
|
||||
Returns sandbox_id on success, None if not found or dead.
|
||||
"""
|
||||
with self._lock:
|
||||
if sandbox_id not in self._warm_pool:
|
||||
return None
|
||||
box, _ = self._warm_pool[sandbox_id]
|
||||
|
||||
# Health check: run a simple command to verify the VM is alive
|
||||
try:
|
||||
result = box.execute_command("echo ok", timeout=5)
|
||||
if "ok" not in result:
|
||||
logger.warning("Warm pool box %s health check failed: %s", sandbox_id, result)
|
||||
with self._lock:
|
||||
self._warm_pool.pop(sandbox_id, None)
|
||||
box.close()
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.warning("Warm pool box %s health check error: %s", sandbox_id, e)
|
||||
with self._lock:
|
||||
self._warm_pool.pop(sandbox_id, None)
|
||||
box.close()
|
||||
return None
|
||||
|
||||
# Promote from warm pool to active
|
||||
with self._lock:
|
||||
warm_entry = self._warm_pool.pop(sandbox_id, None)
|
||||
if warm_entry is None:
|
||||
return None # Raced with another thread
|
||||
box, _ = warm_entry
|
||||
self._boxes[sandbox_id] = box
|
||||
|
||||
logger.info("Reclaimed warm-pool box %s", sandbox_id)
|
||||
return sandbox_id
|
||||
|
||||
def reset(self) -> None:
|
||||
"""Release tracked BoxLite VMs to this instance's warm-pool cleanup.
|
||||
|
||||
``reset_sandbox_provider()`` drops the provider singleton and calls this
|
||||
lightweight hook so config changes take effect on the next provider
|
||||
construction. Teardown belongs to ``shutdown()``; reset intentionally
|
||||
leaves running VMs alive, but keeps them visible to this instance's idle
|
||||
reaper and atexit shutdown instead of orphaning them.
|
||||
"""
|
||||
with self._lock:
|
||||
now = time.time()
|
||||
for sandbox_id, box in self._boxes.items():
|
||||
self._warm_pool.setdefault(sandbox_id, (box, now))
|
||||
self._boxes.clear()
|
||||
self._thread_boxes.clear()
|
||||
self._acquire_locks.clear()
|
||||
|
||||
def shutdown(self) -> None:
|
||||
with self._lock:
|
||||
if self._shutdown_called:
|
||||
return
|
||||
self._shutdown_called = True
|
||||
active = list(self._boxes.values())
|
||||
self._boxes.clear()
|
||||
self._thread_boxes.clear()
|
||||
|
||||
for box in active:
|
||||
self._stop_idle_checker()
|
||||
|
||||
with self._lock:
|
||||
active = list(self._boxes.values())
|
||||
warm = [box for box, _ in self._warm_pool.values()]
|
||||
self._boxes.clear()
|
||||
self._warm_pool.clear()
|
||||
self._thread_boxes.clear()
|
||||
self._acquire_locks.clear()
|
||||
|
||||
for box in active + warm:
|
||||
try:
|
||||
box.close()
|
||||
except Exception as e: # pragma: no cover - defensive
|
||||
|
||||
@ -0,0 +1,127 @@
|
||||
"""Shared warm-pool lifecycle helpers for community sandbox providers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_IDLE_TIMEOUT = 600
|
||||
DEFAULT_REPLICAS = 3
|
||||
IDLE_CHECK_INTERVAL = 60
|
||||
|
||||
|
||||
class WarmPoolLifecycleMixin[WarmEntryT]:
|
||||
"""Mixin for provider warm-pool expiry and replica lifecycle mechanics."""
|
||||
|
||||
DEFAULT_IDLE_TIMEOUT = DEFAULT_IDLE_TIMEOUT
|
||||
DEFAULT_REPLICAS = DEFAULT_REPLICAS
|
||||
IDLE_CHECK_INTERVAL = IDLE_CHECK_INTERVAL
|
||||
_idle_checker_thread_name = "warm-pool-idle-checker"
|
||||
|
||||
_lock: threading.Lock
|
||||
_warm_pool: dict[str, tuple[WarmEntryT, float]]
|
||||
_config: dict[str, Any]
|
||||
_idle_checker_stop: threading.Event
|
||||
_idle_checker_thread: threading.Thread | None
|
||||
|
||||
def _active_count_locked(self) -> int:
|
||||
"""Return active entry count while ``_lock`` is held."""
|
||||
raise NotImplementedError
|
||||
|
||||
def _destroy_warm_entry(self, sandbox_id: str, entry: WarmEntryT, *, reason: str) -> None:
|
||||
"""Destroy a warm-pool entry after it has been removed from the pool."""
|
||||
raise NotImplementedError
|
||||
|
||||
def _replica_count(self) -> tuple[int, int]:
|
||||
"""Return configured replicas and current active + warm entry count."""
|
||||
replicas = int(self._config.get("replicas", DEFAULT_REPLICAS))
|
||||
with self._lock:
|
||||
total = self._active_count_locked() + len(self._warm_pool)
|
||||
return replicas, total
|
||||
|
||||
def _log_replicas_soft_cap(self, replicas: int, sandbox_id: str, evicted: str | None) -> None:
|
||||
"""Log the result of enforcing the warm-pool replica soft cap."""
|
||||
if evicted is not None:
|
||||
logger.info("Evicted warm-pool sandbox %s to stay within replicas=%s", evicted, replicas)
|
||||
return
|
||||
|
||||
logger.warning(
|
||||
"All %s replica slots are in active use; creating sandbox %s beyond the soft limit",
|
||||
replicas,
|
||||
sandbox_id,
|
||||
)
|
||||
|
||||
def _evict_oldest_warm(self) -> str | None:
|
||||
"""Remove and destroy the oldest warm entry by timestamp."""
|
||||
with self._lock:
|
||||
if not self._warm_pool:
|
||||
return None
|
||||
sandbox_id, (entry, _) = min(self._warm_pool.items(), key=lambda item: item[1][1])
|
||||
self._warm_pool.pop(sandbox_id)
|
||||
|
||||
self._destroy_warm_entry(sandbox_id, entry, reason="replica_enforcement")
|
||||
return sandbox_id
|
||||
|
||||
def _reap_expired_warm(self, idle_timeout: float | None = None) -> None:
|
||||
"""Remove and destroy warm entries older than ``idle_timeout`` seconds."""
|
||||
timeout = float(self._config.get("idle_timeout", DEFAULT_IDLE_TIMEOUT) if idle_timeout is None else idle_timeout)
|
||||
if timeout <= 0:
|
||||
return
|
||||
|
||||
now = time.time()
|
||||
expired: list[tuple[str, WarmEntryT]] = []
|
||||
with self._lock:
|
||||
for sandbox_id, (entry, timestamp) in self._warm_pool.items():
|
||||
if now - timestamp > timeout:
|
||||
expired.append((sandbox_id, entry))
|
||||
for sandbox_id, _ in expired:
|
||||
self._warm_pool.pop(sandbox_id, None)
|
||||
|
||||
for sandbox_id, entry in expired:
|
||||
self._destroy_warm_entry(sandbox_id, entry, reason="idle_timeout")
|
||||
|
||||
def _start_idle_checker(self) -> None:
|
||||
"""Start the daemon thread that periodically cleans idle warm entries."""
|
||||
if self._idle_checker_thread is not None and self._idle_checker_thread.is_alive():
|
||||
return
|
||||
|
||||
self._idle_checker_stop.clear()
|
||||
self._idle_checker_thread = threading.Thread(
|
||||
target=self._idle_checker_loop,
|
||||
name=self._idle_checker_thread_name,
|
||||
daemon=True,
|
||||
)
|
||||
self._idle_checker_thread.start()
|
||||
logger.info("Started warm-pool idle checker thread (timeout: %ss)", self._config.get("idle_timeout", DEFAULT_IDLE_TIMEOUT))
|
||||
|
||||
def _stop_idle_checker(self) -> None:
|
||||
"""Stop the idle checker thread and wait for it to exit when running."""
|
||||
self._idle_checker_stop.set()
|
||||
thread = self._idle_checker_thread
|
||||
if thread is not None and thread.is_alive() and thread is not threading.current_thread():
|
||||
thread.join(timeout=5)
|
||||
|
||||
def _idle_checker_loop(self) -> None:
|
||||
"""Run periodic idle cleanup until the stop event is set."""
|
||||
idle_timeout = float(self._config.get("idle_timeout", DEFAULT_IDLE_TIMEOUT))
|
||||
while not self._idle_checker_stop.wait(self.IDLE_CHECK_INTERVAL):
|
||||
try:
|
||||
self._cleanup_idle_resources(idle_timeout)
|
||||
except Exception:
|
||||
logger.exception("Error in warm-pool idle checker loop")
|
||||
|
||||
def _cleanup_idle_resources(self, idle_timeout: float) -> None:
|
||||
"""Clean resources idle longer than ``idle_timeout`` seconds."""
|
||||
self._reap_expired_warm(idle_timeout)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_IDLE_TIMEOUT",
|
||||
"DEFAULT_REPLICAS",
|
||||
"IDLE_CHECK_INTERVAL",
|
||||
"WarmPoolLifecycleMixin",
|
||||
]
|
||||
@ -30,14 +30,16 @@ class SandboxConfig(BaseModel):
|
||||
allow_host_bash: Enable host-side bash execution for LocalSandboxProvider.
|
||||
Dangerous and intended only for fully trusted local workflows.
|
||||
|
||||
AioSandboxProvider and BoxliteProvider shared options:
|
||||
image: Sandbox image to use (Docker/AIO image or BoxLite OCI image)
|
||||
replicas: Maximum active + warm sandboxes/VMs per gateway process (default: 3). When the limit is reached, warm/least-recently-used sandboxes are evicted to make room; active sandboxes are not forcibly stopped.
|
||||
idle_timeout: Idle timeout in seconds before released warm sandboxes/VMs are stopped (default: 600 = 10 minutes). Set to 0 to disable.
|
||||
environment: Environment variables to inject into the sandbox (values starting with $ are resolved from host env)
|
||||
|
||||
AioSandboxProvider specific options:
|
||||
image: Docker image to use (default: enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest)
|
||||
port: Base port for sandbox containers (default: 8080)
|
||||
replicas: Maximum number of concurrent sandbox containers (default: 3). When the limit is reached the least-recently-used sandbox is evicted to make room.
|
||||
container_prefix: Prefix for container names (default: deer-flow-sandbox)
|
||||
idle_timeout: Idle timeout in seconds before sandbox is released (default: 600 = 10 minutes). Set to 0 to disable.
|
||||
mounts: List of volume mounts to share directories with the container
|
||||
environment: Environment variables to inject into the container (values starting with $ are resolved from host env)
|
||||
"""
|
||||
|
||||
use: str = Field(
|
||||
@ -50,7 +52,7 @@ class SandboxConfig(BaseModel):
|
||||
)
|
||||
image: str | None = Field(
|
||||
default=None,
|
||||
description="Docker image to use for the sandbox container",
|
||||
description="Sandbox image to use (Docker/AIO image or BoxLite OCI image)",
|
||||
)
|
||||
port: int | None = Field(
|
||||
default=None,
|
||||
@ -58,7 +60,7 @@ class SandboxConfig(BaseModel):
|
||||
)
|
||||
replicas: int | None = Field(
|
||||
default=None,
|
||||
description="Maximum number of concurrent sandbox containers (default: 3). When the limit is reached the least-recently-used sandbox is evicted to make room.",
|
||||
description="Maximum active + warm sandboxes/VMs per gateway process (default: 3). Warm/least-recently-used entries are evicted to make room; active sandboxes are not forcibly stopped.",
|
||||
)
|
||||
container_prefix: str | None = Field(
|
||||
default=None,
|
||||
@ -66,7 +68,7 @@ class SandboxConfig(BaseModel):
|
||||
)
|
||||
idle_timeout: int | None = Field(
|
||||
default=None,
|
||||
description="Idle timeout in seconds before sandbox is released (default: 600 = 10 minutes). Set to 0 to disable.",
|
||||
description="Idle timeout in seconds before released warm sandboxes/VMs are stopped (default: 600 = 10 minutes). Set to 0 to disable.",
|
||||
)
|
||||
mounts: list[VolumeMountConfig] = Field(
|
||||
default_factory=list,
|
||||
|
||||
@ -64,6 +64,7 @@ postgres = [
|
||||
# always installs this extra because Docker defaults to the redis bridge.
|
||||
redis = ["redis>=5.0.0"]
|
||||
pymupdf = ["pymupdf4llm>=0.0.17"]
|
||||
boxlite = ["boxlite>=0.9.7"]
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
|
||||
@ -626,3 +626,66 @@ def test_destroy_swallows_close_errors_and_still_destroys_backend(tmp_path, capl
|
||||
|
||||
assert "Error closing sandbox sandbox-dest-err during destroy" in caplog.text
|
||||
provider._backend.destroy.assert_called_once()
|
||||
|
||||
|
||||
def test_cleanup_idle_sandboxes_keeps_active_cleanup_and_delegates_warm_expiry(tmp_path):
|
||||
"""AIO active-idle cleanup must remain local while warm expiry uses the shared lifecycle."""
|
||||
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
|
||||
provider = _make_provider(tmp_path)
|
||||
provider._lock = aio_mod.threading.Lock()
|
||||
provider._sandboxes = {"active-old": MagicMock()}
|
||||
provider._sandbox_infos = {
|
||||
"active-old": aio_mod.SandboxInfo(sandbox_id="active-old", sandbox_url="http://active-old"),
|
||||
}
|
||||
provider._thread_sandboxes = {("default", "thread-old"): "active-old"}
|
||||
provider._last_activity = {"active-old": 0.0}
|
||||
provider._warm_pool = {
|
||||
"warm-old": (
|
||||
aio_mod.SandboxInfo(sandbox_id="warm-old", sandbox_url="http://warm-old"),
|
||||
0.0,
|
||||
)
|
||||
}
|
||||
|
||||
calls = []
|
||||
provider.destroy = MagicMock(side_effect=lambda _sandbox_id: calls.append("active"))
|
||||
provider._reap_expired_warm = MagicMock(side_effect=lambda _idle_timeout: calls.append("warm"))
|
||||
|
||||
provider._cleanup_idle_sandboxes(1.0)
|
||||
|
||||
provider.destroy.assert_called_once_with("active-old")
|
||||
provider._reap_expired_warm.assert_called_once_with(1.0)
|
||||
assert calls == ["active", "warm"]
|
||||
|
||||
|
||||
def test_create_sandbox_evicts_oldest_warm_replica_via_shared_lifecycle(tmp_path, monkeypatch):
|
||||
"""Replica enforcement must destroy the oldest warm SandboxInfo before creating another."""
|
||||
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
|
||||
provider = _make_provider(tmp_path)
|
||||
provider._lock = aio_mod.threading.Lock()
|
||||
provider._config = {"replicas": 2}
|
||||
provider._sandboxes = {}
|
||||
provider._sandbox_infos = {}
|
||||
provider._thread_sandboxes = {}
|
||||
provider._last_activity = {}
|
||||
|
||||
oldest_info = aio_mod.SandboxInfo(sandbox_id="warm-oldest", sandbox_url="http://warm-oldest")
|
||||
newest_info = aio_mod.SandboxInfo(sandbox_id="warm-newest", sandbox_url="http://warm-newest")
|
||||
created_info = aio_mod.SandboxInfo(sandbox_id="created", sandbox_url="http://created")
|
||||
provider._warm_pool = {
|
||||
"warm-newest": (newest_info, 20.0),
|
||||
"warm-oldest": (oldest_info, 10.0),
|
||||
}
|
||||
provider._backend = SimpleNamespace(
|
||||
create=MagicMock(return_value=created_info),
|
||||
destroy=MagicMock(),
|
||||
)
|
||||
monkeypatch.setattr(aio_mod.AioSandboxProvider, "_get_extra_mounts", lambda _self, _thread_id, *, user_id=None: [])
|
||||
monkeypatch.setattr(aio_mod, "wait_for_sandbox_ready", lambda _url, *, timeout=60: True)
|
||||
|
||||
sandbox_id = provider._create_sandbox(None, "created", user_id="default")
|
||||
|
||||
assert sandbox_id == "created"
|
||||
provider._backend.destroy.assert_called_once_with(oldest_info)
|
||||
assert "warm-oldest" not in provider._warm_pool
|
||||
assert provider._warm_pool == {"warm-newest": (newest_info, 20.0)}
|
||||
assert provider._sandbox_infos["created"] is created_info
|
||||
|
||||
@ -1,12 +1,15 @@
|
||||
"""Unit tests for the BoxLite community provider.
|
||||
|
||||
These run in CI without BoxLite installed: they cover the lazy-import error path,
|
||||
provider lifecycle, and the path-safety guards — none of which need a live box.
|
||||
provider lifecycle, the path-safety guards, and the warm pool lifecycle — none of
|
||||
which need a live box.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import types
|
||||
|
||||
import pytest
|
||||
@ -14,15 +17,80 @@ import pytest
|
||||
from deerflow.community.boxlite.box import BoxliteBox
|
||||
from deerflow.community.boxlite.provider import BoxliteProvider, _import_simplebox
|
||||
|
||||
# ── Fake BoxLite SDK ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
class _FakeBox:
|
||||
"""A fake SimpleBox that records lifecycle calls without starting real VMs."""
|
||||
|
||||
def __init__(self, *, image=None, name=None, memory_mib=None, cpus=None, **kwargs):
|
||||
self.id = name or "auto-gen-id"
|
||||
self.name = name
|
||||
self._image = image
|
||||
self._started = False
|
||||
self._stopped = False
|
||||
self._exec_history: list[tuple] = []
|
||||
|
||||
async def start(self):
|
||||
self._started = True
|
||||
|
||||
async def exec(self, *argv, env=None, timeout=None):
|
||||
self._exec_history.append((argv, env, timeout))
|
||||
_FakeResult = type("_FakeResult", (), {"stdout": "", "stderr": "", "exit_code": 0})
|
||||
# Health check: box.execute_command("echo ok") → exec("sh", "-lc", "echo ok")
|
||||
if len(argv) >= 3 and argv[0] == "sh" and argv[1] == "-lc" and argv[2] == "echo ok":
|
||||
return type("_FakeResult", (), {"stdout": "ok\n", "stderr": "", "exit_code": 0})()
|
||||
return _FakeResult()
|
||||
|
||||
async def stop(self):
|
||||
self._stopped = True
|
||||
|
||||
|
||||
def _fake_run(coro, *, timeout=None):
|
||||
"""Sync runner that executes coroutines on a temporary event loop (no daemon thread)."""
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
# ── Config stub ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _stub_config(sandbox_attrs=None):
|
||||
"""Stub get_app_config to return a config with given sandbox attrs."""
|
||||
attrs = sandbox_attrs or {}
|
||||
stub = types.SimpleNamespace(sandbox=types.SimpleNamespace(**attrs))
|
||||
return stub
|
||||
|
||||
|
||||
def _no_boxlite(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Make ``import boxlite`` raise, regardless of whether it is installed."""
|
||||
monkeypatch.setitem(sys.modules, "boxlite", None)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _no_existing_boxlite_boxes(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Keep provider startup reconciliation isolated from the real SDK state."""
|
||||
|
||||
class _EmptyRuntime:
|
||||
def start(self):
|
||||
return self
|
||||
|
||||
def stop(self):
|
||||
pass
|
||||
|
||||
def list_info(self):
|
||||
return []
|
||||
|
||||
class _EmptyBoxlite:
|
||||
@staticmethod
|
||||
def default():
|
||||
return _EmptyRuntime()
|
||||
|
||||
monkeypatch.setattr("deerflow.community.boxlite.provider._import_sync_boxlite_runtime", lambda: _EmptyBoxlite)
|
||||
|
||||
|
||||
def test_import_simplebox_missing_raises_actionable(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_no_boxlite(monkeypatch)
|
||||
with pytest.raises(ImportError, match=r"pip install boxlite"):
|
||||
with pytest.raises(ImportError, match=r"deerflow-harness\[boxlite\]"):
|
||||
_import_simplebox()
|
||||
|
||||
|
||||
@ -34,7 +102,7 @@ def test_acquire_without_boxlite_raises_and_shuts_down_cleanly(monkeypatch: pyte
|
||||
|
||||
provider = BoxliteProvider()
|
||||
try:
|
||||
with pytest.raises(ImportError, match=r"pip install boxlite"):
|
||||
with pytest.raises(ImportError, match=r"deerflow-harness\[boxlite\]"):
|
||||
provider.acquire("thread-1", user_id="u")
|
||||
finally:
|
||||
provider.shutdown() # must not raise even though no box was ever created
|
||||
@ -70,3 +138,573 @@ def test_execute_command_rejects_invalid_env_key() -> None:
|
||||
box = BoxliteBox("box-id", box=object(), run=_fail_run)
|
||||
with pytest.raises(ValueError, match=r"POSIX"):
|
||||
box.execute_command("echo hi", env={"BAD KEY": "x"})
|
||||
|
||||
|
||||
def test_execute_command_forwards_timeout_to_sdk_and_loop_runner() -> None:
|
||||
"""Command timeout must bound both BoxLite exec and the loop bridge future."""
|
||||
run_timeouts: list[float | None] = []
|
||||
|
||||
def _recording_run(coro, *, timeout=None):
|
||||
run_timeouts.append(timeout)
|
||||
return asyncio.run(coro)
|
||||
|
||||
fake = _FakeBox(name="box-id")
|
||||
box = BoxliteBox("box-id", box=fake, run=_recording_run)
|
||||
|
||||
output = box.execute_command("echo ok", timeout=5)
|
||||
|
||||
assert "ok" in output
|
||||
assert fake._exec_history[-1] == (("sh", "-lc", "echo ok"), None, 5)
|
||||
assert run_timeouts == [5]
|
||||
|
||||
|
||||
def test_sandbox_id_deterministic(monkeypatch):
|
||||
"""_sandbox_id produces the same id for the same inputs."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
provider = BoxliteProvider()
|
||||
id1 = provider._sandbox_id("thread-1", "user-a")
|
||||
id2 = provider._sandbox_id("thread-1", "user-a")
|
||||
assert id1 == id2
|
||||
assert len(id1) == 8
|
||||
|
||||
|
||||
def test_sandbox_id_different_users(monkeypatch):
|
||||
"""Different users produce different ids for the same thread."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
provider = BoxliteProvider()
|
||||
id_a = provider._sandbox_id("thread-1", "user-a")
|
||||
id_b = provider._sandbox_id("thread-1", "user-b")
|
||||
assert id_a != id_b
|
||||
|
||||
|
||||
def test_sandbox_id_different_threads(monkeypatch):
|
||||
"""Different threads produce different ids for the same user."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
provider = BoxliteProvider()
|
||||
id_a = provider._sandbox_id("thread-1", "user-a")
|
||||
id_b = provider._sandbox_id("thread-2", "user-a")
|
||||
assert id_a != id_b
|
||||
|
||||
|
||||
def test_idle_timeout_zero_is_preserved_and_disables_reaper(monkeypatch):
|
||||
"""idle_timeout=0 is a valid config value and disables the reaper thread."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config({"idle_timeout": 0}),
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
|
||||
assert provider._config["idle_timeout"] == 0
|
||||
assert provider._idle_checker_thread is None
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
def test_create_box_passes_prefixed_sandbox_id_as_name(monkeypatch):
|
||||
"""_create_box gives BoxLite a DeerFlow-owned name prefix."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
# Inject fake SimpleBox and fake loop runner
|
||||
created_boxes = []
|
||||
|
||||
class _RecordingBox(_FakeBox):
|
||||
def __init__(self, **kwargs):
|
||||
super().__init__(**kwargs)
|
||||
created_boxes.append(kwargs)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _RecordingBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
# Replace _loop.run with our sync runner
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
box = provider._create_box("test-sandbox-id")
|
||||
assert len(created_boxes) == 1
|
||||
assert created_boxes[0]["name"] == "deer-flow-boxlite-test-sandbox-id"
|
||||
assert box.id == "test-sandbox-id"
|
||||
|
||||
|
||||
def test_startup_reconciliation_adopts_prefixed_existing_boxes(monkeypatch):
|
||||
"""Existing DeerFlow-named BoxLite boxes are adopted into the warm pool."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
stopped: list[str] = []
|
||||
|
||||
class _NativeBox:
|
||||
def stop(self):
|
||||
stopped.append("adopted")
|
||||
|
||||
class _Runtime:
|
||||
def start(self):
|
||||
return self
|
||||
|
||||
def stop(self):
|
||||
pass
|
||||
|
||||
def list_info(self):
|
||||
return [
|
||||
types.SimpleNamespace(name="deer-flow-boxlite-adopted"),
|
||||
types.SimpleNamespace(name="unrelated-box"),
|
||||
types.SimpleNamespace(name=None),
|
||||
]
|
||||
|
||||
def get(self, name):
|
||||
if name == "deer-flow-boxlite-adopted":
|
||||
return _NativeBox()
|
||||
raise AssertionError(f"unexpected box lookup: {name}")
|
||||
|
||||
class _Boxlite:
|
||||
@staticmethod
|
||||
def default():
|
||||
return _Runtime()
|
||||
|
||||
monkeypatch.setattr("deerflow.community.boxlite.provider._import_sync_boxlite_runtime", lambda: _Boxlite)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
|
||||
assert list(provider._warm_pool) == ["adopted"]
|
||||
adopted_box = provider._warm_pool["adopted"][0]
|
||||
assert adopted_box.id == "adopted"
|
||||
|
||||
provider.shutdown()
|
||||
assert stopped == ["adopted"]
|
||||
|
||||
|
||||
def test_release_parks_in_warm_pool(monkeypatch):
|
||||
"""After release, box is in warm pool, not destroyed."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
# Acquire a box
|
||||
sid = provider.acquire("thread-1", user_id="u1")
|
||||
|
||||
# Verify box is active
|
||||
assert sid in provider._boxes
|
||||
assert sid not in provider._warm_pool
|
||||
|
||||
# Release
|
||||
provider.release(sid)
|
||||
|
||||
# Verify box is in warm pool, not active
|
||||
assert sid not in provider._boxes
|
||||
assert sid in provider._warm_pool
|
||||
box, ts = provider._warm_pool[sid]
|
||||
assert isinstance(box, BoxliteBox)
|
||||
assert not box._box._stopped # VM not destroyed
|
||||
|
||||
|
||||
def test_acquire_reclaims_from_warm_pool(monkeypatch):
|
||||
"""acquire reclaims a warm pool box for the same thread."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
# First acquire → create
|
||||
sid1 = provider.acquire("thread-1", user_id="u1")
|
||||
provider.release(sid1)
|
||||
|
||||
# Second acquire → should reclaim from warm pool
|
||||
sid2 = provider.acquire("thread-1", user_id="u1")
|
||||
assert sid1 == sid2 # Same deterministic ID
|
||||
assert sid2 in provider._boxes
|
||||
assert sid2 not in provider._warm_pool
|
||||
|
||||
|
||||
def test_acquire_different_threads_dont_reclaim_each_other(monkeypatch):
|
||||
"""Thread A's box can't be reclaimed by thread B."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
sid_a = provider.acquire("thread-a", user_id="u1")
|
||||
provider.release(sid_a)
|
||||
|
||||
# Thread B acquires — should NOT get thread A's box
|
||||
sid_b = provider.acquire("thread-b", user_id="u1")
|
||||
assert sid_b != sid_a # Different deterministic ID
|
||||
assert sid_a in provider._warm_pool # A's box still in warm pool
|
||||
assert sid_b in provider._boxes # B's box is new
|
||||
|
||||
|
||||
def test_warm_pool_reclaim_failed_health_check_creates_new(monkeypatch):
|
||||
"""Dead warm pool box is evicted and a new one created."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
sid1 = provider.acquire("thread-1", user_id="u1")
|
||||
provider.release(sid1)
|
||||
assert sid1 in provider._warm_pool
|
||||
|
||||
# Corrupt the warm pool box: close it so health check fails
|
||||
box, _ = provider._warm_pool[sid1]
|
||||
box.close() # Stop VM, marks _closed=True
|
||||
|
||||
# Re-acquire — health check should fail on the dead box
|
||||
# A new box is created with the same deterministic ID
|
||||
sid2 = provider.acquire("thread-1", user_id="u1")
|
||||
assert sid2 == sid1 # Same deterministic ID
|
||||
assert sid2 in provider._boxes
|
||||
|
||||
|
||||
def test_concurrent_same_thread_acquire_creates_one_box(monkeypatch):
|
||||
"""Concurrent acquires for one thread serialize before creating a named box."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
original_create_box = provider._create_box
|
||||
create_started = threading.Event()
|
||||
created: list[str] = []
|
||||
|
||||
def slow_create_box(sandbox_id: str) -> BoxliteBox:
|
||||
create_started.set()
|
||||
time.sleep(0.1)
|
||||
created.append(sandbox_id)
|
||||
return original_create_box(sandbox_id)
|
||||
|
||||
provider._create_box = slow_create_box # type: ignore[method-assign]
|
||||
results: list[str] = []
|
||||
|
||||
def acquire() -> None:
|
||||
results.append(provider.acquire("thread-1", user_id="u1"))
|
||||
|
||||
first = threading.Thread(target=acquire)
|
||||
second = threading.Thread(target=acquire)
|
||||
first.start()
|
||||
assert create_started.wait(timeout=2)
|
||||
second.start()
|
||||
first.join(timeout=2)
|
||||
second.join(timeout=2)
|
||||
|
||||
assert len(results) == 2
|
||||
assert results[0] == results[1]
|
||||
assert len(created) == 1
|
||||
assert results[0] in provider._boxes
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
def test_release_during_shutdown_closes_instead_of_reparking(monkeypatch):
|
||||
"""release() must not park a VM after shutdown has begun."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
sid = provider.acquire("thread-1", user_id="u1")
|
||||
box = provider._boxes[sid]
|
||||
with provider._lock:
|
||||
provider._shutdown_called = True
|
||||
|
||||
provider.release(sid)
|
||||
|
||||
assert sid not in provider._boxes
|
||||
assert sid not in provider._warm_pool
|
||||
assert box._closed
|
||||
provider._loop.close()
|
||||
|
||||
|
||||
def test_reset_parks_running_resources_for_later_cleanup(monkeypatch):
|
||||
"""reset() stops thread reuse but leaves VMs tracked for cleanup."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
sid_active = provider.acquire("thread-active", user_id="u1")
|
||||
sid_warm = provider.acquire("thread-warm", user_id="u1")
|
||||
provider.release(sid_warm)
|
||||
active_box = provider._boxes[sid_active]
|
||||
warm_box = provider._warm_pool[sid_warm][0]
|
||||
checker_thread = provider._idle_checker_thread
|
||||
loop_thread = provider._loop._thread
|
||||
|
||||
provider.reset()
|
||||
|
||||
assert provider._boxes == {}
|
||||
assert provider._warm_pool[sid_active][0] is active_box
|
||||
assert provider._warm_pool[sid_warm][0] is warm_box
|
||||
assert provider._thread_boxes == {}
|
||||
assert provider._acquire_locks == {}
|
||||
assert not active_box._closed
|
||||
assert not warm_box._closed
|
||||
assert not provider._shutdown_called
|
||||
assert not provider._idle_checker_stop.is_set()
|
||||
assert checker_thread is not None
|
||||
assert checker_thread.is_alive()
|
||||
assert loop_thread.is_alive()
|
||||
|
||||
provider.shutdown()
|
||||
assert active_box._closed
|
||||
assert warm_box._closed
|
||||
assert provider._idle_checker_stop.is_set()
|
||||
assert not checker_thread.is_alive()
|
||||
|
||||
|
||||
def test_reset_parked_resources_are_reaped_after_idle_timeout(monkeypatch):
|
||||
"""VMs parked by reset remain visible to warm-pool idle cleanup."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
sid_active = provider.acquire("thread-active", user_id="u1")
|
||||
sid_warm = provider.acquire("thread-warm", user_id="u1")
|
||||
provider.release(sid_warm)
|
||||
active_box = provider._boxes[sid_active]
|
||||
warm_box = provider._warm_pool[sid_warm][0]
|
||||
|
||||
provider.reset()
|
||||
|
||||
provider._warm_pool[sid_active] = (active_box, time.time() - 9999)
|
||||
provider._warm_pool[sid_warm] = (warm_box, time.time() - 9999)
|
||||
provider._reap_expired_warm(idle_timeout=1)
|
||||
|
||||
assert provider._warm_pool == {}
|
||||
assert active_box._closed
|
||||
assert warm_box._closed
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
# ── Task 6: Idle reaper ───────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_idle_reaper_destroys_expired_warm_boxes(monkeypatch):
|
||||
"""Idle reaper daemon destroys warm pool boxes that exceed the idle timeout."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
# Use a very short check interval so the reaper runs quickly
|
||||
monkeypatch.setattr(BoxliteProvider, "IDLE_CHECK_INTERVAL", 0.1)
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
# Acquire and release a box into the warm pool
|
||||
sid = provider.acquire("thread-1", user_id="u1")
|
||||
provider.release(sid)
|
||||
|
||||
assert sid in provider._warm_pool
|
||||
|
||||
# Backdate the warm-pool timestamp so it appears long-expired
|
||||
warm_box = provider._warm_pool[sid][0]
|
||||
provider._warm_pool[sid] = (warm_box, time.time() - 9999)
|
||||
|
||||
# Wait long enough for the reaper to detect and destroy it
|
||||
time.sleep(0.3)
|
||||
|
||||
# Box should be gone from warm pool and closed
|
||||
assert sid not in provider._warm_pool
|
||||
assert warm_box._closed
|
||||
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
# ── Task 7: Replica enforcement ───────────────────────────────────────
|
||||
|
||||
|
||||
def test_replica_enforcement_evicts_oldest_warm(monkeypatch):
|
||||
"""When warm pool exceeds replica limit, the oldest box is evicted."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config({"replicas": 2}),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
# Fill warm pool with 2 boxes from different threads
|
||||
sid_a = provider.acquire("thread-a", user_id="u1")
|
||||
provider.release(sid_a)
|
||||
|
||||
sid_b = provider.acquire("thread-b", user_id="u1")
|
||||
provider.release(sid_b)
|
||||
|
||||
assert len(provider._warm_pool) == 2
|
||||
assert sid_a in provider._warm_pool
|
||||
assert sid_b in provider._warm_pool
|
||||
|
||||
# Make sid_a definitely older by backdating its timestamp
|
||||
box_a = provider._warm_pool[sid_a][0]
|
||||
provider._warm_pool[sid_a] = (box_a, time.time() - 100)
|
||||
# Refresh sid_b's timestamp so it's newer
|
||||
box_b = provider._warm_pool[sid_b][0]
|
||||
provider._warm_pool[sid_b] = (box_b, time.time())
|
||||
|
||||
# Acquiring a third thread triggers replica enforcement:
|
||||
# warm pool count (2) >= replicas (2) → evict oldest (sid_a)
|
||||
sid_c = provider.acquire("thread-c", user_id="u1")
|
||||
|
||||
# Oldest (sid_a) should be evicted (gone from warm pool, closed)
|
||||
assert sid_a not in provider._warm_pool
|
||||
assert box_a._closed
|
||||
# Newer (sid_b) should remain in warm pool
|
||||
assert sid_b in provider._warm_pool
|
||||
# New box (sid_c) should be active
|
||||
assert sid_c in provider._boxes
|
||||
assert sid_c not in provider._warm_pool
|
||||
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
def test_replica_enforcement_counts_active_and_warm(monkeypatch):
|
||||
"""replicas caps active + warm boxes, not warm boxes alone."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config({"replicas": 2}),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
sid_active = provider.acquire("thread-active", user_id="u1")
|
||||
sid_warm = provider.acquire("thread-warm", user_id="u1")
|
||||
provider.release(sid_warm)
|
||||
warm_box = provider._warm_pool[sid_warm][0]
|
||||
|
||||
sid_new = provider.acquire("thread-new", user_id="u1")
|
||||
|
||||
assert sid_active in provider._boxes
|
||||
assert sid_new in provider._boxes
|
||||
assert sid_warm not in provider._warm_pool
|
||||
assert warm_box._closed
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
# ── Task 8: Shutdown and reset including warm pool ────────────────────
|
||||
|
||||
|
||||
def test_shutdown_stops_idle_reaper_and_destroys_all_boxes(monkeypatch):
|
||||
"""shutdown stops the idle reaper thread and destroys all active + warm boxes."""
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider.get_app_config",
|
||||
lambda: _stub_config(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"deerflow.community.boxlite.provider._import_simplebox",
|
||||
lambda: _FakeBox,
|
||||
)
|
||||
|
||||
provider = BoxliteProvider()
|
||||
provider._loop.run = _fake_run
|
||||
|
||||
# Create one active box (thread-1) and one warm pool box (thread-2 released)
|
||||
sid_active = provider.acquire("thread-1", user_id="u1")
|
||||
sid_warm = provider.acquire("thread-2", user_id="u1")
|
||||
provider.release(sid_warm)
|
||||
|
||||
assert sid_active in provider._boxes
|
||||
assert sid_warm in provider._warm_pool
|
||||
|
||||
# Get box references before shutdown
|
||||
box_active = provider._boxes[sid_active]
|
||||
box_warm = provider._warm_pool[sid_warm][0]
|
||||
|
||||
# Remember the idle checker thread
|
||||
checker_thread = provider._idle_checker_thread
|
||||
|
||||
provider.shutdown()
|
||||
|
||||
# Idle checker should be stopped
|
||||
assert provider._idle_checker_stop.is_set()
|
||||
assert checker_thread is not None
|
||||
assert not checker_thread.is_alive()
|
||||
|
||||
# All boxes (active + warm) should be closed
|
||||
assert box_active._closed
|
||||
assert box_warm._closed
|
||||
|
||||
# All collections should be empty
|
||||
assert len(provider._boxes) == 0
|
||||
assert len(provider._warm_pool) == 0
|
||||
assert len(provider._thread_boxes) == 0
|
||||
|
||||
16
backend/tests/test_harness_packaging.py
Normal file
16
backend/tests/test_harness_packaging.py
Normal file
@ -0,0 +1,16 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def test_boxlite_is_optional_harness_dependency() -> None:
|
||||
"""BoxLite should not make core harness installs platform-dependent."""
|
||||
pyproject_path = Path(__file__).resolve().parents[1] / "packages" / "harness" / "pyproject.toml"
|
||||
pyproject = tomllib.loads(pyproject_path.read_text(encoding="utf-8"))
|
||||
|
||||
core_dependencies = pyproject["project"]["dependencies"]
|
||||
optional_dependencies = pyproject["project"]["optional-dependencies"]
|
||||
|
||||
assert not any(dep.startswith("boxlite") for dep in core_dependencies)
|
||||
assert any(dep.startswith("boxlite>=0.9.7") for dep in optional_dependencies["boxlite"])
|
||||
95
backend/tests/test_warm_pool_lifecycle.py
Normal file
95
backend/tests/test_warm_pool_lifecycle.py
Normal file
@ -0,0 +1,95 @@
|
||||
"""Unit tests for shared warm-pool lifecycle mechanics."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from deerflow.community.warm_pool_lifecycle import DEFAULT_IDLE_TIMEOUT, DEFAULT_REPLICAS, WarmPoolLifecycleMixin
|
||||
|
||||
|
||||
class _Provider(WarmPoolLifecycleMixin[str]):
|
||||
_idle_checker_thread_name = "test-warm-pool-reaper"
|
||||
|
||||
def __init__(self, *, replicas: int = DEFAULT_REPLICAS, idle_timeout: float = DEFAULT_IDLE_TIMEOUT, active_count: int = 0) -> None:
|
||||
self._lock = threading.Lock()
|
||||
self._warm_pool: dict[str, tuple[str, float]] = {}
|
||||
self._config: dict[str, Any] = {"replicas": replicas, "idle_timeout": idle_timeout}
|
||||
self._idle_checker_stop = threading.Event()
|
||||
self._idle_checker_thread: threading.Thread | None = None
|
||||
self.active_count = active_count
|
||||
self.destroyed: list[tuple[str, str, str]] = []
|
||||
|
||||
def _active_count_locked(self) -> int:
|
||||
return self.active_count
|
||||
|
||||
def _destroy_warm_entry(self, sandbox_id: str, entry: str, *, reason: str) -> None:
|
||||
self.destroyed.append((sandbox_id, entry, reason))
|
||||
|
||||
|
||||
def test_replica_count_includes_active_and_warm_entries() -> None:
|
||||
provider = _Provider(replicas=2, active_count=1)
|
||||
provider._warm_pool["warm-1"] = ("entry-1", time.time())
|
||||
|
||||
assert provider._replica_count() == (2, 2)
|
||||
|
||||
|
||||
def test_evict_oldest_warm_removes_and_destroys_oldest_entry() -> None:
|
||||
provider = _Provider()
|
||||
provider._warm_pool["new"] = ("entry-new", 200.0)
|
||||
provider._warm_pool["old"] = ("entry-old", 100.0)
|
||||
|
||||
evicted = provider._evict_oldest_warm()
|
||||
|
||||
assert evicted == "old"
|
||||
assert "old" not in provider._warm_pool
|
||||
assert "new" in provider._warm_pool
|
||||
assert provider.destroyed == [("old", "entry-old", "replica_enforcement")]
|
||||
|
||||
|
||||
def test_evict_oldest_warm_returns_none_when_pool_empty() -> None:
|
||||
provider = _Provider()
|
||||
|
||||
assert provider._evict_oldest_warm() is None
|
||||
assert provider.destroyed == []
|
||||
|
||||
|
||||
def test_reap_expired_warm_destroys_only_expired_entries() -> None:
|
||||
provider = _Provider()
|
||||
now = time.time()
|
||||
provider._warm_pool["expired"] = ("entry-expired", now - 100)
|
||||
provider._warm_pool["fresh"] = ("entry-fresh", now)
|
||||
|
||||
provider._reap_expired_warm(idle_timeout=10)
|
||||
|
||||
assert "expired" not in provider._warm_pool
|
||||
assert "fresh" in provider._warm_pool
|
||||
assert provider.destroyed == [("expired", "entry-expired", "idle_timeout")]
|
||||
|
||||
|
||||
def test_reap_expired_warm_noops_when_timeout_disabled() -> None:
|
||||
provider = _Provider(idle_timeout=0)
|
||||
provider._warm_pool["expired"] = ("entry-expired", time.time() - 100)
|
||||
|
||||
provider._reap_expired_warm(idle_timeout=0)
|
||||
|
||||
assert "expired" in provider._warm_pool
|
||||
assert provider.destroyed == []
|
||||
|
||||
|
||||
def test_start_idle_checker_uses_monkeypatchable_interval(monkeypatch) -> None:
|
||||
provider = _Provider(idle_timeout=0.01)
|
||||
monkeypatch.setattr(_Provider, "IDLE_CHECK_INTERVAL", 0.01)
|
||||
provider._warm_pool["expired"] = ("entry-expired", time.time() - 10)
|
||||
|
||||
provider._start_idle_checker()
|
||||
deadline = time.time() + 1
|
||||
while "expired" in provider._warm_pool and time.time() < deadline:
|
||||
time.sleep(0.01)
|
||||
provider._stop_idle_checker()
|
||||
|
||||
assert "expired" not in provider._warm_pool
|
||||
assert provider.destroyed == [("expired", "entry-expired", "idle_timeout")]
|
||||
assert provider._idle_checker_thread is not None
|
||||
assert not provider._idle_checker_thread.is_alive()
|
||||
17
backend/uv.lock
generated
17
backend/uv.lock
generated
@ -485,6 +485,17 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/95/c1/84fc6811122f54b20de2e5afb312ee07a3a47a328755587d1e505475239b/blockbuster-1.5.26-py3-none-any.whl", hash = "sha256:f8e53fb2dd4b6c6ec2f04907ddbd063ca7cd1ef587d24448ef4e50e81e3a79bb", size = 13226, upload-time = "2025-12-05T10:43:48.778Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "boxlite"
|
||||
version = "0.9.7"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/58/77/34fe448e7a1411b95c33df673305d64bacdfb031a5faae9faec82a32eaf8/boxlite-0.9.7-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:715ba342a80432279115ad41edf42b92993134402a7f4c73f23eb1327aa7ec29", size = 35518202, upload-time = "2026-07-01T12:55:41.757Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/02/2f/8ebf72c81afd07b3779ce206e85b32cf7b7baed98b811e966508ab9509cc/boxlite-0.9.7-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:01a2dd8b871213beb037a0c332bd94d9568aea7db0c1d39385ae789b73610055", size = 37616271, upload-time = "2026-07-01T12:55:45.01Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5f/32/6d9d8e9dca8f6e387dfe874a6872b49529b7795b75e814b2a75015a66036/boxlite-0.9.7-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:2281d4613af8f56e05856883aac2cbaf0028d50895a67299cd4932585661f13a", size = 33781989, upload-time = "2026-07-01T12:55:47.944Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e0/79/068b8a7de1fc9724d89ca7086754f0e6c6c80f8c74f9d95267006c49fe33/boxlite-0.9.7-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:b8e68f2efe493112445f3150adac74de592974ba9c05025440e0ddf54870bc53", size = 35519183, upload-time = "2026-07-01T12:55:50.896Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bracex"
|
||||
version = "2.7"
|
||||
@ -885,6 +896,9 @@ dependencies = [
|
||||
]
|
||||
|
||||
[package.optional-dependencies]
|
||||
boxlite = [
|
||||
{ name = "boxlite" },
|
||||
]
|
||||
ollama = [
|
||||
{ name = "langchain-ollama" },
|
||||
]
|
||||
@ -911,6 +925,7 @@ requires-dist = [
|
||||
{ name = "aiosqlite", specifier = ">=0.19" },
|
||||
{ name = "alembic", specifier = ">=1.13" },
|
||||
{ name = "asyncpg", marker = "extra == 'postgres'", specifier = ">=0.29" },
|
||||
{ name = "boxlite", marker = "extra == 'boxlite'", specifier = ">=0.9.7" },
|
||||
{ name = "croniter", specifier = ">=6.0.0" },
|
||||
{ name = "cryptography", specifier = ">=48.0.1" },
|
||||
{ name = "ddgs", specifier = ">=9.10.0" },
|
||||
@ -950,7 +965,7 @@ requires-dist = [
|
||||
{ name = "textual", marker = "extra == 'tui'", specifier = ">=0.80" },
|
||||
{ name = "tiktoken", specifier = ">=0.8.0" },
|
||||
]
|
||||
provides-extras = ["redis", "tui", "groundroute", "ollama", "postgres", "pymupdf"]
|
||||
provides-extras = ["tui", "groundroute", "ollama", "postgres", "redis", "pymupdf", "boxlite"]
|
||||
|
||||
[[package]]
|
||||
name = "defusedxml"
|
||||
|
||||
@ -1086,7 +1086,32 @@ sandbox:
|
||||
# # API_KEY: $MY_API_KEY # Reads from host's MY_API_KEY env var
|
||||
# # DATABASE_URL: $DATABASE_URL # Reads from host's DATABASE_URL env var
|
||||
|
||||
# Option 3: Provisioner-managed AIO Sandbox (docker-compose-dev)
|
||||
# Option 3: BoxLite micro-VM Sandbox
|
||||
# Runs each sandbox as a BoxLite micro-VM. Released boxes stay in an in-process
|
||||
# warm pool and can be reclaimed by the same user/thread without a cold start.
|
||||
# Requires the boxlite runtime and host virtualization support (KVM on Linux,
|
||||
# Hypervisor.framework on macOS).
|
||||
# sandbox:
|
||||
# use: deerflow.community.boxlite:BoxliteProvider
|
||||
# image: python:3.12-slim
|
||||
#
|
||||
# # Optional: Per-box memory and CPU limits.
|
||||
# # memory_mib: 1024
|
||||
# # cpus: 2
|
||||
#
|
||||
# # Optional: Maximum active + warm BoxLite VMs per gateway process (default: 3).
|
||||
# # Active boxes are never evicted; only warm-pool boxes are stopped to make room.
|
||||
# # replicas: 3
|
||||
#
|
||||
# # Optional: Seconds before an idle warm-pool VM is stopped (default: 600).
|
||||
# # Set to 0 to keep warm VMs until shutdown or replica eviction.
|
||||
# # idle_timeout: 600
|
||||
#
|
||||
# # Optional: Environment variables to inject into every command.
|
||||
# # environment:
|
||||
# # PYTHONUNBUFFERED: "1"
|
||||
#
|
||||
# Option 4: Provisioner-managed AIO Sandbox (docker-compose-dev)
|
||||
# Each sandbox_id gets a dedicated Pod in k3s, managed by the provisioner.
|
||||
# Recommended for production or advanced users who want better isolation and scalability.:
|
||||
# sandbox:
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user