fix: enforce custom agent skill allowlists in sandboxes (#5077)

* fix: enforce agent skill allowlists in sandboxes

* fix: guard E2B skill projection resets

* fix: preserve agent skill isolation across delegation

* fix: close sandbox skill isolation bypasses

* fix(sandbox): close skill isolation review gaps

* fix(sandbox): harden skill isolation lifecycle
This commit is contained in:
Aari 2026-08-31 14:35:08 +08:00 committed by GitHub
parent 42796d7086
commit 317577e285
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
39 changed files with 3380 additions and 261 deletions

View File

@ -853,6 +853,8 @@ When you install `.skill` archives through the Gateway, DeerFlow accepts standar
Disabling a skill also removes it from the sandbox filesystem view, so shell commands and structured file tools follow the same enabled state. Local, Docker/AIO, hostPath provisioner, and newly created E2B sandboxes source `/mnt/skills` from enabled-only projections that update when public, custom, legacy, or managed integration skills are toggled, edited, created, deleted, or installed. Structured `read_file` calls (including line ranges and read-before-write checks) use the sandbox provider's mount mapping, so the user identity captured when the sandbox was acquired remains authoritative. Managed integration packages remain shared, while their projected filesystem visibility follows each user's enabled state. Multi-worker Gateways re-read on-disk enable state while rebuilding user projections, so a toggle handled by one worker is honored by another worker's next sandbox acquire. Existing E2B sandboxes retain their creation-time snapshot until they are recreated. PVC-backed provisioner skills keep their configured PVC snapshot/layout for now; dynamic PVC materialization is tracked separately.
For `LocalSandboxProvider`, this is a managed tool-path boundary rather than host filesystem isolation. Explicit per-Agent skill policies are accepted only while host bash is disabled (the default), because a host subprocess can address canonical paths without using the provider's virtual-path mappings. Use Docker/AIO, the Kubernetes provisioner, or E2B when the filesystem boundary must remain enforceable alongside shell access.
Managed integrations install shared read-only skill packs without mixing them
into custom skills. The Lark/Feishu CLI integration is available under
`Settings → Integrations → Lark / Feishu CLI`; an administrator installs or
@ -1205,6 +1207,12 @@ The Hash counts remote VMs and in-flight creates, repairs interrupted creates
from E2B metadata, grace-protects stale inventory omissions, and blocks new
creates while Redis or initial inventory is unavailable. Run Redis with persistence, non-evicting memory, and HA.
E2B snapshots `skills.container_path` when the provider starts and includes the
canonical root in its thread identity, warm-pool seed, and remote metadata. A
VM created for a different root is never adopted; reconciliation reaps it after
the configured grace period once no live peer owns it. Restart the Gateway after
changing the root.
E2B acquisition uses a bounded executor. Waiting acquisitions do not use the
default asyncio executor.

View File

@ -544,10 +544,13 @@ provider in `config.yaml`.
Notes specific to `E2BSandboxProvider`:
- Each DeerFlow thread is bound to its E2B sandbox via metadata
(`deer_flow_user`, `deer_flow_thread`). Startup and periodic reconciliation
probe every bounded candidate, adopt one healthy canonical sandbox, and reap
duplicates after a grace period. Provider-tagged entries without a complete
user/thread identity are reaped only after the orphan TTL.
(`deer_flow_user`, `deer_flow_thread`, `deer_flow_skills_root`). Startup and
periodic reconciliation probe every bounded candidate, adopt one healthy
canonical sandbox, and reap duplicates after a grace period. A sandbox whose
skills root differs from the provider's startup snapshot is never adopted and
is reaped after the same grace period once no live peer owns it.
Provider-tagged entries without a complete user/thread identity are reaped
only after the orphan TTL.
- Ownership leases prevent one gateway from adopting or destroying a sandbox
another live gateway is responsible for. The default in-memory store is safe
only for one gateway process. Multi-worker/load-balanced deployments must use
@ -752,6 +755,15 @@ skills:
container_path: /mnt/skills
```
For the AIO provider (including the Kubernetes provisioner) and E2B,
`skills.container_path` is captured when the provider starts and must be one
canonical absolute, non-root POSIX path. Do not use redundant separators,
`.`/`..`, or a path that contains or sits below DeerFlow's reserved mounts
(`/mnt/user-data`, `/mnt/acp-workspace`, or `/mnt/integrations/lark-cli`).
Restart the Gateway after changing it so sandbox identities and mounts use the
same root. E2B also records the root in remote metadata and refuses to adopt a
VM created for another root.
**How Skills Work**:
- Skills are stored in `deer-flow/skills/{public,custom}/`
- Each skill has a `SKILL.md` file with metadata
@ -777,6 +789,12 @@ This field is a discovery and activation allowlist; it does not activate every l
The same semantics apply to `subagents.agents.<name>.skills` and `subagents.custom_agents.<name>.skills`: omitted or `null` exposes all enabled skills, `[]` exposes none, and a list limits discovery and activation. A passive subagent skill never removes baseline tools; its `allowed-tools` declaration becomes active only after slash activation or a completed `SKILL.md` read.
`LocalSandboxProvider` enforces this filesystem view through its managed virtual
path mappings only. Explicit per-Agent skill policies therefore fail closed when
`sandbox.allow_host_bash` is enabled, because host subprocesses can bypass those
mappings. Keep host bash disabled (the default), or use AIO/provisioner/E2B when
shell access and filesystem isolation are both required.
### Title Generation
Automatic conversation title generation:

View File

@ -97,6 +97,13 @@ does not interrupt active filesystem or E2B SDK calls.
The provider checks mount limits before upload. It rechecks each opened file descriptor against its preflight size before SDK upload.
For policy-scoped turns, clearing the four managed remote skill categories and
uploading their prepared projection is one per-user/thread/skills-root critical
section, shared with acquire and release. The provider snapshots that canonical
root at startup and carries it through warm-pool identity and E2B metadata; a VM
from another root is never adopted. A second policy sync cannot reset the remote
tree until the first upload pass has completed.
An invalid mount does not block later mounts.
Each successful upload logs its source, destination, file count, byte count, and elapsed time.

View File

@ -461,6 +461,7 @@ def build_middlewares(
custom_middlewares: list[AgentMiddleware] | None = None,
*,
available_skills: set[str] | None = None,
owns_agent_skill_projection: bool = True,
app_config: AppConfig | None = None,
deferred_setup=None,
mcp_routing_middleware: AgentMiddleware | None = None,
@ -481,6 +482,9 @@ def build_middlewares(
model_name: Resolved runtime model name; gates vision-only middleware.
agent_name: If provided, MemoryMiddleware will use per-agent memory storage.
custom_middlewares: Optional list of custom middlewares to inject into the chain.
owns_agent_skill_projection: Whether this lead middleware chain owns the
thread's physical skill projection. Prompt-only bootstrap agents do
not; their narrow skill set must not replace the thread view.
app_config: Explicit AppConfig; falls back to ``get_app_config()`` when omitted.
deferred_setup: Optional deferred-MCP-tool setup that attaches
``DeferredToolFilterMiddleware`` when ``tool_search`` is enabled.
@ -506,6 +510,10 @@ def build_middlewares(
"app_config": resolved_app_config,
"lazy_init": True,
}
if available_skills is not None:
runtime_middleware_kwargs["available_skills"] = available_skills
if not owns_agent_skill_projection:
runtime_middleware_kwargs["owns_agent_skill_projection"] = False
if authorization_provider is not None:
runtime_middleware_kwargs["authorization_provider"] = authorization_provider
if authorization_provider is not None and deferred_setup is not None:
@ -1025,6 +1033,7 @@ def _assemble_lead_agent(config: RunnableConfig, *, app_config: AppConfig) -> Le
model_name=model_name,
agent_name=agent_name,
available_skills=set(_BOOTSTRAP_SKILL_NAMES),
owns_agent_skill_projection=False,
app_config=resolved_app_config,
deferred_setup=setup,
mcp_routing_middleware=mcp_routing_middleware,

View File

@ -49,7 +49,11 @@ it to that middleware's declaration in the same change.
output wording.
4. **ThreadDataMiddleware** - Creates per-thread directories under the user's isolation scope (`backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/{workspace,uploads,outputs}`); resolves identity via `resolve_runtime_user_id(runtime)`, including Gateway runtime context and standalone LangGraph Server auth, then falls back to the request ContextVar / `"default"`
5. **UploadsMiddleware** - Tracks and injects newly uploaded files into conversation (lead agent only); upload existence checks use the same runtime-resolved user bucket as thread-data creation
6. **SandboxMiddleware** - Acquires sandbox, stores `sandbox_id` in state
6. **SandboxMiddleware** - Acquires sandbox, stores `sandbox_id` in state. The
lead runtime normally owns the thread's physical Agent-skill projection;
delegated subagents and the prompt-only bootstrap agent are non-owners, so
their narrower discovery allowlists never rebuild the shared thread view or
force eager sandbox acquisition.
7. **DanglingToolCallMiddleware** - Injects placeholder ToolMessages for AIMessage tool_calls that lack responses (e.g., user interruption), preserving raw provider tool-call payloads in `additional_kwargs["tool_calls"]`; malformed tool-call names and arguments are sanitized in the model-bound request so strict OpenAI-compatible providers do not reject the next request
8. **LLMErrorHandlingMiddleware** - Normalizes provider/model invocation failures into recoverable assistant-facing errors before later stages run
9. **Authorization / GuardrailMiddleware** - Up to two independent pre-tool-call gates run here. When `authorization.enabled`, the `AuthorizationProvider` instance already used for Layer 1 capability filtering is wrapped by `GuardrailAuthorizationAdapter` and reused for Layer 2 execution checks. A generated `tool_search` bypasses the adapter's second provider call only when the current build has a concrete deferred setup; its catalog was already filtered by Layer 1, and an ordinary same-named tool without that deferred setup receives no exemption. When `guardrails.enabled`, the explicitly configured `GuardrailProvider` is appended after authorization and still evaluates every call, including `tool_search`. Authorization therefore runs outermost and can deny before an external guardrail call; both use the existing middleware's fail-closed, audit, sync/async, and error-`ToolMessage` behavior. See the authorization RFC and [docs/GUARDRAILS.md](../../../../../docs/GUARDRAILS.md).

View File

@ -167,6 +167,8 @@ def _build_runtime_middlewares(
receipts_render_mode: str = "delegation_only",
authorization_provider=None,
authorization_infrastructure_tool_names: frozenset[str] = frozenset(),
available_skills: set[str] | None = None,
owns_agent_skill_projection: bool = True,
) -> list[AgentMiddleware]:
"""Build shared base middlewares for agent execution."""
from deerflow.agents.middlewares.input_sanitization_middleware import InputSanitizationMiddleware
@ -199,7 +201,13 @@ def _build_runtime_middlewares(
from deerflow.agents.middlewares.uploads_middleware import UploadsMiddleware
thread_hooks.append(UploadsMiddleware())
thread_hooks.append(SandboxMiddleware(lazy_init=lazy_init))
thread_hooks.append(
SandboxMiddleware(
lazy_init=lazy_init,
available_skills=available_skills,
owns_agent_skill_projection=owns_agent_skill_projection,
)
)
# Layer 3 — post-processing append-only middlewares.
tail: list[AgentMiddleware] = []
@ -313,6 +321,8 @@ def build_lead_runtime_middlewares(
lazy_init: bool = True,
authorization_provider=None,
deferred_setup: "DeferredToolSetup | None" = None,
available_skills: set[str] | None = None,
owns_agent_skill_projection: bool = True,
) -> list[AgentMiddleware]:
"""Middlewares shared by lead agent runtime before lead-only middlewares."""
return _build_runtime_middlewares(
@ -324,6 +334,8 @@ def build_lead_runtime_middlewares(
# results (default "delegation_only"); stamping stays always-on.
receipts_render_mode=app_config.verification.receipts_render_mode,
authorization_provider=authorization_provider,
available_skills=available_skills,
owns_agent_skill_projection=owns_agent_skill_projection,
authorization_infrastructure_tool_names=(frozenset({deferred_setup.tool_search_tool.name}) if authorization_provider is not None and deferred_setup is not None and deferred_setup.tool_search_tool is not None else frozenset()),
)
@ -361,6 +373,7 @@ def build_subagent_runtime_middlewares(
receipts_render_mode="always",
authorization_provider=authorization_provider,
authorization_infrastructure_tool_names=(frozenset({deferred_setup.tool_search_tool.name}) if authorization_provider is not None and deferred_setup is not None and deferred_setup.tool_search_tool is not None else frozenset()),
owns_agent_skill_projection=False,
)
# Enabled/configured skills are discoverable metadata, not automatically

View File

@ -13,6 +13,7 @@ The provider itself handles:
import asyncio
import atexit
import contextlib
import hashlib
import logging
import os
import signal
@ -37,6 +38,7 @@ from deerflow.community.warm_pool_lifecycle import (
)
from deerflow.config import get_app_config
from deerflow.config.paths import VIRTUAL_PATH_PREFIX, get_paths, join_host_path
from deerflow.constants import DEFAULT_SKILLS_CONTAINER_PATH
from deerflow.integrations.lark_cli import INTEGRATION_ID as LARK_CLI_INTEGRATION_ID
from deerflow.integrations.lark_cli import LARK_CLI_SANDBOX_CONFIG_DIR, LARK_CLI_SANDBOX_DATA_DIR, LARK_CLI_SANDBOX_LOCKS_DIR, LARK_CLI_SANDBOX_RUNTIME_DIR, ensure_lark_cli_credential_tree, lark_skills_installed
from deerflow.runtime.user_context import get_effective_user_id
@ -44,6 +46,7 @@ from deerflow.sandbox.acquire_serialization import AcquireSerializer
from deerflow.sandbox.identity import derive_sandbox_scope_token
from deerflow.sandbox.sandbox import Sandbox
from deerflow.sandbox.sandbox_provider import SandboxProvider
from deerflow.skills.types import SkillCategory
from .aio_sandbox import AioSandbox
from .backend import SANDBOX_LOCAL_PROVIDER_READY_TIMEOUT, SandboxBackend, wait_for_sandbox_ready, wait_for_sandbox_ready_async
@ -57,7 +60,7 @@ from .ownership import (
make_sandbox_ownership_store,
resolve_ownership_config,
)
from .remote_backend import RemoteSandboxBackend
from .remote_backend import RemoteSandboxBackend, _normalize_skills_container_path
from .sandbox_info import SandboxInfo
logger = logging.getLogger(__name__)
@ -143,6 +146,8 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
API_KEY: $MY_API_KEY
"""
supports_agent_skill_isolation = True
# How long `_held_teardown_lease` waits for its heartbeat thread to exit
# before deferring the final lease release to that (still-running) thread.
# The store's socket timeout bounds each operation, but context exit can
@ -265,6 +270,13 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
idle_timeout = getattr(sandbox_config, "idle_timeout", None)
replicas = getattr(sandbox_config, "replicas", None)
configured_skills_path = getattr(
getattr(config, "skills", None),
"container_path",
None,
)
if not isinstance(configured_skills_path, str):
configured_skills_path = DEFAULT_SKILLS_CONTAINER_PATH
return {
"image": sandbox_config.image or DEFAULT_IMAGE,
@ -283,6 +295,9 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
# provisioner URL for dynamic pod management (e.g. http://provisioner:8002)
"provisioner_url": getattr(sandbox_config, "provisioner_url", None) or "",
"provisioner_api_key": getattr(sandbox_config, "provisioner_api_key", None) or "",
"skills_container_path": _normalize_skills_container_path(
configured_skills_path,
),
}
@staticmethod
@ -723,6 +738,32 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
"""
return derive_sandbox_scope_token(user_id=user_id, thread_id=thread_id)
@staticmethod
def _thread_skill_projection_active(thread_id: str, user_id: str) -> bool:
return get_paths().thread_skills_view_dir(thread_id, user_id=user_id).exists()
@staticmethod
def _policy_scoped_sandbox_id(
thread_id: str,
user_id: str,
skills_container_path: str,
) -> str:
"""Return a root-aware domain-separated identity for a policy sandbox."""
normalized_root = _normalize_skills_container_path(skills_container_path)
seed = b"agent-skills-v2\0" + user_id.encode() + b"\0" + thread_id.encode() + b"\0" + normalized_root.encode()
return hashlib.sha256(seed).hexdigest()[:16]
@staticmethod
def _custom_root_sandbox_id(
thread_id: str,
user_id: str,
skills_container_path: str,
) -> str:
"""Return an identity for a shared-view sandbox at a custom root."""
normalized_root = _normalize_skills_container_path(skills_container_path)
seed = b"skills-root-v1\0" + user_id.encode() + b"\0" + thread_id.encode() + b"\0" + normalized_root.encode()
return hashlib.sha256(seed).hexdigest()[:16]
def _assert_active_identity_available_locked(
self,
sandbox_id: str,
@ -758,17 +799,37 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
def _get_extra_mounts(self, thread_id: str | None, *, user_id: str | None = None) -> list[tuple[str, str, bool]]:
"""Collect all extra mounts for a sandbox (thread-specific + skills)."""
mounts: list[tuple[str, str, bool]] = []
skills_container_path = self._configured_skills_container_path()
if thread_id:
mounts.extend(self._get_thread_mounts(thread_id, user_id=user_id))
logger.info(f"Adding thread mounts for thread {thread_id}: {mounts}")
skills_mounts = self._get_skills_mounts(user_id=user_id)
skills_mounts = self._get_skills_mounts(
thread_id,
user_id=user_id,
skills_container_path=skills_container_path,
)
if skills_mounts:
mounts.extend(skills_mounts)
logger.info(f"Adding skills mounts: {skills_mounts}")
user_skill_mounts = self._get_user_skill_mounts(user_id=user_id)
effective_user_id = self._effective_acquire_user_id(user_id)
thread_projection_active = bool(
thread_id
and self._thread_skill_projection_active(
thread_id,
effective_user_id,
)
)
user_skill_mounts = (
[]
if thread_projection_active
else self._get_user_skill_mounts(
user_id=user_id,
skills_container_path=skills_container_path,
)
)
if user_skill_mounts:
mounts.extend(user_skill_mounts)
logger.info(f"Adding user skill mounts: {user_skill_mounts}")
@ -780,6 +841,34 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
return self._dedupe_mounts_by_container_path(mounts)
def _local_config_mount_exclusion_root(
self,
thread_id: str | None,
*,
user_id: str,
) -> str | None:
"""Return the skills subtree owned by a policy-scoped local sandbox."""
if not isinstance(self._backend, LocalContainerBackend) or not thread_id:
return None
if not self._thread_skill_projection_active(thread_id, user_id):
return None
return self._configured_skills_container_path()
def _configured_skills_container_path(self) -> str:
"""Return the provider-startup skills root used by IDs and mounts."""
# A few mount-helper callers intentionally construct an uninitialized
# provider. Production instances always use the startup snapshot, while
# that narrow compatibility path loads the same validated value lazily.
config = getattr(self, "_config", None)
if not isinstance(config, dict):
config = self._load_config()
return _normalize_skills_container_path(
config.get(
"skills_container_path",
DEFAULT_SKILLS_CONTAINER_PATH,
)
)
@staticmethod
def _dedupe_mounts_by_container_path(mounts: list[tuple[str, str, bool]]) -> list[tuple[str, str, bool]]:
"""Keep the first mount for each container path.
@ -825,7 +914,12 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
]
@staticmethod
def _get_skills_mounts(*, user_id: str | None = None) -> list[tuple[str, str, bool]]:
def _get_skills_mounts(
thread_id: str | None = None,
*,
user_id: str | None = None,
skills_container_path: str | None = None,
) -> list[tuple[str, str, bool]]:
"""Get skills directory mount configurations for three-way skills layout.
Mirrors ``LocalSandboxProvider._build_thread_path_mappings`` for AIO
@ -841,12 +935,30 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
mounts: list[tuple[str, str, bool]] = []
try:
config = get_app_config()
container_path = config.skills.container_path
container_path = _normalize_skills_container_path(skills_container_path or config.skills.container_path)
effective_user_id = AioSandboxProvider._effective_acquire_user_id(user_id)
AioSandboxProvider._ensure_skills_projection(effective_user_id)
paths = get_paths()
host_base_dir = str(paths.host_base_dir)
if thread_id and AioSandboxProvider._thread_skill_projection_active(
thread_id,
effective_user_id,
):
host_root = paths.host_thread_skills_view_dir(
thread_id,
user_id=effective_user_id,
)
return [
(
join_host_path(host_root, category.value),
f"{container_path}/{category.value}",
True,
)
for category in SkillCategory
]
AioSandboxProvider._ensure_skills_projection(effective_user_id)
# 1. Public skills: global, read-only — static, shared by all threads
mounts.append(
(
@ -907,7 +1019,11 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
return None
@staticmethod
def _get_user_skill_mounts(*, user_id: str | None = None) -> list[tuple[str, str, bool]]:
def _get_user_skill_mounts(
*,
user_id: str | None = None,
skills_container_path: str | None = None,
) -> list[tuple[str, str, bool]]:
"""Mount enabled managed integration skills into AIO sandboxes.
Per-user custom skills are already mounted by ``_get_skills_mounts``.
@ -917,7 +1033,7 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
try:
config = get_app_config()
paths = get_paths()
skills_container_path = config.skills.container_path
resolved_skills_container_path = _normalize_skills_container_path(skills_container_path or config.skills.container_path)
effective_user_id = AioSandboxProvider._effective_acquire_user_id(user_id)
AioSandboxProvider._ensure_skills_projection(effective_user_id)
return [
@ -929,7 +1045,7 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
"skills_view",
"integrations",
),
f"{skills_container_path}/integrations",
f"{resolved_skills_container_path}/integrations",
True,
),
]
@ -1275,7 +1391,26 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
def _sandbox_id_for_thread(self, thread_id: str | None, user_id: str | None) -> str:
"""Return deterministic IDs for thread sandboxes and random IDs otherwise."""
return self._deterministic_sandbox_id(thread_id, self._effective_acquire_user_id(user_id)) if thread_id else str(uuid.uuid4())[:8]
if not thread_id:
return str(uuid.uuid4())[:8]
effective_user_id = self._effective_acquire_user_id(user_id)
skills_container_path = self._configured_skills_container_path()
if self._thread_skill_projection_active(thread_id, effective_user_id):
return self._policy_scoped_sandbox_id(
thread_id,
effective_user_id,
skills_container_path,
)
# Preserve the historic deterministic ID for the default root while
# preventing a custom-root Pod/container from being reused after the
# configured mount destination changes.
if skills_container_path != DEFAULT_SKILLS_CONTAINER_PATH:
return self._custom_root_sandbox_id(
thread_id,
effective_user_id,
skills_container_path,
)
return self._deterministic_sandbox_id(thread_id, effective_user_id)
def _reuse_in_process_sandbox(self, thread_id: str | None, *, user_id: str | None = None, post_lock: bool = False) -> str | None:
"""Reuse an active in-process sandbox for a thread if one is still tracked."""
@ -1284,22 +1419,44 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
effective_user_id = self._effective_acquire_user_id(user_id)
key = self._thread_key(thread_id, effective_user_id)
root_scoped_identity = (
self._thread_skill_projection_active(
thread_id,
effective_user_id,
)
or self._configured_skills_container_path() != DEFAULT_SKILLS_CONTAINER_PATH
)
expected_id = self._sandbox_id_for_thread(thread_id, effective_user_id)
stale_id: str | None = None
with self._lock:
if key not in self._thread_sandboxes:
return None
existing_id = self._thread_sandboxes[key]
if self._being_torn_down_locally(existing_id):
if root_scoped_identity and existing_id != expected_id:
stale_id = existing_id
elif self._being_torn_down_locally(existing_id):
# A reaper thread in this process is stopping this container.
# Same answer as a peer's `del:` lease: cold-start instead.
logger.info("Cached sandbox %s is being destroyed by this instance; not reusing it", existing_id)
return None
if existing_id in self._sandboxes:
elif existing_id in self._sandboxes:
info = self._sandbox_infos.get(existing_id)
else:
del self._thread_sandboxes[key]
return None
if stale_id is not None:
logger.info(
"Replacing sandbox %s with expected identity %s for user/thread %s/%s",
stale_id,
expected_id,
effective_user_id,
thread_id,
)
self.destroy(stale_id)
return None
alive = self._check_tracked_sandbox_alive(existing_id, info) if info is not None else True
if alive is False:
self._drop_unhealthy_sandbox(
@ -1975,6 +2132,10 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
extra_mounts = self._get_extra_mounts(thread_id, user_id=effective_user_id)
provision_lark_cli_runtime = self._lark_integration_active(effective_user_id)
provision_lark_cli_broker = self._lark_broker_active(effective_user_id)
config_mount_exclusion_root = self._local_config_mount_exclusion_root(
thread_id,
user_id=effective_user_id,
)
# Enforce replicas: only warm-pool containers count toward eviction budget.
# Active sandboxes are in use by live threads and must not be forcibly stopped.
@ -1983,6 +2144,11 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
evicted = self._evict_oldest_warm()
self._log_replicas_soft_cap(replicas, sandbox_id, evicted)
create_kwargs = {}
if config_mount_exclusion_root is not None:
create_kwargs["config_mount_exclusion_root"] = config_mount_exclusion_root
if isinstance(self._backend, RemoteSandboxBackend):
create_kwargs["skills_container_path"] = self._configured_skills_container_path()
info = self._backend.create(
thread_id,
sandbox_id,
@ -1990,6 +2156,7 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
user_id=effective_user_id,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
**create_kwargs,
)
# Wait for sandbox to be ready
@ -2009,6 +2176,11 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
extra_mounts = await asyncio.to_thread(self._get_extra_mounts, thread_id, user_id=effective_user_id)
provision_lark_cli_runtime = await asyncio.to_thread(self._lark_integration_active, effective_user_id)
provision_lark_cli_broker = await asyncio.to_thread(self._lark_broker_active, effective_user_id)
config_mount_exclusion_root = await asyncio.to_thread(
self._local_config_mount_exclusion_root,
thread_id,
user_id=effective_user_id,
)
# Enforce replicas: only warm-pool containers count toward eviction budget.
# Active sandboxes are in use by live threads and must not be forcibly stopped.
@ -2017,6 +2189,11 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
evicted = await asyncio.to_thread(self._evict_oldest_warm)
self._log_replicas_soft_cap(replicas, sandbox_id, evicted)
create_kwargs = {}
if config_mount_exclusion_root is not None:
create_kwargs["config_mount_exclusion_root"] = config_mount_exclusion_root
if isinstance(self._backend, RemoteSandboxBackend):
create_kwargs["skills_container_path"] = self._configured_skills_container_path()
info = await asyncio.to_thread(
self._backend.create,
thread_id,
@ -2025,6 +2202,7 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
user_id=effective_user_id,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
**create_kwargs,
)
# Wait for sandbox to be ready without blocking the event loop.

View File

@ -11,6 +11,7 @@ import ipaddress
import json
import logging
import os
import posixpath
import shlex
import socket
import subprocess
@ -522,6 +523,7 @@ class LocalContainerBackend(SandboxBackend):
sandbox_id: str,
extra_mounts: list[tuple[str, str, bool]] | None = None,
*,
config_mount_exclusion_root: str | None = None,
user_id: str | None = None,
provision_lark_cli_runtime: bool = False,
provision_lark_cli_broker: bool = False,
@ -532,6 +534,10 @@ class LocalContainerBackend(SandboxBackend):
thread_id: Thread ID for which the sandbox is being created. Useful for backends that want to organize sandboxes by thread.
sandbox_id: Deterministic sandbox identifier (used in container name).
extra_mounts: Additional volume mounts as (host_path, container_path, read_only) tuples.
config_mount_exclusion_root: Exclude config-level mounts at or
below this container path. Policy-scoped skill projections use
this to prevent a nested operator mount from overlaying an
excluded skill back into the restricted view.
user_id: User bucket already reflected in extra_mounts. Accepted for
interface compatibility with remote backends.
provision_lark_cli_runtime: Ignored the local backend provisions the
@ -559,7 +565,12 @@ class LocalContainerBackend(SandboxBackend):
for _attempt in range(10):
port = get_free_port(start_port=_next_start)
try:
container_id = self._start_container(container_name, port, extra_mounts)
container_id = self._start_container(
container_name,
port,
extra_mounts,
config_mount_exclusion_root=config_mount_exclusion_root,
)
break
except RuntimeError as exc:
release_port(port)
@ -788,6 +799,8 @@ class LocalContainerBackend(SandboxBackend):
container_name: str,
port: int,
extra_mounts: list[tuple[str, str, bool]] | None = None,
*,
config_mount_exclusion_root: str | None = None,
) -> str:
"""Start a new container.
@ -795,6 +808,8 @@ class LocalContainerBackend(SandboxBackend):
container_name: Name for the container.
port: Host port to map to container port 8080.
extra_mounts: Additional volume mounts.
config_mount_exclusion_root: Config-level mounts at or below this
container root are omitted for this container only.
Returns:
The container ID.
@ -957,8 +972,21 @@ class LocalContainerBackend(SandboxBackend):
for key, value in self._environment.items():
cmd.extend(["-e", f"{key}={value}"])
# Config-level volume mounts
# Config-level volume mounts. A policy-scoped skills view owns its
# complete container subtree; keeping a more-specific config mount
# would let Docker overlay an excluded skill inside that view.
exclusion_root = None
if config_mount_exclusion_root is not None:
exclusion_root = posixpath.normpath(config_mount_exclusion_root.rstrip("/") or "/")
for mount in self._config_mounts:
mount_path = posixpath.normpath(str(mount.container_path).rstrip("/") or "/")
if exclusion_root is not None and (mount_path == exclusion_root or mount_path.startswith(exclusion_root.rstrip("/") + "/")):
logger.info(
"Skipping config mount inside policy-scoped skills root: %s",
mount.container_path,
)
continue
cmd.extend(
_format_container_mount(
self._runtime,

View File

@ -18,9 +18,12 @@ Architecture:
from __future__ import annotations
import logging
import posixpath
from pathlib import PurePosixPath
import requests
from deerflow.constants import DEFAULT_SKILLS_CONTAINER_PATH
from deerflow.runtime.user_context import get_effective_user_id
from deerflow.skills.storage import user_should_see_legacy_skills
@ -31,22 +34,57 @@ logger = logging.getLogger(__name__)
_PROVISIONER_EXTRA_MOUNT_PATHS = {
"/mnt/acp-workspace",
"/mnt/skills/custom",
"/mnt/skills/integrations",
"/mnt/integrations/lark-cli/config",
"/mnt/integrations/lark-cli/config/locks",
"/mnt/integrations/lark-cli/data",
"/mnt/integrations/lark-cli/runtime",
}
_MANAGED_SKILL_CATEGORY_NAMES = (
"public",
"custom",
"legacy",
"integrations",
)
_RESERVED_SANDBOX_MOUNT_PATHS = (
"/mnt/user-data",
"/mnt/acp-workspace",
"/mnt/integrations/lark-cli",
)
_LARK_CLI_RUNTIME_CONTAINER_PATH = "/mnt/integrations/lark-cli/runtime"
_LARK_CLI_CONFIG_CONTAINER_PATH = "/mnt/integrations/lark-cli/config"
_LARK_CLI_DATA_CONTAINER_PATH = "/mnt/integrations/lark-cli/data"
def _normalize_skills_container_path(container_path: str) -> str:
"""Return a canonical skills root that cannot overlap platform mounts."""
candidate = container_path
if not candidate or not candidate.startswith("/") or candidate.startswith("//"):
raise ValueError("The skills container path must be an absolute non-root path")
normalized = posixpath.normpath(candidate)
if normalized != candidate:
raise ValueError("The skills container path must not contain redundant separators, '.' or '..'")
root = PurePosixPath(normalized)
for reserved_path in _RESERVED_SANDBOX_MOUNT_PATHS:
reserved = PurePosixPath(reserved_path)
if root == reserved or root.is_relative_to(reserved) or reserved.is_relative_to(root):
raise ValueError(f"The skills container path {normalized!r} overlaps reserved sandbox path {reserved_path!r}")
return normalized
def _managed_skill_category_mount_paths(
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
) -> set[str]:
root = _normalize_skills_container_path(skills_container_path)
return {posixpath.join(root, category) for category in _MANAGED_SKILL_CATEGORY_NAMES}
def _provisioner_extra_mounts_payload(
extra_mounts: list[tuple[str, str, bool]] | None,
*,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
provision_lark_cli_runtime: bool = False,
provision_lark_cli_broker: bool = False,
) -> list[dict[str, object]]:
@ -67,6 +105,7 @@ def _provisioner_extra_mounts_payload(
available for the provisioner to place; the runtime entry is dropped in
both modes.
"""
allowed_paths = _PROVISIONER_EXTRA_MOUNT_PATHS | _managed_skill_category_mount_paths(skills_container_path)
if not extra_mounts:
return []
@ -74,7 +113,7 @@ def _provisioner_extra_mounts_payload(
payload: list[dict[str, object]] = []
for host_path, container_path, read_only in extra_mounts:
if container_path not in _PROVISIONER_EXTRA_MOUNT_PATHS:
if container_path not in allowed_paths:
continue
if drop_runtime and container_path == _LARK_CLI_RUNTIME_CONTAINER_PATH:
continue
@ -130,6 +169,7 @@ class RemoteSandboxBackend(SandboxBackend):
extra_mounts: list[tuple[str, str, bool]] | None = None,
*,
user_id: str | None = None,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
provision_lark_cli_runtime: bool = False,
provision_lark_cli_broker: bool = False,
) -> SandboxInfo:
@ -143,6 +183,7 @@ class RemoteSandboxBackend(SandboxBackend):
sandbox_id,
extra_mounts,
user_id=user_id,
skills_container_path=skills_container_path,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
)
@ -216,22 +257,26 @@ class RemoteSandboxBackend(SandboxBackend):
extra_mounts: list[tuple[str, str, bool]] | None = None,
*,
user_id: str | None = None,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
provision_lark_cli_runtime: bool = False,
provision_lark_cli_broker: bool = False,
) -> SandboxInfo:
"""POST /api/sandboxes → create Pod + Service."""
effective_user_id = user_id or get_effective_user_id()
include_legacy_skills = user_should_see_legacy_skills(effective_user_id)
normalized_skills_container_path = _normalize_skills_container_path(skills_container_path)
payload = {
"sandbox_id": sandbox_id,
"thread_id": thread_id,
"user_id": effective_user_id,
"include_legacy_skills": include_legacy_skills,
"skills_container_path": normalized_skills_container_path,
"provision_lark_cli_runtime": provision_lark_cli_runtime,
"provision_lark_cli_broker": provision_lark_cli_broker,
}
provisioner_extra_mounts = _provisioner_extra_mounts_payload(
extra_mounts,
skills_container_path=normalized_skills_container_path,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
)

View File

@ -37,9 +37,11 @@ from __future__ import annotations
import asyncio
import atexit
import hashlib
import json
import logging
import os
import posixpath
import shlex
import signal
import threading
@ -51,8 +53,8 @@ from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass
from decimal import Decimal, InvalidOperation
from functools import partial
from pathlib import Path
from typing import Any
from pathlib import Path, PurePosixPath
from typing import TYPE_CHECKING, Any
from e2b import SandboxQuery
from e2b_code_interpreter import Sandbox as E2BClientSandbox
@ -81,6 +83,9 @@ from .capacity import (
)
from .e2b_sandbox import DEFAULT_E2B_HOME_DIR, E2BSandbox, _is_sandbox_gone_error
if TYPE_CHECKING:
from deerflow.skills.projection import SkillProjectionPaths
logger = logging.getLogger(__name__)
@ -112,11 +117,76 @@ _MAX_MOUNT_PASS_FILES = 2000
# Deadline checks stop preflight work and new writes. Active SDK writes finish.
_MOUNT_PASS_DEADLINE_SECONDS = 120
# Recursive skill projection replacement must never target an operating-system
# tree. The configured E2B home is handled separately: an isolated descendant
# such as /home/user/skills is supported, while the home directory itself and
# every ancestor remain protected.
_E2B_PROTECTED_SYSTEM_TREES = frozenset(
PurePosixPath(path)
for path in (
"/bin",
"/boot",
"/dev",
"/etc",
"/home",
"/lib",
"/lib32",
"/lib64",
"/libx32",
"/lost+found",
"/media",
"/opt",
"/proc",
"/root",
"/run",
"/sbin",
"/snap",
"/srv",
"/sys",
"/tmp",
"/usr",
"/var",
)
)
def _mount_deadline_reason(deadline_seconds: int) -> str:
return f"time budget {deadline_seconds}s"
def _validate_skills_reset_root(container_path: str, *, home_dir: str) -> str:
"""Return a canonical E2B root that is safe for recursive managed resets."""
candidate = container_path.rstrip("/")
if not candidate or not candidate.startswith("/") or candidate.startswith("//"):
raise ValueError("The skills container path is not a safe E2B skills reset target: it must be an absolute non-root path")
normalized = posixpath.normpath(candidate)
if normalized != candidate:
raise ValueError("The skills container path is not a safe E2B skills reset target: it must not contain redundant separators, '.' or '..'")
root = PurePosixPath(normalized)
protected_roots = {
PurePosixPath("/mnt/user-data"),
PurePosixPath("/mnt/acp-workspace"),
}
normalized_home = posixpath.normpath(home_dir.rstrip("/") or DEFAULT_E2B_HOME_DIR)
home_root: PurePosixPath | None = None
if normalized_home.startswith("/") and not normalized_home.startswith("//"):
home_root = PurePosixPath(normalized_home)
protected_roots.add(home_root)
for protected in protected_roots:
if protected == root or protected.is_relative_to(root):
raise ValueError(f"The skills container path is not a safe E2B skills reset target: {normalized!r} equals or contains protected path {str(protected)!r}")
is_isolated_home_subtree = home_root is not None and root != home_root and root.is_relative_to(home_root)
if not is_isolated_home_subtree:
for protected in _E2B_PROTECTED_SYSTEM_TREES:
if root == protected or root.is_relative_to(protected):
raise ValueError(f"The skills container path is not a safe E2B skills reset target: {normalized!r} is inside protected operating-system tree {str(protected)!r}")
return normalized
class _MountPassLimitExceeded(Exception):
"""Stop the current mount upload pass at its aggregate resource limit."""
@ -148,6 +218,7 @@ META_KEY_GATEWAY = "deer_flow_gateway"
META_KEY_CREATED_AT = "deer_flow_created_at"
META_KEY_CAPACITY_LEDGER = "deer_flow_capacity_ledger"
META_KEY_CAPACITY_RESERVATION = "deer_flow_capacity_reservation"
META_KEY_SKILLS_ROOT = "deer_flow_skills_root"
META_VAL_PROVIDER = "e2b_sandbox_provider"
E2B_EXTRA_CONFIG_KEYS = frozenset(
{
@ -187,6 +258,7 @@ class E2BSandboxProvider(SandboxProvider):
# remote backend in AioSandboxProvider sets the same flag).
uses_thread_data_mounts = False
needs_upload_permission_adjustment = True
supports_agent_skill_isolation = True
# ── Construction & config ────────────────────────────────────────────
@ -194,11 +266,13 @@ class E2BSandboxProvider(SandboxProvider):
self._lock = threading.Lock()
# Active sandboxes, keyed by DeerFlow-side sandbox id (== e2b id).
self._sandboxes: dict[str, E2BSandbox] = {}
# (user_id, thread_id) -> sandbox id for fast in-process lookup.
self._thread_sandboxes: dict[tuple[str, str], str] = {}
# Per-(user,thread) serializer for acquire() and release() state
# (user_id, thread_id, skills_root) -> sandbox id for fast in-process
# lookup. The provider snapshots the root at startup, but keeping it in
# the key makes the identity boundary explicit and fail-safe.
self._thread_sandboxes: dict[tuple[str, str, str], str] = {}
# Per-(user,thread,skills_root) serializer for acquire() and release() state
# transitions without holding the provider-wide lock across remote IO.
self._acquire_serializer: AcquireSerializer[tuple[str, str]] = AcquireSerializer(thread_name_prefix="e2b-sandbox-lock-wait")
self._acquire_serializer: AcquireSerializer[tuple[str, str, str]] = AcquireSerializer(thread_name_prefix="e2b-sandbox-lock-wait")
# Warm pool: released sandboxes whose remote micro-VM is still alive.
# ``OrderedDict`` maintains insertion / move_to_end order for LRU.
self._warm_pool: OrderedDict[str, tuple[str, float]] = OrderedDict()
@ -255,7 +329,8 @@ class E2BSandboxProvider(SandboxProvider):
def _load_config(self) -> dict[str, Any]:
"""Read e2b options off ``SandboxConfig`` (``extra="allow"``)."""
sandbox_config = get_app_config().sandbox
app_config = get_app_config()
sandbox_config = app_config.sandbox
unknown_keys = sorted(set(getattr(sandbox_config, "model_extra", None) or {}) - E2B_EXTRA_CONFIG_KEYS)
if unknown_keys:
logger.warning(
@ -295,11 +370,18 @@ class E2BSandboxProvider(SandboxProvider):
logger.warning("E2BSandboxProvider: overflow_policy is 'burst' but burst_limit is 0; falling back to 'reject'")
overflow_policy = "reject"
home_dir = _opt("home_dir") or DEFAULT_E2B_HOME_DIR
skills_container_path = _validate_skills_reset_root(
app_config.skills.container_path,
home_dir=home_dir,
)
return {
"api_key": api_key,
"template": _opt("template") or _opt("image") or DEFAULT_TEMPLATE,
"domain": _opt("domain"),
"home_dir": _opt("home_dir") or DEFAULT_E2B_HOME_DIR,
"home_dir": home_dir,
"skills_container_path": skills_container_path,
"idle_timeout": idle_timeout,
"replicas": replicas,
"overflow_policy": overflow_policy,
@ -308,7 +390,7 @@ class E2BSandboxProvider(SandboxProvider):
"mounts": _opt("mounts") or [],
"environment": self._resolve_env_vars(_opt("environment") or {}),
"ownership": _opt("ownership"),
"stream_bridge": getattr(get_app_config(), "stream_bridge", None),
"stream_bridge": getattr(app_config, "stream_bridge", None),
"reconciliation_interval_seconds": max(
1.0,
float(_opt("reconciliation_interval_seconds", DEFAULT_RECONCILIATION_INTERVAL_SECONDS)),
@ -378,18 +460,18 @@ class E2BSandboxProvider(SandboxProvider):
def _effective_acquire_user_id(user_id: str | None) -> str:
return user_id or get_effective_user_id()
@staticmethod
def _thread_key(thread_id: str, user_id: str) -> tuple[str, str]:
return (user_id, thread_id)
def _thread_key(self, thread_id: str, user_id: str) -> tuple[str, str, str]:
return (user_id, thread_id, self._config["skills_container_path"])
@staticmethod
def _stable_seed(thread_id: str, user_id: str) -> str:
"""Warm-pool lookup seed derived from user/thread scope.
def _stable_seed(self, thread_id: str, user_id: str) -> str:
"""Warm-pool lookup seed derived from user/thread/root scope.
For E2B this value is the warm-pool lookup seed, not the
provider-issued remote id (RFC #4741 §6).
"""
return derive_sandbox_scope_token(user_id=user_id, thread_id=thread_id)
base_scope = derive_sandbox_scope_token(user_id=user_id, thread_id=thread_id)
skills_root = self._config["skills_container_path"]
return hashlib.sha256(f"{base_scope}\0{skills_root}".encode()).hexdigest()[:16]
def _metadata_matches_capacity_ledger(
self,
@ -617,13 +699,18 @@ class E2BSandboxProvider(SandboxProvider):
META_KEY_PROVIDER: META_VAL_PROVIDER,
META_KEY_USER: user_id,
META_KEY_THREAD: thread_id,
META_KEY_SKILLS_ROOT: self._config["skills_container_path"],
}
)
candidates = sorted(
(
(sandbox_id, metadata)
for entry in entries
if (sandbox_id := self._entry_id(entry)) and (metadata := self._entry_metadata(entry)).get(META_KEY_USER) == user_id and metadata.get(META_KEY_THREAD) == thread_id and self._metadata_matches_capacity_ledger(metadata)
if (sandbox_id := self._entry_id(entry))
and (metadata := self._entry_metadata(entry)).get(META_KEY_USER) == user_id
and metadata.get(META_KEY_THREAD) == thread_id
and metadata.get(META_KEY_SKILLS_ROOT) == self._config["skills_container_path"]
and self._metadata_matches_capacity_ledger(metadata)
),
key=lambda item: (item[1].get(META_KEY_CREATED_AT, ""), item[0]),
)
@ -1075,6 +1162,7 @@ class E2BSandboxProvider(SandboxProvider):
META_KEY_PROVIDER: META_VAL_PROVIDER,
META_KEY_GATEWAY: self._owner_id,
META_KEY_CREATED_AT: str(time.time()),
META_KEY_SKILLS_ROOT: self._config["skills_container_path"],
}
if self._deployment_capacity is not None:
metadata[META_KEY_CAPACITY_LEDGER] = self._deployment_capacity.key
@ -1170,7 +1258,7 @@ class E2BSandboxProvider(SandboxProvider):
# One-shot mount uploads. e2b has no host bind-mount, so we copy
# files from ``host_path`` into ``container_path`` at sandbox start.
try:
self._apply_mounts(client, user_id=user_id)
self._apply_mounts(client, user_id=user_id, thread_id=thread_id)
except Exception as e:
logger.warning("Failed to apply some mounts to e2b sandbox %s: %s", sandbox_id, e)
@ -1448,7 +1536,7 @@ class E2BSandboxProvider(SandboxProvider):
)
groups: dict[tuple[str, str], list[tuple[str, dict[str, Any]]]] = {}
orphans: list[tuple[str, dict[str, Any]]] = []
stale_entries: list[tuple[str, dict[str, Any], float]] = []
present_ids: set[str] = set()
for entry in entries:
@ -1459,15 +1547,34 @@ class E2BSandboxProvider(SandboxProvider):
metadata = self._entry_metadata(entry)
user_id = metadata.get(META_KEY_USER)
thread_id = metadata.get(META_KEY_THREAD)
if isinstance(user_id, str) and user_id and isinstance(thread_id, str) and thread_id:
skills_root = metadata.get(META_KEY_SKILLS_ROOT)
has_thread_identity = isinstance(user_id, str) and user_id and isinstance(thread_id, str) and thread_id
if has_thread_identity and skills_root == self._config["skills_container_path"]:
groups.setdefault((user_id, thread_id), []).append((sandbox_id, metadata))
elif has_thread_identity:
# A VM from an older root must never be adopted by this
# provider. Reap it after the shorter duplicate grace once no
# peer still owns it, so it does not strand deployment capacity.
stale_entries.append(
(
sandbox_id,
metadata,
float(self._config["reconciliation_grace_seconds"]),
)
)
else:
orphans.append((sandbox_id, metadata))
stale_entries.append(
(
sandbox_id,
metadata,
float(self._config["reconciliation_orphan_ttl_seconds"]),
)
)
for (user_id, thread_id), candidates in groups.items():
candidates.sort(key=lambda item: (item[1].get(META_KEY_CREATED_AT, ""), item[0]))
with self._lock:
local_id = self._thread_sandboxes.get((user_id, thread_id))
local_id = self._thread_sandboxes.get(self._thread_key(thread_id, user_id))
if local_id:
candidates.sort(key=lambda item: item[0] != local_id)
@ -1557,7 +1664,7 @@ class E2BSandboxProvider(SandboxProvider):
self._release_ownership(sandbox_id)
self._safe_close_client(client)
for sandbox_id, metadata in orphans:
for sandbox_id, metadata, minimum_age in stale_entries:
if time.monotonic() >= deadline:
stats.budget_exhausted = True
break
@ -1567,7 +1674,7 @@ class E2BSandboxProvider(SandboxProvider):
age = time.time() - float(created_at) if created_at is not None else observed_at - first_seen
except (TypeError, ValueError):
age = observed_at - first_seen
if age < float(self._config["reconciliation_orphan_ttl_seconds"]):
if age < minimum_age:
stats.deferred += 1
continue
if not self._claim_ownership(sandbox_id, for_destroy=True):
@ -1777,9 +1884,10 @@ class E2BSandboxProvider(SandboxProvider):
f"if [ ! -e /mnt/acp-workspace ] || [ -L /mnt/acp-workspace ]; then "
f" sudo ln -sfn {shlex.quote(home_dir)}/acp-workspace /mnt/acp-workspace; "
f"fi; "
# /mnt/skills is left alone here; the optional ``mounts`` config
# uploads its content via _apply_mounts and creates the directory
# on demand. We only ensure that /mnt itself is traversable.
# The configured skills root is left alone here; the optional
# ``mounts`` config uploads its content via _apply_mounts and
# creates the directory on demand. We only ensure that /mnt itself
# is traversable for the default layout.
f"sudo chmod a+rx /mnt 2>/dev/null || true; "
f"echo BOOTSTRAP_OK"
)
@ -1795,7 +1903,11 @@ class E2BSandboxProvider(SandboxProvider):
if exit_code not in (0, None) or "BOOTSTRAP_OK" not in stdout:
raise RuntimeError(f"e2b bootstrap script failed with exit code {exit_code}; stderr={stderr.strip()}")
def _skill_projection_mounts(self, user_id: str) -> list[tuple[Path, str, bool]]:
def _skill_projection_mounts(
self,
user_id: str,
thread_id: str | None = None,
) -> list[tuple[Path, str, bool]]:
"""Best-effort: a projection failure must not drop configured mounts too.
Unlike Local/AIO's ``_ensure_skills_projection``, this used to raise
@ -1805,14 +1917,28 @@ class E2BSandboxProvider(SandboxProvider):
no mounts applied at all). Swallowing here keeps the two mount
sources independent, matching the other two providers.
"""
from deerflow.config.paths import get_paths
from deerflow.skills.projection import ensure_skill_projections
from deerflow.skills.storage import get_or_new_user_skill_storage
try:
# The middleware performs a strict, fail-closed upload after acquire
# for a policy-scoped thread. Do not first upload the shared view and
# briefly hydrate the VM with skills outside the Agent allowlist.
if (
thread_id
and get_paths()
.thread_skills_view_dir(
thread_id,
user_id=user_id,
)
.exists()
):
return []
config = get_app_config()
storage = get_or_new_user_skill_storage(user_id, app_config=config)
projection = ensure_skill_projections(storage)
container_root = config.skills.container_path.rstrip("/")
container_root = self._config["skills_container_path"]
return [
(projection.public, f"{container_root}/public", True),
(projection.custom, f"{container_root}/custom", True),
@ -1823,7 +1949,13 @@ class E2BSandboxProvider(SandboxProvider):
logger.warning("Could not ensure skills projection for user %s: %s", user_id, exc, exc_info=True)
return []
def _apply_mounts(self, client: E2BClientSandbox, *, user_id: str | None = None) -> None:
def _apply_mounts(
self,
client: E2BClientSandbox,
*,
user_id: str | None = None,
thread_id: str | None = None,
) -> None:
started_at = time.monotonic()
deadline_seconds = self._config.get("mount_upload_deadline_seconds", _MOUNT_PASS_DEADLINE_SECONDS)
budget = _MountUploadBudget(
@ -1844,9 +1976,9 @@ class E2BSandboxProvider(SandboxProvider):
)
effective_user_id = user_id or get_effective_user_id()
projection_mounts = self._skill_projection_mounts(effective_user_id)
projection_mounts = self._skill_projection_mounts(effective_user_id, thread_id) if thread_id is not None else self._skill_projection_mounts(effective_user_id)
configured_mounts = self._config.get("mounts") or []
skills_root = get_app_config().skills.container_path.rstrip("/")
skills_root = self._config["skills_container_path"]
mounts: list[tuple[Path, str, bool]] = list(projection_mounts)
for mount in configured_mounts:
@ -1893,6 +2025,95 @@ class E2BSandboxProvider(SandboxProvider):
except Exception as e:
logger.warning("Failed to upload mount %s -> %s: %s", host_path, container_path, e)
def sync_agent_skills(
self,
sandbox_id: str,
*,
thread_id: str,
user_id: str,
projection: SkillProjectionPaths,
) -> None:
"""Atomically rebuild E2B's managed skills for one thread scope.
The remote reset and all four category uploads form one critical
section. Reuse the acquire/release lock for the same user/thread so a
concurrent policy cannot wipe or repopulate the sandbox mid-upload.
"""
with self._acquire_serializer.hold(self._thread_key(thread_id, user_id)):
self._sync_agent_skills_locked(
sandbox_id,
projection=projection,
)
def _sync_agent_skills_locked(
self,
sandbox_id: str,
*,
projection: SkillProjectionPaths,
) -> None:
with self._lock:
sandbox = self._sandboxes.get(sandbox_id)
if sandbox is None:
raise RuntimeError(f"E2B sandbox {sandbox_id} is not available for skill synchronization")
skills_root = _validate_skills_reset_root(
self._config["skills_container_path"],
home_dir=sandbox.home_dir,
)
# A sandbox-visible marker cannot prove integrity: the sandbox user can
# modify the marker and replace category directories between turns.
# Rebuild on every policy sync so the no-follow root check and managed
# directory replacement always run before the sandbox is handed out.
marker_path = f"{skills_root}/.deerflow-projection-signature"
category_paths = [
f"{skills_root}/public",
f"{skills_root}/custom",
f"{skills_root}/legacy",
f"{skills_root}/integrations",
]
quoted_root = shlex.quote(skills_root)
quoted_categories = " ".join(shlex.quote(path) for path in category_paths)
quoted_managed_paths = " ".join(shlex.quote(path) for path in (*category_paths, marker_path))
reset_script = (
f"set -e; if [ -L {quoted_root} ]; then echo 'Refusing symlinked skills root' >&2; exit 2; fi; "
f"sudo rm -rf -- {quoted_managed_paths}; "
f"sudo mkdir -p -- {quoted_categories}; "
f'sudo chown "$(id -u):$(id -g)" -- {quoted_root} {quoted_categories}; '
"echo SKILLS_RESET_OK"
)
result = sandbox.client.commands.run(reset_script)
stdout = getattr(result, "stdout", "") or ""
stderr = getattr(result, "stderr", "") or ""
exit_code = getattr(result, "exit_code", 0)
if exit_code not in (0, None) or "SKILLS_RESET_OK" not in stdout:
raise RuntimeError(f"Failed to reset E2B skill projection (exit_code={exit_code}, stderr={stderr.strip()})")
started_at = time.monotonic()
deadline_seconds = self._config.get("mount_upload_deadline_seconds", _MOUNT_PASS_DEADLINE_SECONDS)
budget = _MountUploadBudget(
deadline=started_at + deadline_seconds,
deadline_seconds=deadline_seconds,
)
for source, destination in zip(
(
projection.public,
projection.custom,
projection.legacy,
projection.integrations,
),
category_paths,
strict=True,
):
self._upload_tree(
sandbox.client,
source,
destination,
True,
budget=budget,
)
# ── Output mirroring ────────────────────────────────────────────────
_SYNC_BACK_SUBDIRS = ("outputs", "workspace")
_SYNC_MANIFEST_NAME = ".e2b-output-sync.json"
@ -2370,7 +2591,7 @@ class E2BSandboxProvider(SandboxProvider):
self._release_internal(sandbox_id)
return
user_id, thread_id = thread_key
user_id, thread_id, _skills_root = thread_key
with self._acquire_serializer.hold(self._thread_key(thread_id, user_id)):
self._release_internal(sandbox_id)
@ -2385,7 +2606,7 @@ class E2BSandboxProvider(SandboxProvider):
"""
sandbox: E2BSandbox | None = None
seed: str | None = None
removed_keys: list[tuple[str, str]] = []
removed_keys: list[tuple[str, str, str]] = []
transition_slot_held = False
with self._lock:
@ -2398,7 +2619,7 @@ class E2BSandboxProvider(SandboxProvider):
for key in removed_keys:
self._thread_sandboxes.pop(key, None)
if removed_keys:
user_id, thread_id = removed_keys[0]
user_id, thread_id, _skills_root = removed_keys[0]
seed = self._stable_seed(thread_id, user_id)
# E2BSandbox.close() clears its client reference. Keep this reference
@ -2416,7 +2637,7 @@ class E2BSandboxProvider(SandboxProvider):
sync_failed_due_to_dead_vm = False
if seed is not None and removed_keys:
user_id_sync, thread_id_sync = removed_keys[0]
user_id_sync, thread_id_sync, _skills_root = removed_keys[0]
try:
self._sync_outputs_to_host(sandbox, thread_id=thread_id_sync, user_id=user_id_sync)
except Exception as e: # pragma: no cover - defensive

View File

@ -10,7 +10,7 @@ Setup: Copy `config.example.yaml` to `config.yaml` in the **project root** direc
**Config Hot-Reload Boundary**: Gateway dependencies route through `get_app_config()` on every request, so per-run fields like `models[*].max_tokens`, `summarization.*`, `title.*`, `memory.*`, `subagents.*`, `verification.*`, `tools[*]`, and the agent system prompt pick up `config.yaml` edits on the next message. `AppConfig` is intentionally **not** cached on `app.state``lifespan()` keeps a local `startup_config` variable for one-shot bootstrap work and passes it to `langgraph_runtime(app, startup_config)`.
Infrastructure fields are **restart-required**. The authoritative list lives in `packages/harness/deerflow/config/reload_boundary.py::STARTUP_ONLY_FIELDS` and is mirrored by the standardised `"startup-only:"` prefix on the corresponding `Field(description=...)` in `AppConfig`, so IDE hover on those fields surfaces the reason inline (no need to context-switch into this table). Currently registered: `plugins`, `database`, `checkpointer`, `run_events`, `stream_bridge`, `sandbox`, `log_level`, `logging`, `channels`, `channel_connections`, `scheduler`, `mcp_tasks`, `subagent_runtime`, `subagent_batches`, `run_ownership`. Adding a new restart-required field requires updating the registry; drift is pinned by `tests/test_reload_boundary.py`. `scheduler.recursion_limit` is the exception inside that section: it is read from `get_app_config()` at each scheduled dispatch, so a YAML edit applies to the next run without restarting the poller.
Infrastructure fields are **restart-required**. The authoritative list lives in `packages/harness/deerflow/config/reload_boundary.py::STARTUP_ONLY_FIELDS` and is mirrored by the standardised `"startup-only:"` prefix on the corresponding `Field(description=...)` in `AppConfig` or an explicitly registered nested config model, so IDE hover on those fields surfaces the reason inline (no need to context-switch into this table). Currently registered: `plugins`, `database`, `checkpointer`, `run_events`, `agent_storage`, `stream_bridge`, `sandbox`, `skills.container_path`, `log_level`, `logging`, `channels`, `channel_connections`, `scheduler`, `mcp_tasks`, `subagent_runtime`, `subagent_batches`, `run_ownership`, `dedupe_storage`. Adding a new restart-required field requires updating the registry; drift is pinned by `tests/test_reload_boundary.py`. `scheduler.recursion_limit` is the exception inside that section: it is read from `get_app_config()` at each scheduled dispatch, so a YAML edit applies to the next run without restarting the poller.
**Persistence backend resolution**: the unified `database` section selects the
Gateway's LangGraph checkpointer, LangGraph Store, and DeerFlow SQL repositories.
@ -53,7 +53,7 @@ Extensions are optional only in the fallback *search* mode (priority 3-4 above):
- `tools[]` - Tool configs with `use` variable path and `group`
- `tool_groups[]` - Logical groupings for tools
- `sandbox.use` - Sandbox provider class path
- `skills.path` / `skills.container_path` - Host and container paths to skills directory
- `skills.path` / `skills.container_path` - Host and container paths to skills directory. AIO and E2B snapshot the container path at provider startup. Their local/remote backends and the Kubernetes provisioner require one canonical absolute non-root path outside reserved platform mounts; custom roots participate in deterministic sandbox identity, and E2B records the root in remote metadata.
- `skills.deferred_discovery` - When `true`, replaces the full-metadata `<available_skills>` prompt block with a compact `<skill_index>` (names only) and registers the `describe_skill` tool so the agent fetches metadata on demand. Defaults to `false` (legacy full-metadata injection)
- `title` - Auto-title generation (enabled, max_words, max_chars, model_name; null model_name uses fast local fallback, explicit model_name uses the prompt_template LLM path)
- `summarization` - Context summarization (enabled, trigger conditions, keep policy)

View File

@ -295,6 +295,18 @@ class Paths:
"""Enabled managed integration skills exposed to one user's sandboxes."""
return self.user_skills_view_dir(user_id) / "integrations"
def thread_skills_view_dir(self, thread_id: str, *, user_id: str) -> Path:
"""Sandbox-visible skill projection scoped to one user/thread.
The directory lives below the thread root so ordinary thread deletion
also removes its policy projection.
"""
return self.thread_dir(thread_id, user_id=user_id) / "skills_view"
def host_thread_skills_view_dir(self, thread_id: str, *, user_id: str) -> str:
"""Host path for a thread-scoped skill projection."""
return _join_host_path(self.host_thread_dir(thread_id, user_id=user_id), "skills_view")
def thread_dir(self, thread_id: str, *, user_id: str | None = None) -> Path:
"""
Host path for a thread's data.

View File

@ -10,7 +10,7 @@ change at runtime.
The registry covers two kinds of entries:
- Top-level ``AppConfig`` fields (``database``, ``checkpointer``,
- ``AppConfig`` fields and explicitly registered nested fields (``database``, ``checkpointer``,
``run_events``, ``stream_bridge``, ``sandbox``, ``log_level``). For
these, :func:`format_field_description` produces the standardised
``"startup-only: ..."`` prefix that the matching Pydantic
@ -50,6 +50,10 @@ STARTUP_ONLY_FIELDS: dict[str, str] = {
"agent_storage": ("langgraph_runtime() validates agent_storage.backend against database.backend once at startup, and the db backend's synchronous SQLAlchemy engine is process-cached on first use; switching backend needs a restart."),
"stream_bridge": ("make_stream_bridge() constructs the stream-bridge singleton once during startup."),
"sandbox": ("get_sandbox_provider() caches the provider singleton (``_default_sandbox_provider``); a different ``sandbox.use`` class path only takes effect on next process start."),
"skills.container_path": (
"AioSandboxProvider and E2BSandboxProvider normalize and capture the skills mount root when their provider singleton starts; "
"sandbox identity, mounts, remote metadata, and skill synchronization must keep using that one root until the Gateway restarts."
),
"log_level": (
"apply_logging_level() runs only during app.py startup; it sets the deerflow/app logger levels and may lower root handler thresholds so configured messages can propagate. A freshly reloaded AppConfig does not retrigger it."
),
@ -97,9 +101,9 @@ def iter_startup_only_field_paths() -> Iterator[str]:
def is_startup_only_field(field_path: str) -> bool:
"""Return ``True`` when *field_path* is registered as restart-required.
Accepts only top-level paths (``"database"``, ``"sandbox"`` etc.);
nested keys like ``"database.url"`` are not modelled here because the
boundary is per-section, not per-leaf.
Most entries are top-level paths (``"database"``, ``"sandbox"`` etc.).
A nested path is registered only when one leaf has a different reload
boundary from the rest of its section, such as ``skills.container_path``.
"""
return field_path in STARTUP_ONLY_FIELDS
@ -112,7 +116,8 @@ def format_field_description(field_path: str, *, field_doc: str | None = None) -
side against the other.
Args:
field_path: A registered top-level field path (e.g. ``"log_level"``).
field_path: A registered field path (e.g. ``"log_level"`` or
``"skills.container_path"``).
field_doc: Optional human-facing description for the field itself
(allowed values, semantics, etc.). When supplied, it is
appended after the ``startup-only:`` marker block separated by

View File

@ -3,6 +3,7 @@ from pathlib import Path
from pydantic import BaseModel, Field
from deerflow.config.reload_boundary import format_field_description
from deerflow.config.runtime_paths import project_root, resolve_path
from deerflow.constants import DEFAULT_SKILLS_CONTAINER_PATH
@ -27,7 +28,10 @@ class SkillsConfig(BaseModel):
)
container_path: str = Field(
default=DEFAULT_SKILLS_CONTAINER_PATH,
description="Path where skills are mounted in the sandbox container",
description=format_field_description(
"skills.container_path",
field_doc="Path where skills are mounted in the sandbox container.",
),
)
deferred_discovery: bool = Field(
default=False,

View File

@ -1,18 +1,33 @@
### Sandbox System (`packages/harness/deerflow/sandbox/`)
**Interface**: Abstract `Sandbox` with `execute_command(command, env=None)`, `read_file`, `write_file`, `list_dir`, `glob`, and `grep`. `grep` accepts either one text file or a directory tree. The optional `env` injects per-call environment variables (request-scoped secrets — see Request-Scoped Secrets below); `LocalSandbox` merges it into the host subprocess environment and `AioSandbox` routes env-bearing commands through the `bash.exec(env=...)` API on a fresh session.
**Provider Pattern**: `SandboxProvider` with `acquire`, `acquire_async`, `get`, `release` lifecycle. Async agent/tool paths call async sandbox lifecycle hooks so Docker sandbox creation, discovery, cross-process locking, readiness polling, and release stay off the event loop.
**Shared components** (RFC #4741): remote providers derive their deterministic sandbox id through `derive_sandbox_scope_token` (`sandbox/identity.py`, keyword-only; the sha256/16-hex derivation is a compatibility contract — changing it orphans existing containers), and serialize provider-selected acquire/release transitions through `AcquireSerializer` (`sandbox/acquire_serialization.py`): per-key `threading.Lock` table with holder/waiter refcount reclamation (no unbounded per-thread lock growth), a bounded dedicated executor so async waits never touch the event loop or the default executor, worker-owned cancellation cleanup that does not depend on a cancelled event loop task resuming, and idempotent `close()` called from provider `shutdown()`/`reset()`. AIO/E2B key by `(user_id, thread_id)`; BoxLite/Tenki/OpenSandbox key by the derived sandbox id. `thread_id=None` acquires (random uuid ids) never enter the serializer.
**Provider Pattern**: `SandboxProvider` with `acquire`, `acquire_async`, `get`, `release` lifecycle. Async agent/tool paths call async sandbox lifecycle hooks so Docker sandbox creation, discovery, cross-process locking, readiness polling, and release stay off the event loop. Providers that can enforce a lead Agent's explicit skill policy across the current Agent-accessible tool surface set `supports_agent_skill_isolation=True`; bind-mount providers observe the prepared thread roots directly, while upload providers implement `sync_agent_skills`. Host-backed providers must report the capability as false whenever an enabled shell can bypass their path mappings. The middleware fails closed before acquisition for an explicit policy on an unsupported provider.
**Shared components** (RFC #4741): remote providers derive their deterministic sandbox id through `derive_sandbox_scope_token` (`sandbox/identity.py`, keyword-only; the sha256/16-hex derivation is a compatibility contract — changing it orphans existing containers), and serialize provider-selected acquire/release transitions through `AcquireSerializer` (`sandbox/acquire_serialization.py`): per-key `threading.Lock` table with holder/waiter refcount reclamation (no unbounded per-thread lock growth), a bounded dedicated executor so async waits never touch the event loop or the default executor, worker-owned cancellation cleanup that does not depend on a cancelled event loop task resuming, and idempotent `close()` called from provider `shutdown()`/`reset()`. AIO keys by `(user_id, thread_id)`; E2B keys by `(user_id, thread_id, skills_root)`; BoxLite/Tenki/OpenSandbox key by the derived sandbox id. `thread_id=None` acquires (random uuid ids) never enter the serializer.
**Authorization gate** (`sandbox:execute`, RFC #4063 Phase 3): every sandbox-backed tool call passes through the gate in `deerflow/authz/sandbox_authz.py` - a binary `authorize(principal, "sandbox", "execute", target="*")` check before either reusing a persisted sandbox id or calling `provider.acquire`. Rechecking reuse is required because authorization config and user roles can change while the sandbox remains cached. Sync tool invocations call `authorize_sandbox_execution`; async tool invocations await `authorize_sandbox_execution_async` exactly once. A task-local `ContextVar` scopes that single decision across the complete composed tool invocation, including `ReadBeforeWriteMiddleware`'s pre-write inspection, tool body, and post-read mark; the value is copied into `asyncio.to_thread` workers. Authorization denial is converted to the normal error `ToolMessage` at the composed middleware boundary and is explicitly excluded from the gate's generic fail-open handlers. Async config loading and provider class discovery/import are offloaded before `aauthorize()` so reused sandbox calls do not hash config files or import custom modules on the event loop; provider construction remains on the running event loop because async providers may initialize loop-affine clients. The gate lives at the single tool initialization entry point (`ensure_sandbox_initialized` / `ensure_sandbox_initialized_async` in `tools.py`), while `SandboxMiddleware.before_agent` / `abefore_agent` apply the matching sync/async check to eager acquisition. Deny raises `SandboxAuthorizationError` (`sandbox/exceptions.py`), which propagates out of ordinary tool execution as a friendly error `ToolMessage` ("sandbox execution is not permitted for your role") - the eager path catches it and skips acquisition instead, deferring the deny to the first sandbox-touching tool call so both paths share the same semantics. Provider errors (authorization calls and provider resolution) follow `authorization.fail_closed` / `fail_open`; no readable `config.yaml` or `authorization.enabled: false` makes the gate a no-op (`safe_app_config` tolerates missing config). Gateway auxiliary sync paths (uploads/artifacts routers) call `try_acquire_sandbox_for_request` (`app/gateway/authz.py`), which gates via `authorize_sandbox_for_request` and skips the sync on deny - the upload/artifact edit itself still succeeds. Tests: `tests/test_sandbox_authorization.py` and `tests/blocking_io/test_sandbox_authorization.py`.
**Environment policy** (`sandbox/env_policy.py`): `execute_command` no longer inherits the full `os.environ`. `build_sandbox_env()` scrubs secret-looking names (`*KEY*`/`*SECRET*`/`*TOKEN*`/`*PASS*`/`*CREDENTIAL*`) from the inherited environment before layering injected request secrets on top, so platform credentials (e.g. `OPENAI_API_KEY`) never leak into skill subprocesses. Benign vars (`PATH`, `HOME`, `LANG`, `VIRTUAL_ENV`, ...) are preserved.
**Implementations**:
- `LocalSandboxProvider` - Local filesystem execution. `acquire(thread_id)` returns a per-thread `LocalSandbox` (id `local:{thread_id}`) whose `path_mappings` resolve `/mnt/user-data/{workspace,uploads,outputs}` and `/mnt/acp-workspace` to that thread's host directories, so the public `Sandbox` API honours the `/mnt/user-data` contract uniformly with AIO. `acquire()` / `acquire(None)` keeps the legacy generic singleton (id `local`) for callers without a thread context. Per-thread sandboxes are held in an LRU cache (default 256 entries) guarded by a `threading.Lock`. Public, custom, legacy, and managed integration skill mappings point at stable enabled-only projection roots rather than raw skill directories. On Windows, Git Bash/MSYS argument-conversion exclusions are limited to safe non-root virtual path prefixes; do not restore a blanket conversion disable, because host-native CLI launchers need normal MSYS path conversion for their own installation paths.
- `AioSandboxProvider` (`packages/harness/deerflow/community/`) - Docker-based isolation. Active-cache and warm-pool entries are checked with the backend during acquire/reuse; definitively dead containers are dropped from all in-process maps so the thread can discover or create a fresh sandbox instead of reusing a stale client. Backend health-check failures are treated as unknown, not dead; local discovery likewise treats an unverifiable container as not adoptable and falls through to create rather than failing acquire. `get()` remains an in-memory lookup for event-loop-safe tool paths — it never touches the ownership store (that would be blocking IO on the event loop); ownership is published on acquire/reclaim and refreshed off the event loop by the dedicated renewal thread (`_renew_owned_leases`). `reset()` closes the per-instance acquire serializer so replacing the singleton cannot retain its executor workers; full remote sandbox teardown remains `shutdown()`. `uses_thread_data_mounts` defaults to backend detection (`LocalContainerBackend=True`, remote/provisioner backends=False), while the optional `sandbox.thread_data_mounts` boolean takes precedence for deployments that guarantee the Gateway and sandbox share the same thread user-data directories. Setting it `true` skips upload-time sandbox acquire/sync; a false positive leaves uploads unavailable to the sandbox. Local-container and hostPath-provisioner mounts use the same stable skill projection roots; PVC-backed skills remain governed by the operator-supplied PVC layout until PVC materialization is implemented. Readiness probes and `agent_sandbox` clients classify loopback/private IPs, single-label cluster hosts, and Docker/Podman internal hostnames as direct control-plane destinations and set `trust_env=False`; external FQDNs and public IPs retain environment proxy support.
- `LocalSandboxProvider` - Local filesystem execution. `acquire(thread_id)` returns a per-user/thread `LocalSandbox` (id `local:{user_id}:{thread_id}`) whose `path_mappings` resolve `/mnt/user-data/{workspace,uploads,outputs}` and `/mnt/acp-workspace` to that thread's host directories, so the public `Sandbox` API honours the `/mnt/user-data` contract uniformly with AIO. `acquire()` / `acquire(None)` keeps the legacy generic singleton (id `local`) for callers without a thread context. Per-thread sandboxes are held in an LRU cache (default 256 entries) guarded by a `threading.Lock`. Shared runs use category mappings; a policy-scoped run replaces them with one `/mnt/skills` root mapping to the coherent thread view, so structured file tools resolve through one managed boundary. This is not a host filesystem security boundary: an enabled host `bash` subprocess can use canonical paths without `PathMapping`, so `supports_agent_skill_isolation` is dynamic and explicit Agent policies fail closed while host bash is enabled. On Windows, Git Bash/MSYS argument-conversion exclusions are limited to safe non-root virtual path prefixes; do not restore a blanket conversion disable, because host-native CLI launchers need normal MSYS path conversion for their own installation paths.
- `AioSandboxProvider` (`packages/harness/deerflow/community/`) - Docker-based isolation. Active-cache and warm-pool entries are checked with the backend during acquire/reuse; definitively dead containers are dropped from all in-process maps so the thread can discover or create a fresh sandbox instead of reusing a stale client. Backend health-check failures are treated as unknown, not dead; local discovery likewise treats an unverifiable container as not adoptable and falls through to create rather than failing acquire. `get()` remains an in-memory lookup for event-loop-safe tool paths — it never touches the ownership store (that would be blocking IO on the event loop); ownership is published on acquire/reclaim and refreshed off the event loop by the dedicated renewal thread (`_renew_owned_leases`). `reset()` closes the per-instance acquire serializer so replacing the singleton cannot retain its executor workers; full remote sandbox teardown remains `shutdown()`. `uses_thread_data_mounts` defaults to backend detection (`LocalContainerBackend=True`, remote/provisioner backends=False), while the optional `sandbox.thread_data_mounts` boolean takes precedence for deployments that guarantee the Gateway and sandbox share the same thread user-data directories. Setting it `true` skips upload-time sandbox acquire/sync; a false positive leaves uploads unavailable to the sandbox. An explicit Agent policy uses four thread projection category mounts and a distinct deterministic sandbox identity, preventing reuse of an older container created with shared mounts. `skills.container_path` is a provider-startup snapshot shared by mount construction, sandbox identity, the remote Gateway request, and provisioner validation; custom roots are identity-scoped so a container or Pod created for one destination cannot be reused after the root changes. The Gateway and provisioner independently require one canonical absolute root that does not overlap reserved platform mounts, and both derive the four category allowlist entries from that root. The provisioner accepts all four category overrides; when all are present it suppresses the default hostPath or skills-PVC mount. With `USERDATA_PVC_NAME`, the thread projection categories use subpaths on that shared data PVC. Readiness probes and `agent_sandbox` clients classify loopback/private IPs, single-label cluster hosts, and Docker/Podman internal hostnames as direct control-plane destinations and set `trust_env=False`; external FQDNs and public IPs retain environment proxy support.
- `E2BSandboxProvider` (`packages/harness/deerflow/community/e2b_sandbox/`) provides E2B remote isolation.
New sandboxes receive a one-shot upload from the enabled-only public, custom,
legacy, and managed integration projections. Existing E2B VMs keep their
creation-time snapshot because E2B has no shared host mount.
Acquire and release share an `AcquireSerializer` hold keyed by `(user_id, thread_id)`. The serializer does
New unrestricted sandboxes receive a one-shot upload from the enabled-only
public, custom, legacy, and managed integration projections. For a thread
with an explicit Agent policy, creation skips that shared upload; after
acquire, `sync_agent_skills` clears only DeerFlow's four managed category
directories and signature before strictly uploading the signed thread
projection. It rejects non-canonical paths, protected mounts/homes, symlinked
roots, and standard operating-system trees before destructive work. E2B
rebuilds the managed categories on every policy sync; it deletes legacy
sandbox-visible signature markers instead of trusting them as proof that the
remote tree is intact. Reset plus all four uploads hold the same per-user/thread
and skills-root serializer used by acquire and release, so overlapping policy syncs cannot interleave
a wipe with another run's upload. Delegated subagents are non-owners of the lead's thread
projection: they reuse that filesystem view and never rebuild it from their
discovery/activation policy.
The provider snapshots `skills.container_path` at startup and carries it through
mount construction, the warm-pool seed, remote metadata, discovery, reconciliation,
and synchronization. A VM from a different root is never adopted and is reaped
after the duplicate grace period once no live peer owns it. Acquire and release
share an `AcquireSerializer` hold keyed by `(user_id, thread_id, skills_root)`. The serializer does
not cover remote IO. `burst_limit` adds capacity only for the `burst` policy.
The `wait` policy fails the turn after `acquire_timeout`. The runtime does not
retry the turn automatically. E2B acquisition uses a bounded executor.
@ -73,7 +88,7 @@
**Virtual Path System**:
- Agent sees: `/mnt/user-data/{workspace,uploads,outputs}`, `/mnt/skills`
- Physical: `backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/...`; raw skills stay under `deer-flow/skills/` and managed integration storage, while sandboxes read `backend/.deer-flow/skills_view/public/` and `backend/.deer-flow/users/{user_id}/skills_view/{custom,legacy,integrations}/`
- Physical: `backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/...`; raw skills stay under `deer-flow/skills/` and managed integration storage. Unrestricted sandboxes read `backend/.deer-flow/skills_view/public/` and `backend/.deer-flow/users/{user_id}/skills_view/{custom,legacy,integrations}/`; explicit lead Agent policies read `backend/.deer-flow/users/{user_id}/threads/{thread_id}/skills_view/{public,custom,legacy,integrations}/`.
- Translation: `LocalSandboxProvider` builds per-thread `PathMapping`s at acquire time. Sandbox-backed readers resolve only `/mnt/user-data/...` in the tool layer; skills, ACP workspaces, and configured custom mounts stay virtual so the provider mount table remains the single source of acquire-time identity and visibility. Full reads, ranged reads, and read-before-write hashing share this path. `tools.py` keeps `replace_virtual_path()` / `replace_virtual_paths_in_command()` as a defense-in-depth layer (and for path validation). AIO has the directories volume-mounted at the same virtual paths inside its container, so both implementations accept `/mnt/user-data/...` natively.
- Detection: `is_local_sandbox()` accepts both `sandbox_id == "local"` (legacy / no-thread) and `sandbox_id.startswith("local:")` (per-thread)

View File

@ -3,9 +3,11 @@ import threading
from collections import OrderedDict
from pathlib import Path
from deerflow.constants import DEFAULT_SKILLS_CONTAINER_PATH
from deerflow.sandbox.local.local_sandbox import LocalSandbox, PathMapping
from deerflow.sandbox.sandbox import Sandbox
from deerflow.sandbox.sandbox_provider import SandboxProvider
from deerflow.sandbox.security import is_host_bash_allowed
logger = logging.getLogger(__name__)
@ -60,11 +62,27 @@ class LocalSandboxProvider(SandboxProvider):
next ``acquire``; the evicted thread's next ``acquire`` rebuilds a fresh
sandbox (losing only its ``_agent_written_paths`` reverse-resolve hint,
which gracefully degrades read_file output).
The managed ``/mnt/skills`` projection is a logical boundary, not a host
filesystem security boundary. When host bash is enabled, a subprocess can
address canonical host paths without going through ``PathMapping``. The
provider therefore advertises Agent skill isolation only while host bash
remains disabled.
"""
uses_thread_data_mounts = True
needs_upload_permission_adjustment = False
@property
def supports_agent_skill_isolation(self) -> bool:
"""Whether the current tool surface can enforce the managed view."""
try:
return not is_host_bash_allowed()
except Exception:
# An unreadable config must not turn a host-process provider into
# an isolation boundary by accident.
return False
def __init__(self, max_cached_threads: int = DEFAULT_MAX_CACHED_THREAD_SANDBOXES):
"""Initialize the local sandbox provider with static path mappings.
@ -73,6 +91,7 @@ class LocalSandboxProvider(SandboxProvider):
the LRU cache. When exceeded, the least-recently-used entry is
evicted on the next ``acquire``.
"""
self._skills_container_path = DEFAULT_SKILLS_CONTAINER_PATH
self._path_mappings = self._setup_path_mappings()
self._generic_sandbox: LocalSandbox | None = None
self._thread_sandboxes: OrderedDict[tuple[str, str], LocalSandbox] = OrderedDict()
@ -101,6 +120,7 @@ class LocalSandboxProvider(SandboxProvider):
config = get_app_config()
container_path = config.skills.container_path
self._skills_container_path = container_path.rstrip("/")
projection = self._ensure_skills_projection()
# Public skills: global, read-only — static, shared by all threads
@ -132,6 +152,7 @@ class LocalSandboxProvider(SandboxProvider):
# ``/mnt/skills/custom`` to the init-time user's directory.
# Map custom mounts from sandbox config
_RESERVED_CONTAINER_PATHS = {container_path}
_RESERVED_CONTAINER_PREFIXES = [
f"{container_path}/public",
f"{container_path}/custom",
@ -163,7 +184,7 @@ class LocalSandboxProvider(SandboxProvider):
continue
# Reject mounts that conflict with reserved container paths
if any(container_path == p or container_path.startswith(p + "/") for p in _RESERVED_CONTAINER_PREFIXES):
if container_path in _RESERVED_CONTAINER_PATHS or any(container_path == p or container_path.startswith(p + "/") for p in _RESERVED_CONTAINER_PREFIXES):
logger.warning(
"Mount container_path conflicts with reserved prefix, skipping: %s",
mount.container_path,
@ -217,7 +238,7 @@ class LocalSandboxProvider(SandboxProvider):
return (user_id, thread_id)
@staticmethod
def _ensure_skills_projection(user_id: str | None = None):
def _ensure_skills_projection(user_id: str | None = None, *, thread_id: str | None = None):
"""Best-effort: a projection failure must not fail sandbox acquire.
Mirrors the surrounding skill-mount setup, which has always logged
@ -227,7 +248,11 @@ class LocalSandboxProvider(SandboxProvider):
acquire once the underlying condition clears.
"""
from deerflow.config import get_app_config
from deerflow.skills.projection import ensure_skill_projections
from deerflow.skills.projection import (
ensure_skill_projections,
get_thread_skill_projection_paths,
thread_skill_projection_exists,
)
from deerflow.skills.storage import get_or_new_skill_storage, get_or_new_user_skill_storage
try:
@ -236,9 +261,17 @@ class LocalSandboxProvider(SandboxProvider):
storage = get_or_new_skill_storage(app_config=config)
else:
storage = get_or_new_user_skill_storage(user_id, app_config=config)
if thread_id is not None and thread_skill_projection_exists(storage, thread_id):
return get_thread_skill_projection_paths(storage, thread_id)
return ensure_skill_projections(storage)
except Exception as exc:
logger.warning("Could not ensure skills projection for user %s: %s", user_id, exc, exc_info=True)
logger.warning(
"Could not ensure skills projection for user/thread %s/%s: %s",
user_id,
thread_id,
exc,
exc_info=True,
)
return None
@staticmethod
@ -326,7 +359,7 @@ class LocalSandboxProvider(SandboxProvider):
),
]
# Per-user category mounts stay present for the sandbox lifetime. Their
# Category mounts stay present for the sandbox lifetime. Their
# enabled-only contents change beneath these stable roots.
try:
config = get_app_config()
@ -334,30 +367,73 @@ class LocalSandboxProvider(SandboxProvider):
projection = skill_projection if skill_projection is not None else LocalSandboxProvider._ensure_skills_projection(effective_user_id)
if projection is not None:
mappings.extend(
[
PathMapping(
container_path=f"{skills_container_path}/custom",
local_path=str(projection.custom),
read_only=True,
),
PathMapping(
container_path=f"{skills_container_path}/legacy",
local_path=str(projection.legacy),
read_only=True,
),
PathMapping(
container_path=f"{skills_container_path}/integrations",
local_path=str(projection.integrations),
read_only=True,
),
]
thread_projection_root = paths.thread_skills_view_dir(
thread_id,
user_id=effective_user_id,
)
if projection.public.parent == thread_projection_root:
mappings.append(
PathMapping(
container_path=skills_container_path,
local_path=str(thread_projection_root),
read_only=True,
)
)
else:
mappings.extend(
[
PathMapping(
container_path=f"{skills_container_path}/public",
local_path=str(projection.public),
read_only=True,
),
PathMapping(
container_path=f"{skills_container_path}/custom",
local_path=str(projection.custom),
read_only=True,
),
PathMapping(
container_path=f"{skills_container_path}/legacy",
local_path=str(projection.legacy),
read_only=True,
),
PathMapping(
container_path=f"{skills_container_path}/integrations",
local_path=str(projection.integrations),
read_only=True,
),
]
)
except Exception as exc:
logger.warning("Could not setup per-thread skills projection mounts: %s", exc, exc_info=True)
return mappings
def _without_managed_skill_mappings(
self,
mappings: list[PathMapping],
*,
policy_scoped: bool,
) -> list[PathMapping]:
"""Drop mappings that would overlap the selected managed skill view.
Ordinary sandboxes retain operator-defined mounts elsewhere below the
configured skills root for backward compatibility. A policy-scoped
sandbox removes the entire subtree before installing its coherent root
mapping, so a nested custom mount cannot bypass the Agent allowlist.
"""
root = self._skills_container_path
managed_categories = tuple(f"{root}/{category}" for category in ("public", "custom", "legacy", "integrations"))
def conflicts(mapping: PathMapping) -> bool:
path = mapping.container_path.rstrip("/")
if path == root:
return True
prefixes = (root,) if policy_scoped else managed_categories
return any(path == prefix or path.startswith(prefix + "/") for prefix in prefixes)
return [mapping for mapping in mappings if not conflicts(mapping)]
def acquire(self, thread_id: str | None = None, *, user_id: str | None = None) -> str:
"""Return a sandbox id scoped to *thread_id* (or the generic singleton).
@ -390,23 +466,28 @@ class LocalSandboxProvider(SandboxProvider):
# triggers a full rebuild (~400 ms measured locally) under the
# cross-process projection lock, serializing concurrent acquires and
# mutations for that user. Acceptable for an editing-frequency event.
skill_projection = self._ensure_skills_projection(effective_user_id)
skill_projection = self._ensure_skills_projection(
effective_user_id,
thread_id=thread_id,
)
key = self._thread_key(thread_id, effective_user_id)
# Fast path under lock.
with self._lock:
cached = self._thread_sandboxes.get(key)
if cached is not None:
# Mark as most-recently used so frequently-touched threads
# survive eviction.
self._thread_sandboxes.move_to_end(key)
if cached is not None:
return cached.id
# ``_build_thread_path_mappings`` touches the filesystem
# (``ensure_thread_dirs``); release the lock during I/O.
new_mappings = list(self._path_mappings)
self._append_public_skill_mapping(new_mappings, skill_projection)
from deerflow.config.paths import get_paths
policy_scoped = bool(
skill_projection is not None
and skill_projection.public.parent
== get_paths().thread_skills_view_dir(
thread_id,
user_id=effective_user_id,
)
)
new_mappings = self._without_managed_skill_mappings(
list(self._path_mappings),
policy_scoped=policy_scoped,
)
new_mappings += self._build_thread_path_mappings(
thread_id,
user_id=effective_user_id,
@ -414,11 +495,15 @@ class LocalSandboxProvider(SandboxProvider):
)
with self._lock:
# Re-check after the lock-free I/O: another caller may have
# populated the cache while we were computing mappings.
cached = self._thread_sandboxes.get(key)
if cached is None:
cached = LocalSandbox(self._sandbox_id_for_thread(thread_id, effective_user_id), path_mappings=new_mappings)
if cached is None or cached.path_mappings != new_mappings:
replacement = LocalSandbox(
self._sandbox_id_for_thread(thread_id, effective_user_id),
path_mappings=new_mappings,
)
if cached is not None:
replacement._agent_written_paths.update(cached._agent_written_paths)
cached = replacement
self._thread_sandboxes[key] = cached
self._evict_until_within_cap_locked()
else:

View File

@ -9,7 +9,7 @@ from langchain.agents.middleware import AgentMiddleware
from langchain_core.messages import ToolMessage
from langgraph.prebuilt.tool_node import ToolCallRequest
from langgraph.runtime import Runtime
from langgraph.types import Command
from langgraph.types import Command, Overwrite
from deerflow.agents.thread_state import SandboxStateField, ThreadDataState
from deerflow.authz.sandbox_authz import (
@ -20,7 +20,7 @@ from deerflow.authz.sandbox_authz import (
)
from deerflow.runtime.user_context import resolve_runtime_user_id
from deerflow.sandbox import get_sandbox_provider
from deerflow.sandbox.exceptions import SandboxAuthorizationError
from deerflow.sandbox.exceptions import SandboxAuthorizationError, SandboxRuntimeError
from deerflow.sandbox.overwrite import unwrap_sandbox
logger = logging.getLogger(__name__)
@ -46,16 +46,67 @@ class SandboxMiddleware(AgentMiddleware[SandboxMiddlewareState]):
state_schema = SandboxMiddlewareState
def __init__(self, lazy_init: bool = True):
def __init__(
self,
lazy_init: bool = True,
*,
available_skills: set[str] | None = None,
owns_agent_skill_projection: bool = True,
):
"""Initialize sandbox middleware.
Args:
lazy_init: If True, defer sandbox acquisition until first tool call.
If False, acquire sandbox eagerly in before_agent().
Default is True for optimal performance.
owns_agent_skill_projection: Whether this middleware may create or
rebuild the thread's physical skill projection. Delegated
subagents share the lead thread sandbox and must preserve the
lead-owned view instead of applying their discovery policy to it.
"""
super().__init__()
self._lazy_init = lazy_init
self._available_skills = set(available_skills) if available_skills is not None else None
self._owns_agent_skill_projection = owns_agent_skill_projection
def _prepare_agent_skill_projection(self, thread_id: str, *, user_id: str):
"""Build the run's physical skill view before any sandbox is reused."""
if not self._owns_agent_skill_projection:
# Subagents inherit the lead's thread id and sandbox state. Their
# skill lists scope discovery/activation only; rebuilding here
# would widen or narrow the shared filesystem for every concurrent
# agent using this sandbox.
return None
from deerflow.config.paths import get_paths
# Preserve the zero-copy shared view for ordinary threads. A thread
# that previously used a restricted Agent keeps its stable mount root;
# an unrestricted run repopulates that root with all enabled skills.
if self._available_skills is None and not get_paths().thread_skills_view_dir(thread_id, user_id=user_id).exists():
return None
provider = get_sandbox_provider()
if not provider.supports_agent_skill_isolation:
if self._available_skills is not None:
raise SandboxRuntimeError(f"Sandbox provider {provider.__class__.__name__} cannot enforce per-Agent skill filesystem isolation")
# The thread projection may have been created under a different
# provider. An unrestricted run does not need that policy view and
# may safely use this provider's ordinary shared skill behavior.
return None
from deerflow.config import get_app_config
from deerflow.skills.projection import ensure_thread_skill_projection
from deerflow.skills.storage import get_or_new_user_skill_storage
app_config = get_app_config()
storage = get_or_new_user_skill_storage(user_id, app_config=app_config)
return ensure_thread_skill_projection(storage, thread_id, self._available_skills)
@staticmethod
def _require_projection_support(provider, projection) -> None:
if projection is not None and not provider.supports_agent_skill_isolation:
raise SandboxRuntimeError(f"Sandbox provider {provider.__class__.__name__} cannot enforce per-Agent skill filesystem isolation")
def _acquire_sandbox(self, thread_id: str, *, user_id: str) -> str:
provider = get_sandbox_provider()
@ -74,62 +125,109 @@ class SandboxMiddleware(AgentMiddleware[SandboxMiddlewareState]):
@override
def before_agent(self, state: SandboxMiddlewareState, runtime: Runtime) -> dict | None:
# Skip acquisition if lazy_init is enabled
if self._lazy_init:
thread_id = (runtime.context or {}).get("thread_id")
if thread_id is None:
return super().before_agent(state, runtime)
user_id = resolve_runtime_user_id(runtime)
projection = self._prepare_agent_skill_projection(thread_id, user_id=user_id)
# Preserve lazy initialization for threads that use the shared view.
# A policy-scoped view is acquired eagerly so an old shared-view
# sandbox cannot survive into this run through checkpoint state.
if self._lazy_init and projection is None:
return super().before_agent(state, runtime)
# Eager initialization (original behavior)
if "sandbox" not in state or state["sandbox"] is None:
thread_id = (runtime.context or {}).get("thread_id")
if thread_id is None:
return super().before_agent(state, runtime)
existing_sandbox_id = self._read_sandbox_id_from_state(state)
if existing_sandbox_id is None or projection is not None:
# Phase 3: enforce sandbox:execute authorization before acquiring
# (eager path). On deny, skip the eager acquisition instead of
# raising: an exception here is outside any tool call, so it would
# surface as a run-level graph error rather than the RFC §9
# friendly ToolMessage. Skipping defers to the lazy gate inside
# ``ensure_sandbox_initialized``, which denies per-tool with the
# friendly message on the first sandbox-touching tool call.
# friendly ToolMessage. Shared-view runs skip and defer to the lazy
# gate inside ``ensure_sandbox_initialized``. Policy-scoped runs
# abort here because retaining an older checkpointed sandbox would
# bypass the new filesystem view.
try:
authorize_sandbox_execution(
context=runtime.context or {},
app_config=safe_app_config(),
)
except SandboxAuthorizationError:
if projection is not None:
# An explicit skill policy cannot leave a checkpointed,
# previously shared sandbox reusable by downstream tools.
# Abort this run before the model can reach that state.
raise
logger.info("Sandbox execution denied for this role; skipping eager sandbox acquisition (thread_id=%s)", thread_id)
return None
sandbox_id = self._acquire_sandbox(thread_id, user_id=resolve_runtime_user_id(runtime))
provider = get_sandbox_provider()
self._require_projection_support(provider, projection)
sandbox_id = self._acquire_sandbox(thread_id, user_id=user_id)
if projection is not None:
provider.sync_agent_skills(
sandbox_id,
thread_id=thread_id,
user_id=user_id,
projection=projection,
)
logger.info(f"Assigned sandbox {sandbox_id} to thread {thread_id}")
if existing_sandbox_id == sandbox_id:
return super().before_agent(state, runtime)
if existing_sandbox_id is not None:
return {
"sandbox": Overwrite({"sandbox_id": sandbox_id}),
}
return {"sandbox": {"sandbox_id": sandbox_id}}
return super().before_agent(state, runtime)
@override
async def abefore_agent(self, state: SandboxMiddlewareState, runtime: Runtime) -> dict | None:
# Skip acquisition if lazy_init is enabled
if self._lazy_init:
thread_id = (runtime.context or {}).get("thread_id")
if thread_id is None:
return await super().abefore_agent(state, runtime)
user_id = resolve_runtime_user_id(runtime)
projection = await asyncio.to_thread(
self._prepare_agent_skill_projection,
thread_id,
user_id=user_id,
)
if self._lazy_init and projection is None:
return await super().abefore_agent(state, runtime)
# Eager initialization (original behavior), but use the async provider
# hook so blocking sandbox startup/polling runs outside the event loop.
if "sandbox" not in state or state["sandbox"] is None:
thread_id = (runtime.context or {}).get("thread_id")
if thread_id is None:
return await super().abefore_agent(state, runtime)
existing_sandbox_id = self._read_sandbox_id_from_state(state)
if existing_sandbox_id is None or projection is not None:
# Phase 3: enforce sandbox:execute authorization before acquiring
# (eager path, async counterpart of the gate in before_agent). On
# deny, skip the eager acquisition — the lazy gate inside
# ``ensure_sandbox_initialized`` denies per-tool with the RFC §9
# friendly message on the first sandbox-touching tool call.
# deny, shared-view runs skip and defer to the lazy tool gate;
# policy-scoped runs abort before an older sandbox can be reused.
try:
await authorize_sandbox_execution_async(
context=runtime.context or {},
app_config=await safe_app_config_async(),
)
except SandboxAuthorizationError:
if projection is not None:
raise
logger.info("Sandbox execution denied for this role; skipping eager sandbox acquisition (thread_id=%s)", thread_id)
return None
sandbox_id = await self._acquire_sandbox_async(thread_id, user_id=resolve_runtime_user_id(runtime))
provider = get_sandbox_provider()
self._require_projection_support(provider, projection)
sandbox_id = await self._acquire_sandbox_async(thread_id, user_id=user_id)
if projection is not None:
await provider.sync_agent_skills_async(
sandbox_id,
thread_id=thread_id,
user_id=user_id,
projection=projection,
)
logger.info(f"Assigned sandbox {sandbox_id} to thread {thread_id}")
if existing_sandbox_id == sandbox_id:
return await super().abefore_agent(state, runtime)
if existing_sandbox_id is not None:
return {
"sandbox": Overwrite({"sandbox_id": sandbox_id}),
}
return {"sandbox": {"sandbox_id": sandbox_id}}
return await super().abefore_agent(state, runtime)
@ -198,7 +296,7 @@ class SandboxMiddleware(AgentMiddleware[SandboxMiddlewareState]):
def _read_sandbox_id_from_state(state: object) -> str | None:
if not isinstance(state, dict):
return None
sandbox_state = state.get("sandbox")
sandbox_state, _ = unwrap_sandbox(state.get("sandbox"))
if not isinstance(sandbox_state, dict):
return None
sandbox_id = sandbox_state.get("sandbox_id")

View File

@ -1,17 +1,25 @@
import asyncio
import threading
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING
from deerflow.config import get_app_config
from deerflow.reflection import resolve_class
from deerflow.sandbox.sandbox import Sandbox
if TYPE_CHECKING:
from deerflow.skills.projection import SkillProjectionPaths
class SandboxProvider(ABC):
"""Abstract base class for sandbox providers"""
uses_thread_data_mounts: bool = False
needs_upload_permission_adjustment: bool = True
# Capability for enforcing a lead Agent's physical skill view across the
# provider's current Agent-accessible tool surface. Host-backed providers
# must return False whenever shell access can bypass managed path mappings.
supports_agent_skill_isolation: bool = False
@abstractmethod
def acquire(self, thread_id: str | None = None, *, user_id: str | None = None) -> str:
@ -32,6 +40,37 @@ class SandboxProvider(ABC):
"""
return await asyncio.to_thread(self.acquire, thread_id, user_id=user_id)
def sync_agent_skills(
self,
sandbox_id: str,
*,
thread_id: str,
user_id: str,
projection: "SkillProjectionPaths",
) -> None:
"""Synchronize a prepared thread skill projection into a sandbox.
Bind-mount providers observe the stable projection roots directly and
use this no-op implementation. Upload-based providers override it.
"""
async def sync_agent_skills_async(
self,
sandbox_id: str,
*,
thread_id: str,
user_id: str,
projection: "SkillProjectionPaths",
) -> None:
"""Async wrapper for upload-based skill synchronization."""
await asyncio.to_thread(
self.sync_agent_skills,
sandbox_id,
thread_id=thread_id,
user_id=user_id,
projection=projection,
)
@abstractmethod
def get(self, sandbox_id: str) -> Sandbox | None:
"""Get a sandbox environment by ID.

View File

@ -4,8 +4,8 @@
- **Format**: Directory with `SKILL.md` (YAML frontmatter: name, description, license, allowed-tools as a spec-compatible string or YAML list, argument-hint, required-secrets). Exact portable spellings such as `Bash`, `WebFetch`, `WebSearch`, `Glob`, `Grep`, `Read`, `Write`, and `Edit` map to `bash`, `web_fetch`, `web_search`, `glob`, `grep`, `read_file`, `write_file`, and `str_replace`; lowercase or otherwise unknown scalar names and YAML-list entries preserve their exact runtime spelling. Argument-scoped entries remain literal and inactive because the tool policy does not inspect arguments; the scalar tokenizer keeps spaces, quotes, and escaped parentheses inside patterns intact.
- **Loading**: `load_skills()` recursively scans public, per-user custom, global integration, and legacy custom locations for `SKILL.md`, parses metadata, and reads enabled state from extensions_config.json plus per-user skill state for non-public categories; that directory is a package boundary, so no nested `SKILL.md` is registered as a runtime skill. A custom skill directory may be a one-level symlink to an external directory for compatibility with operator-managed skill trees; activation still rejects a symlinked `SKILL.md` or deeper path escape. SkillScan has a deliberately narrower packaging rule: known eval fixtures are permitted as support data, while other nested `SKILL.md` files are reported as package defects. It parses runtime metadata and reads enabled state from extensions_config.json.
- **External reload**: `POST /api/skills/reload` is an admin-only, process-local invalidation hook for trusted MinIO/NFS/CSI writes. `SkillStorage` instances do not cache a catalog — `load_skills()` scans on every call — so the route clears all `(app_config, user_id)` entries and the rendered prompt-section LRU, then waits up to the shared refresh timeout for the existing off-loop single-flight refresh. Each invalidation receives a generation-bound result handle; a successful scan atomically replaces the global enabled-skills cache, while a loader-level failure propagates to the HTTP waiter and preserves the last-known-good global cache. Per-user/config scans capture the refresh version and cannot repopulate shared caches if invalidation occurs while they are loading. A timed-out HTTP wait fails generically while the daemon refresh worker continues. Subsequent runs rescan after a successful reload; active runs keep their existing snapshot. Each Uvicorn worker/Kubernetes Pod must be targeted separately. Direct mount writes bypass install/edit validation, SkillScan, and history, so mounted roots are an operator-controlled trust boundary.
- **Tool policy**: Agent `allowed-tools` declarations apply dynamically only to slash-activated skills and skills captured in `ThreadState.skill_context` through configured `read_file` loads; passive enabled skills and custom-agent/subagent skill allowlists remain discoverable without clamping the baseline toolset. Subagents render only skill discovery metadata at startup and reuse the same adjacent `SkillActivationMiddleware` + `SkillToolPolicyMiddleware` pair as the lead; their configured `skills` field limits discovery and activation instead of eagerly loading bodies or unioning policies. Slash policy is dominant for its run, preventing subsequently read skills from widening explicit authority; autonomous captured skills use the existing union only when no slash source exists. `tool_search` and `describe_skill` stay available as framework discovery infrastructure, while every discovered or promoted business tool still requires active-policy permission for schema visibility and execution; `task`, `list_background_tasks`, and `cancel_background_task` likewise require explicit declarations. Each active model call intentionally reloads the full live registry so enable/disable changes, frontmatter edits, and custom/public name-shadow winners take effect without a stale TTL or unsafe direct-path cache; all tool calls produced by that model step reuse the resulting source-and-path-signed decision. Registry failures and all-invalid active sets fail closed, while stale individual paths are skipped when another valid skill remains. This is best-effort behavioral scoping, not a hard security boundary: alternate loading paths are not captured and bounded autonomous context may evict entries.
- **Sandbox projection**: `skills/projection.py` materializes enabled-only trees at `{base_dir}/skills_view/public` and `{base_dir}/users/{user_id}/skills_view/{custom,legacy,integrations}`. It copies files into the view (`_copy_into_view`) so a sandbox write cannot mutate the canonical skill inode; the operational trade-off is an O(total bytes) I/O and per-user storage multiplier across rebuilds, prioritized for write isolation over zero-copy hardlinks. Steady-state freshness checks combine source and view metadata tree digests, so in-sandbox view tampering is detected and repaired on the next acquire. Storage writes, archive installs, deletes, and toggles rebuild under a cross-process lock; Gateway boot ensures only the shared public view, while each user view is repaired lazily on first sandbox acquire. Managed integration packages are global, but their projected category is per-user because enabled state is isolated. Rebuilds stage a complete tree and reconcile it with per-file atomic replacement, so unrelated enabled skills remain continuously visible; disable/delete paths remove only the affected package before mutating to preserve fail-closed behavior. User projection rebuilds re-read global enable state from disk instead of the process singleton, so a toggle handled by another Gateway worker is reflected on the next acquire. Gateway public-skill toggles take the public projection lock before the shared `extensions_config_write_lock`, re-read an existing config from disk, persist the full model shape, and rebuild before responding; keep this as one worker-owned critical section so MCP writes cannot interleave and request cancellation cannot release either lock while the worker still runs. The shared public steady-state signature check runs without the global projection lock; stale/error paths take the lock and re-check before rebuilding or clearing. User-scope checks remain serialized per user. Category root inodes remain stable so live bind mounts observe content changes without sandbox recreation. Projection failures clear the affected view before raising.
- **Tool policy**: Agent `allowed-tools` declarations apply dynamically only to slash-activated skills and skills captured in `ThreadState.skill_context` through configured `read_file` loads; passive enabled skills and skill allowlists do not clamp the baseline tool set. A lead custom Agent's explicit `skills` list is additionally enforced at the sandbox filesystem layer (see Sandbox projection); subagent skill lists still scope discovery and activation only because concurrently delegated subagents share the lead thread sandbox. Subagents render only skill discovery metadata at startup and reuse the same adjacent `SkillActivationMiddleware` + `SkillToolPolicyMiddleware` pair as the lead; their configured `skills` field limits discovery and activation instead of eagerly loading bodies or unioning policies. Slash policy is dominant for its run, preventing subsequently read skills from widening explicit authority; autonomous captured skills use the existing union only when no slash source exists. `tool_search` and `describe_skill` stay available as framework discovery infrastructure, while every discovered or promoted business tool still requires active-policy permission for schema visibility and execution; `task`, `list_background_tasks`, and `cancel_background_task` likewise require explicit declarations. Each active model call intentionally reloads the full live registry so enable/disable changes, frontmatter edits, and custom/public name-shadow winners take effect without a stale TTL or unsafe direct-path cache; all tool calls produced by that model step reuse the resulting source-and-path-signed decision. Registry failures and all-invalid active sets fail closed, while stale individual paths are skipped when another valid skill remains. The dynamic `allowed-tools` policy remains best-effort behavioral scoping: alternate loading paths are not captured and bounded autonomous context may evict entries.
- **Sandbox projection**: `skills/projection.py` materializes enabled-only shared trees at `{base_dir}/skills_view/public` and `{base_dir}/users/{user_id}/skills_view/{custom,legacy,integrations}`. A lead Agent with an explicit `skills` allowlist (including `[]`) gets the intersection of enabled public/user-visible skills and that allowlist at `{base_dir}/users/{user_id}/threads/{thread_id}/skills_view/{public,custom,legacy,integrations}`. `skills=None` keeps the shared zero-copy mount until a thread has used an explicit policy; later unrestricted runs repopulate the same stable thread root with all enabled skills. Rebuilds sign source state, view state, and normalized policy in a manifest, revoke every old category before adding the new policy, stage copies in temporary directories, and atomically replace files. Category root inodes stay stable for live bind mounts; concurrent readers can briefly see fewer skills during a policy change, never a skill revoked by the new policy. Policy-scoped copies reject absolute symlinks and relative symlinks that resolve outside their own skill package, preventing a permitted package from linking back to an omitted source. It copies files into the view (`_copy_into_view`) so a sandbox write cannot mutate the canonical skill inode; the operational trade-off is an O(total bytes) I/O and per-user/thread storage multiplier across rebuilds, prioritized for write isolation over zero-copy hardlinks. Steady-state freshness checks combine source and view metadata tree digests, so in-sandbox view tampering is detected and repaired on the next acquire. Storage writes, archive installs, deletes, and toggles rebuild shared scopes under a cross-process lock; Gateway boot ensures only the shared public view, user views are repaired lazily on acquire, and Agent thread views are recomputed before sandbox reuse. Managed integration packages are global, but their projected category is per-user because enabled state is isolated. User projection rebuilds re-read global enable state from disk instead of the process singleton, so a toggle handled by another Gateway worker is reflected on the next acquire. Gateway public-skill toggles take the public projection lock before the shared `extensions_config_write_lock`, re-read an existing config from disk, persist the full model shape, and rebuild before responding; keep this as one worker-owned critical section so MCP writes cannot interleave and request cancellation cannot release either lock while the worker still runs. The shared public steady-state signature check runs without the global projection lock; stale/error paths take the lock and re-check before rebuilding or clearing. User and thread scope checks are serialized per scope. Projection failures clear the affected view before raising.
- **Injection (legacy / default)**: Enabled skills are listed in the agent system prompt with full metadata and container paths (`<available_skills>` block). Controlled by `skills.deferred_discovery: false` (default).
- **Deferred discovery** (`skills.deferred_discovery: true`): Skills are listed by name only in a compact `<skill_index>` block, keeping the system prompt prefix-cache friendly. The agent calls the `describe_skill` tool at runtime to fetch full metadata for skills it wants to use, then loads the SKILL.md via `read_file`. Two new modules support this path:
- `skills/catalog.py``SkillCatalog` (immutable, searchable; query forms: `select:a,b`, `+prefix`, free-text regex); `select:` returns all requested skills without a result cap; other modes cap at `MAX_RESULTS=5`.

View File

@ -34,6 +34,7 @@ _locks_guard = threading.Lock()
_process_locks: dict[Path, threading.RLock] = {}
_MANIFEST_VERSION = 1
_MAX_REBUILD_ATTEMPTS = 2
_THREAD_PROJECTION_POLICY_VERSION = 1
@dataclass(frozen=True)
@ -66,6 +67,29 @@ def get_skill_projection_paths(storage: SkillStorage) -> SkillProjectionPaths:
)
def get_thread_skill_projection_paths(storage: SkillStorage, thread_id: str) -> SkillProjectionPaths:
"""Return stable category roots for one user's thread policy view."""
from deerflow.config.paths import get_paths
paths = getattr(storage, "_paths", None) or get_paths()
user_id = getattr(storage, "user_id", None)
if user_id is None:
raise ValueError("Thread skill projections require user-scoped skill storage")
root = paths.thread_skills_view_dir(thread_id, user_id=user_id)
return SkillProjectionPaths(
public=root / SkillCategory.PUBLIC.value,
custom=root / SkillCategory.CUSTOM.value,
legacy=root / SkillCategory.LEGACY.value,
integrations=root / SkillCategory.INTEGRATION.value,
)
def thread_skill_projection_exists(storage: SkillStorage, thread_id: str) -> bool:
"""Whether this thread has crossed into a stable policy-scoped view."""
paths = get_thread_skill_projection_paths(storage, thread_id)
return paths.public.parent.exists()
def _lock_for(path: Path) -> threading.RLock:
resolved = path.resolve()
with _locks_guard:
@ -103,7 +127,42 @@ def _copy_into_view(source: str, target: str, *, follow_symlinks: bool = True) -
return target
def _stage_skill(source: Path, target: Path, nested_skill_roots: set[Path]) -> None:
def _validate_projected_skill_symlinks(source: Path) -> None:
"""Reject links that would escape a policy-scoped skill package.
Relative links to support files inside the same package remain useful and
resolve inside the copied projection. Absolute links would keep pointing at
the canonical host tree, and links outside the package could expose an
omitted skill, so both fail closed before any live view is changed.
"""
package_root = source.resolve(strict=True)
for current_root, dir_names, file_names in os.walk(source, followlinks=False):
current = Path(current_root)
for name in (*dir_names, *file_names):
path = current / name
if not path.is_symlink():
continue
raw_target = Path(os.readlink(path))
if raw_target.is_absolute():
raise ValueError(f"Policy-scoped skill contains an absolute symlink: {path}")
try:
resolved_target = path.resolve(strict=True)
except (OSError, RuntimeError) as exc:
raise ValueError(f"Policy-scoped skill contains an invalid symlink: {path}") from exc
if not resolved_target.is_relative_to(package_root):
raise ValueError(f"Policy-scoped skill symlink escapes its package: {path}")
def _stage_skill(
source: Path,
target: Path,
nested_skill_roots: set[Path],
*,
enforce_symlink_boundary: bool = False,
) -> None:
if enforce_symlink_boundary:
_validate_projected_skill_symlinks(source)
def _exclude_nested_skills(current: str, names: list[str]) -> list[str]:
relative_root = Path(current).relative_to(source)
return [name for name in names if relative_root / name in nested_skill_roots]
@ -190,14 +249,25 @@ def _sync_staged_category(root: Path, staging: Path) -> None:
(staging / relative_path).replace(target)
def _replace_category(root: Path, desired: dict[Path, Skill], skill_boundaries: set[Path]) -> None:
def _replace_category(
root: Path,
desired: dict[Path, Skill],
skill_boundaries: set[Path],
*,
enforce_symlink_boundary: bool = False,
) -> None:
"""Reconcile entries beneath a stable category root without blanking it."""
root.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix=f".{root.name}.projection-", dir=root.parent) as staging_dir:
staging = Path(staging_dir)
for relative_path, skill in desired.items():
nested_roots = {boundary.relative_to(relative_path) for boundary in skill_boundaries if boundary != relative_path and boundary.is_relative_to(relative_path)}
_stage_skill(skill.skill_dir, staging / relative_path, nested_roots)
_stage_skill(
skill.skill_dir,
staging / relative_path,
nested_roots,
enforce_symlink_boundary=enforce_symlink_boundary,
)
_sync_staged_category(root, staging)
@ -343,7 +413,9 @@ def _view_signature(paths: SkillProjectionPaths, scope: str) -> str:
digest = hashlib.sha256()
if scope == "public":
_update_tree_digest(digest, paths.public, "public_view")
elif scope == "user":
elif scope in {"user", "thread"}:
if scope == "thread":
_update_tree_digest(digest, paths.public, "public_view")
_update_tree_digest(digest, paths.custom, "custom_view")
_update_tree_digest(digest, paths.legacy, "legacy_view")
_update_tree_digest(digest, paths.integrations, "integrations_view")
@ -352,6 +424,37 @@ def _view_signature(paths: SkillProjectionPaths, scope: str) -> str:
return digest.hexdigest()
def _thread_source_signature(
storage: SkillStorage,
allowed_skills: set[str] | None,
) -> str:
"""Sign every input that changes a thread's effective skill view."""
digest = hashlib.sha256()
digest.update(f"policy-version:{_THREAD_PROJECTION_POLICY_VERSION}\0".encode())
digest.update(f"public:{_source_signature(storage, 'public')}\0".encode())
digest.update(f"user:{_source_signature(storage, 'user')}\0".encode())
policy = None if allowed_skills is None else sorted(allowed_skills)
digest.update(json.dumps(policy, separators=(",", ":"), ensure_ascii=True).encode())
return digest.hexdigest()
def _thread_projection_is_fresh(
storage: SkillStorage,
paths: SkillProjectionPaths,
allowed_skills: set[str] | None,
) -> bool:
scope_root = paths.public.parent
if not all(path.is_dir() for path in (paths.public, paths.custom, paths.legacy, paths.integrations)):
return False
manifest_before = _read_manifest(scope_root)
if manifest_before is None or manifest_before.get("version") != _MANIFEST_VERSION:
return False
source_sig = _thread_source_signature(storage, allowed_skills)
view_sig = _view_signature(paths, "thread")
manifest_after = _read_manifest(scope_root)
return manifest_before == manifest_after and manifest_before.get("source_signature") == source_sig and manifest_before.get("view_signature") == view_sig
def _load_public_skills(storage: SkillStorage, *, enabled_only: bool) -> list[Skill]:
from deerflow.config.extensions_config import ExtensionsConfig
@ -443,6 +546,109 @@ def _rebuild_user_locked(storage: SkillStorage, paths: SkillProjectionPaths) ->
raise
def _rebuild_thread_locked(
storage: SkillStorage,
paths: SkillProjectionPaths,
allowed_skills: set[str] | None,
) -> None:
"""Rebuild one thread's effective view, removing old authority first.
Category roots stay inode-stable for live bind mounts. Clearing all four
before repopulating is deliberately fail-closed: concurrent readers may
briefly observe fewer allowed skills, but never a skill removed by the new
policy.
"""
scope_root = paths.public.parent
category_roots = (paths.public, paths.custom, paths.legacy, paths.integrations)
try:
for _attempt in range(_MAX_REBUILD_ATTEMPTS):
before = _thread_source_signature(storage, allowed_skills)
# User-scoped loading performs the same name-shadow resolution as
# prompt discovery. Use that effective catalog for every category;
# independently loading public skills would expose both copies when
# a custom or integration skill shadows a built-in skill by name.
all_effective_skills = storage.load_skills(enabled_only=False)
enabled_skills = [skill for skill in all_effective_skills if skill.enabled]
if allowed_skills is not None:
enabled_skills = [skill for skill in enabled_skills if skill.name in allowed_skills]
# Revoke the previous policy before exposing any part of the new
# one. The enclosing cross-process lock serializes competing runs.
_clear_projection_scope(scope_root, *category_roots)
_replace_category(
paths.public,
_by_relative_path(enabled_skills, SkillCategory.PUBLIC),
_category_boundaries(all_effective_skills, SkillCategory.PUBLIC),
enforce_symlink_boundary=True,
)
_replace_category(
paths.custom,
_by_relative_path(enabled_skills, SkillCategory.CUSTOM),
_category_boundaries(all_effective_skills, SkillCategory.CUSTOM),
enforce_symlink_boundary=True,
)
_replace_category(
paths.legacy,
_by_relative_path(enabled_skills, SkillCategory.LEGACY),
_category_boundaries(all_effective_skills, SkillCategory.LEGACY),
enforce_symlink_boundary=True,
)
_replace_category(
paths.integrations,
_by_relative_path(enabled_skills, SkillCategory.INTEGRATION),
_category_boundaries(all_effective_skills, SkillCategory.INTEGRATION),
enforce_symlink_boundary=True,
)
after = _thread_source_signature(storage, allowed_skills)
if before == after:
_write_manifest(scope_root, after, _view_signature(paths, "thread"))
return
raise RuntimeError("Skills changed repeatedly while rebuilding the thread sandbox projection")
except Exception:
_clear_projection_scope(scope_root, *category_roots)
raise
def ensure_thread_skill_projection(
storage: SkillStorage,
thread_id: str,
allowed_skills: set[str] | None,
) -> SkillProjectionPaths | None:
"""Ensure the filesystem view for one run's effective Agent skill policy.
``None`` keeps the existing shared projection for threads that have never
needed policy isolation. Once a thread has a scoped view, later unrestricted
runs rebuild that same stable mount with all enabled skills so switching
agents cannot leave the thread accidentally restricted.
"""
paths = get_thread_skill_projection_paths(storage, thread_id)
scope_root = paths.public.parent
if allowed_skills is None and not scope_root.exists():
return None
try:
fresh = _thread_projection_is_fresh(storage, paths, allowed_skills)
except Exception:
fresh = False
if fresh:
return paths
with _projection_lock(scope_root):
try:
if not _thread_projection_is_fresh(storage, paths, allowed_skills):
_rebuild_thread_locked(storage, paths, allowed_skills)
except Exception:
_clear_projection_scope(
scope_root,
paths.public,
paths.custom,
paths.legacy,
paths.integrations,
)
raise
return paths
def rebuild_skill_projections(
storage: SkillStorage,
*,

View File

@ -128,6 +128,61 @@ def test_start_container_logs_redacted_env_values(monkeypatch, caplog):
assert "visible-value" not in log_output
def test_start_container_filters_nested_config_mounts_for_policy_scoped_skills(
monkeypatch,
):
backend = LocalContainerBackend(
image="sandbox:latest",
base_port=8080,
container_prefix="sandbox",
config_mounts=[
SimpleNamespace(
host_path="/host/excluded-skill",
container_path="/mnt/skills/public/excluded-skill",
read_only=True,
),
SimpleNamespace(
host_path="/host/unrelated",
container_path="/mnt/unrelated",
read_only=True,
),
SimpleNamespace(
host_path="/host/sibling",
container_path="/mnt/skills-extra",
read_only=True,
),
],
environment={},
)
monkeypatch.setattr(backend, "_runtime", "docker")
captured_cmd: list[str] = []
def fake_run(cmd, **kwargs):
captured_cmd.extend(cmd)
return SimpleNamespace(stdout="container-id\n", stderr="", returncode=0)
monkeypatch.setattr("subprocess.run", fake_run)
backend._start_container(
"sandbox-test",
18080,
extra_mounts=[
(
"/host/thread-view/public",
"/mnt/skills/public",
True,
)
],
config_mount_exclusion_root="/mnt/skills",
)
command = " ".join(captured_cmd)
assert "/host/excluded-skill" not in command
assert "/host/unrelated" in command
assert "/host/sibling" in command
assert "/host/thread-view/public" in command
def _capture_start_container_command(monkeypatch, backend: LocalContainerBackend, runtime: str = "docker") -> list[str]:
monkeypatch.setattr(backend, "_runtime", runtime)
captured_cmd: list[str] = []
@ -812,7 +867,11 @@ def test_discover_brackets_ipv6_sandbox_host_for_url(monkeypatch, sandbox_host):
def test_create_brackets_ipv6_sandbox_host_for_url(monkeypatch, sandbox_host):
backend = _backend_for_inspect_tests()
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", sandbox_host)
monkeypatch.setattr(backend, "_start_container", lambda name, port, mounts=None: "container-id")
monkeypatch.setattr(
backend,
"_start_container",
lambda name, port, mounts=None, **_kwargs: "container-id",
)
monkeypatch.setattr("deerflow.community.aio_sandbox.local_backend.get_free_port", lambda start_port=None: 18082)
info = backend.create(thread_id="t", sandbox_id="sbx-ipv6")

View File

@ -43,6 +43,23 @@ def test_load_config_preserves_thread_data_mounts_override(sandbox_overrides, ex
provider = aio_mod.AioSandboxProvider.__new__(aio_mod.AioSandboxProvider)
assert provider._load_config()["thread_data_mounts"] is expected
assert provider._load_config()["skills_container_path"] == "/mnt/skills"
def test_load_config_snapshots_custom_skills_container_path(monkeypatch):
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
sandbox_config = SandboxConfig(
use="deerflow.community.aio_sandbox:AioSandboxProvider",
)
app_config = SimpleNamespace(
sandbox=sandbox_config,
stream_bridge=None,
skills=SimpleNamespace(container_path="/custom-skills"),
)
monkeypatch.setattr(aio_mod, "get_app_config", lambda: app_config)
provider = aio_mod.AioSandboxProvider.__new__(aio_mod.AioSandboxProvider)
assert provider._load_config()["skills_container_path"] == "/custom-skills"
@pytest.mark.parametrize(
@ -256,6 +273,7 @@ def test_get_extra_mounts_provisioner_payload_has_unique_container_paths(tmp_pat
monkeypatch.setattr(remote_backend, "user_should_see_legacy_skills", lambda *_args, **_kwargs: False)
provider = _make_provider(tmp_path)
provider._config["skills_container_path"] = config.skills.container_path
mounts = provider._get_extra_mounts("thread-1", user_id="alice")
container_paths = [container for _host, container, _read_only in mounts]
@ -279,7 +297,9 @@ def test_get_extra_mounts_provisioner_payload_has_unique_container_paths(tmp_pat
assert len(validated_paths) == len(set(validated_paths))
assert set(validated_paths) == {
"/mnt/acp-workspace",
"/mnt/skills/public",
"/mnt/skills/custom",
"/mnt/skills/legacy",
"/mnt/skills/integrations",
lark_cli.LARK_CLI_SANDBOX_CONFIG_DIR,
lark_cli.LARK_CLI_SANDBOX_LOCKS_DIR,
@ -288,6 +308,337 @@ def test_get_extra_mounts_provisioner_payload_has_unique_container_paths(tmp_pat
}
def test_thread_skill_projection_mounts_all_categories(
tmp_path,
monkeypatch,
):
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
paths = Paths(base_dir=tmp_path / "home")
projection_root = paths.thread_skills_view_dir(
"thread-policy",
user_id="alice",
)
for category in ("public", "custom", "legacy", "integrations"):
(projection_root / category).mkdir(parents=True, exist_ok=True)
config = SimpleNamespace(skills=SimpleNamespace(container_path="/mnt/skills"))
monkeypatch.setattr(aio_mod, "get_app_config", lambda: config)
monkeypatch.setattr(aio_mod, "get_paths", lambda: paths)
mounts = aio_mod.AioSandboxProvider._get_skills_mounts(
"thread-policy",
user_id="alice",
)
assert {container_path: host_path for host_path, container_path, _ in mounts} == {f"/mnt/skills/{category}": str(projection_root / category) for category in ("public", "custom", "legacy", "integrations")}
assert all(read_only for _host, _container, read_only in mounts)
def test_thread_skill_projection_uses_distinct_sandbox_identity(
tmp_path,
monkeypatch,
):
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
paths = Paths(base_dir=tmp_path / "home")
monkeypatch.setattr(aio_mod, "get_paths", lambda: paths)
monkeypatch.setattr(
aio_mod,
"get_app_config",
lambda: SimpleNamespace(skills=SimpleNamespace(container_path="/mnt/skills")),
)
provider = _make_provider(tmp_path)
shared_id = provider._sandbox_id_for_thread("thread-policy", "alice")
paths.thread_skills_view_dir(
"thread-policy",
user_id="alice",
).mkdir(parents=True)
policy_id = provider._sandbox_id_for_thread("thread-policy", "alice")
assert policy_id != shared_id
assert policy_id == provider._sandbox_id_for_thread("thread-policy", "alice")
assert policy_id != provider._deterministic_sandbox_id(
"thread-policy:agent-skills-v1",
"alice",
)
def test_policy_scoped_sandbox_identity_changes_with_skills_container_root(
tmp_path,
monkeypatch,
):
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
paths = Paths(base_dir=tmp_path / "home")
paths.thread_skills_view_dir(
"thread-policy",
user_id="alice",
).mkdir(parents=True)
monkeypatch.setattr(aio_mod, "get_paths", lambda: paths)
config = SimpleNamespace(skills=SimpleNamespace(container_path="/mnt/skills"))
monkeypatch.setattr(aio_mod, "get_app_config", lambda: config)
provider = _make_provider(tmp_path)
default_root_id = provider._sandbox_id_for_thread(
"thread-policy",
"alice",
)
config.skills.container_path = "/custom-skills"
provider._config["skills_container_path"] = config.skills.container_path
custom_root_id = provider._sandbox_id_for_thread(
"thread-policy",
"alice",
)
assert custom_root_id != default_root_id
def test_shared_sandbox_identity_changes_when_custom_skills_root_changes(
tmp_path,
monkeypatch,
):
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
paths = Paths(base_dir=tmp_path / "home")
monkeypatch.setattr(aio_mod, "get_paths", lambda: paths)
config = SimpleNamespace(skills=SimpleNamespace(container_path="/custom-skills-a"))
monkeypatch.setattr(aio_mod, "get_app_config", lambda: config)
provider = _make_provider(tmp_path)
provider._config["skills_container_path"] = config.skills.container_path
first_root_id = provider._sandbox_id_for_thread(
"thread-shared",
"alice",
)
config.skills.container_path = "/custom-skills-b"
provider._config["skills_container_path"] = config.skills.container_path
second_root_id = provider._sandbox_id_for_thread(
"thread-shared",
"alice",
)
assert second_root_id != first_root_id
def test_cached_sandbox_is_replaced_when_expected_identity_changes(
tmp_path,
monkeypatch,
):
provider = _make_provider(tmp_path)
provider._config["skills_container_path"] = "/custom-skills"
provider._thread_sandboxes = {("alice", "thread-shared"): "stale-root-id"}
provider._sandboxes = {"stale-root-id": object()}
provider._sandbox_infos = {}
monkeypatch.setattr(
provider,
"_sandbox_id_for_thread",
lambda *_args, **_kwargs: "new-root-id",
)
destroy = MagicMock()
monkeypatch.setattr(provider, "destroy", destroy)
assert (
provider._reuse_in_process_sandbox(
"thread-shared",
user_id="alice",
)
is None
)
destroy.assert_called_once_with("stale-root-id")
def test_policy_scoped_create_excludes_local_config_mounts_below_skills_root(
tmp_path,
monkeypatch,
):
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
paths = Paths(base_dir=tmp_path / "home")
paths.thread_skills_view_dir(
"thread-policy",
user_id="alice",
).mkdir(parents=True)
monkeypatch.setattr(aio_mod, "get_paths", lambda: paths)
monkeypatch.setattr(
aio_mod,
"get_app_config",
lambda: SimpleNamespace(skills=SimpleNamespace(container_path="/mnt/skills")),
)
provider = _make_provider(tmp_path)
provider._config = {
"replicas": 3,
"skills_container_path": "/mnt/skills",
}
provider._thread_locks = {}
provider._warm_pool = {}
provider._sandbox_infos = {}
provider._thread_sandboxes = {}
provider._last_activity = {}
provider._lock = aio_mod.threading.Lock()
captured: dict = {}
backend = object.__new__(aio_mod.LocalContainerBackend)
def _create(thread_id, sandbox_id, **kwargs):
captured.update(kwargs)
return aio_mod.SandboxInfo(
sandbox_id=sandbox_id,
sandbox_url="http://sandbox",
)
backend.create = _create
provider._backend = backend
monkeypatch.setattr(aio_mod, "wait_for_sandbox_ready", lambda *_a, **_k: True)
monkeypatch.setattr(provider, "_get_extra_mounts", lambda *_a, **_k: [])
monkeypatch.setattr(
aio_mod.AioSandboxProvider,
"_lark_integration_active",
staticmethod(lambda user_id=None: False),
)
monkeypatch.setattr(
aio_mod.AioSandboxProvider,
"_lark_broker_active",
staticmethod(lambda user_id=None: False),
)
monkeypatch.setattr(
provider,
"_register_created_sandbox",
lambda *a, **k: "sandbox-policy",
)
provider._create_sandbox(
"thread-policy",
"sandbox-policy",
user_id="alice",
)
assert captured["config_mount_exclusion_root"] == "/mnt/skills"
def test_remote_create_forwards_configured_skills_container_path(
tmp_path,
monkeypatch,
):
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
provider = _make_provider(tmp_path)
provider._config = {
"replicas": 3,
"skills_container_path": "/custom-skills",
}
provider._thread_locks = {}
provider._warm_pool = {}
provider._sandbox_infos = {}
provider._thread_sandboxes = {}
provider._last_activity = {}
provider._lock = aio_mod.threading.Lock()
captured: dict = {}
backend = aio_mod.RemoteSandboxBackend("http://provisioner:8002")
def _create(thread_id, sandbox_id, **kwargs):
captured.update(kwargs)
return aio_mod.SandboxInfo(
sandbox_id=sandbox_id,
sandbox_url="http://sandbox",
)
backend.create = _create
provider._backend = backend
monkeypatch.setattr(
aio_mod,
"get_app_config",
lambda: SimpleNamespace(skills=SimpleNamespace(container_path="/custom-skills")),
)
monkeypatch.setattr(aio_mod, "wait_for_sandbox_ready", lambda *_a, **_k: True)
monkeypatch.setattr(provider, "_get_extra_mounts", lambda *_a, **_k: [])
monkeypatch.setattr(
aio_mod.AioSandboxProvider,
"_lark_integration_active",
staticmethod(lambda user_id=None: False),
)
monkeypatch.setattr(
aio_mod.AioSandboxProvider,
"_lark_broker_active",
staticmethod(lambda user_id=None: False),
)
monkeypatch.setattr(
provider,
"_register_created_sandbox",
lambda *a, **k: "sandbox-custom-root",
)
provider._create_sandbox(
"thread-custom-root",
"sandbox-custom-root",
user_id="alice",
)
assert captured["skills_container_path"] == "/custom-skills"
@pytest.mark.anyio
async def test_remote_create_async_forwards_configured_skills_container_path(
tmp_path,
monkeypatch,
):
aio_mod = importlib.import_module("deerflow.community.aio_sandbox.aio_sandbox_provider")
provider = _make_provider(tmp_path)
provider._config = {
"replicas": 3,
"skills_container_path": "/custom-skills",
}
provider._thread_locks = {}
provider._warm_pool = {}
provider._sandbox_infos = {}
provider._thread_sandboxes = {}
provider._last_activity = {}
provider._lock = aio_mod.threading.Lock()
captured: dict = {}
backend = aio_mod.RemoteSandboxBackend("http://provisioner:8002")
def _create(thread_id, sandbox_id, **kwargs):
captured.update(kwargs)
return aio_mod.SandboxInfo(
sandbox_id=sandbox_id,
sandbox_url="http://sandbox",
)
backend.create = _create
provider._backend = backend
monkeypatch.setattr(
aio_mod,
"get_app_config",
lambda: SimpleNamespace(skills=SimpleNamespace(container_path="/custom-skills")),
)
async def _ready(*_args, **_kwargs):
return True
monkeypatch.setattr(aio_mod, "wait_for_sandbox_ready_async", _ready)
monkeypatch.setattr(provider, "_get_extra_mounts", lambda *_a, **_k: [])
monkeypatch.setattr(
aio_mod.AioSandboxProvider,
"_lark_integration_active",
staticmethod(lambda user_id=None: False),
)
monkeypatch.setattr(
aio_mod.AioSandboxProvider,
"_lark_broker_active",
staticmethod(lambda user_id=None: False),
)
monkeypatch.setattr(
provider,
"_register_created_sandbox",
lambda *a, **k: "sandbox-custom-root",
)
await provider._create_sandbox_async(
"thread-custom-root",
"sandbox-custom-root",
user_id="alice",
)
assert captured["skills_container_path"] == "/custom-skills"
def test_join_host_path_preserves_windows_drive_letter_style():
base = r"C:\Users\demo\deer-flow\backend\.deer-flow"
@ -583,6 +934,7 @@ def test_remote_backend_create_forwards_effective_user_id(monkeypatch):
"thread_id": "thread-42",
"user_id": "user-7",
"include_legacy_skills": True,
"skills_container_path": "/mnt/skills",
"provision_lark_cli_runtime": False,
"provision_lark_cli_broker": False,
}

View File

@ -3,6 +3,7 @@
from __future__ import annotations
import asyncio
import hashlib
import importlib
import json
import os
@ -244,7 +245,15 @@ class FakeOwnershipStore:
return None
def _make_provider(*, replicas: int = 3, idle_timeout: int = 1800, overflow_policy: str = "wait", acquire_timeout: int = 30, burst_limit: int = 0) -> Any:
def _make_provider(
*,
replicas: int = 3,
idle_timeout: int = 1800,
overflow_policy: str = "wait",
acquire_timeout: int = 30,
burst_limit: int = 0,
skills_container_path: str = "/mnt/skills",
) -> Any:
"""Build a ``E2BSandboxProvider`` instance bypassing ``__init__``."""
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
provider = mod.E2BSandboxProvider.__new__(mod.E2BSandboxProvider)
@ -280,6 +289,7 @@ def _make_provider(*, replicas: int = 3, idle_timeout: int = 1800, overflow_poli
"template": "code-interpreter-v1",
"domain": None,
"home_dir": "/home/user",
"skills_container_path": skills_container_path,
"idle_timeout": idle_timeout,
"replicas": replicas,
"overflow_policy": overflow_policy,
@ -366,6 +376,471 @@ def test_apply_mounts_uploads_only_enabled_skill_projection(monkeypatch, tmp_pat
assert "/mnt/skills/integrations/lark-cli/disabled-integration/SKILL.md" not in uploaded_paths
def test_policy_scoped_thread_skips_shared_projection_during_create(
monkeypatch,
tmp_path,
):
paths = Paths(base_dir=tmp_path)
paths.thread_skills_view_dir("thread-1", user_id="user-1").mkdir(parents=True)
monkeypatch.setattr("deerflow.config.paths.get_paths", lambda: paths)
provider = _make_provider()
assert provider._skill_projection_mounts("user-1", "thread-1") == []
def test_sync_agent_skills_rebuilds_managed_remote_tree_despite_matching_legacy_marker(
monkeypatch,
tmp_path,
):
from deerflow.skills.projection import SkillProjectionPaths
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
root = tmp_path / "skills-view"
projection = SkillProjectionPaths(
public=root / "public",
custom=root / "custom",
legacy=root / "legacy",
integrations=root / "integrations",
)
for category in (
projection.public,
projection.custom,
projection.legacy,
projection.integrations,
):
category.mkdir(parents=True, exist_ok=True)
_write_skill(projection.public, "allowed-skill")
manifest_path = root / ".projection-manifest.json"
manifest_path.write_text(
json.dumps(
{
"version": 1,
"source_signature": "source-a",
"view_signature": "view-a",
}
),
encoding="utf-8",
)
legacy_signature = hashlib.sha256(manifest_path.read_bytes()).hexdigest()
monkeypatch.setattr(
mod,
"get_app_config",
lambda: SimpleNamespace(skills=SimpleNamespace(container_path="/mnt/skills")),
)
files = FakeFilesAPI(
{
"/mnt/skills/public/excluded-skill/SKILL.md": b"excluded",
"/mnt/skills/.deerflow-projection-signature": legacy_signature.encode(),
"/mnt/skills/unmanaged.txt": b"keep",
}
)
def reset_remote_tree(command: str):
assert "sudo rm -rf -- /mnt/skills;" not in command
assert "sudo chown -R" not in command
assert "if [ -L /mnt/skills ]" in command
managed_paths = (
"/mnt/skills/public",
"/mnt/skills/custom",
"/mnt/skills/legacy",
"/mnt/skills/integrations",
"/mnt/skills/.deerflow-projection-signature",
)
for managed_path in managed_paths:
assert managed_path in command
for path in list(files.store):
if any(path == managed_path or path.startswith(f"{managed_path}/") for managed_path in managed_paths):
files.store.pop(path)
return SimpleNamespace(
stdout="SKILLS_RESET_OK\n",
stderr="",
exit_code=0,
)
chmod_ok = SimpleNamespace(stdout="", stderr="", exit_code=0)
commands = FakeCommandsAPI([reset_remote_tree, chmod_ok, reset_remote_tree, chmod_ok])
client = FakeClient(sandbox_id="sandbox-1", commands=commands, files=files)
provider = _make_provider()
provider._sandboxes["sandbox-1"] = mod.E2BSandbox(
id="sandbox-1",
client=client,
home_dir="/home/user",
)
provider.sync_agent_skills(
"sandbox-1",
thread_id="thread-1",
user_id="user-1",
projection=projection,
)
assert "/mnt/skills/public/excluded-skill/SKILL.md" not in files.store
assert files.store["/mnt/skills/public/allowed-skill/SKILL.md"].startswith(b"---")
assert files.store["/mnt/skills/unmanaged.txt"] == b"keep"
assert "/mnt/skills/.deerflow-projection-signature" not in files.store
assert files.read_calls == []
first_command_count = len(commands.calls)
first_write_count = len(files.write_calls)
provider.sync_agent_skills(
"sandbox-1",
thread_id="thread-1",
user_id="user-1",
projection=projection,
)
assert len(commands.calls) == first_command_count * 2
assert len(files.write_calls) == first_write_count * 2
def test_sync_agent_skills_serializes_reset_and_upload_for_same_thread(
monkeypatch,
tmp_path,
):
from deerflow.skills.projection import SkillProjectionPaths
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
projections: list[SkillProjectionPaths] = []
for name in ("policy-a", "policy-b"):
root = tmp_path / name
projection = SkillProjectionPaths(
public=root / "public",
custom=root / "custom",
legacy=root / "legacy",
integrations=root / "integrations",
)
for category in (
projection.public,
projection.custom,
projection.legacy,
projection.integrations,
):
category.mkdir(parents=True)
projections.append(projection)
monkeypatch.setattr(
mod,
"get_app_config",
lambda: SimpleNamespace(skills=SimpleNamespace(container_path="/mnt/skills")),
)
first_upload_started = threading.Event()
allow_first_upload_to_finish = threading.Event()
second_sync_started = threading.Event()
second_reset_started = threading.Event()
reset_count = 0
reset_count_lock = threading.Lock()
def reset_remote_tree(_command: str):
nonlocal reset_count
with reset_count_lock:
reset_count += 1
current_reset = reset_count
if current_reset == 2:
second_reset_started.set()
return SimpleNamespace(stdout="SKILLS_RESET_OK\n", stderr="", exit_code=0)
commands = FakeCommandsAPI([reset_remote_tree, reset_remote_tree])
client = FakeClient(sandbox_id="sandbox-1", commands=commands)
provider = _make_provider()
provider._sandboxes["sandbox-1"] = mod.E2BSandbox(
id="sandbox-1",
client=client,
home_dir="/home/user",
)
first_projection_root = projections[0].public.parent
def blocking_upload(_client, source, _destination, _read_only, *, budget):
del budget
if source.parent == first_projection_root and not first_upload_started.is_set():
first_upload_started.set()
assert allow_first_upload_to_finish.wait(timeout=5)
monkeypatch.setattr(provider, "_upload_tree", blocking_upload)
errors: list[BaseException] = []
def sync(
projection: SkillProjectionPaths,
*,
started: threading.Event | None = None,
) -> None:
try:
if started is not None:
started.set()
provider.sync_agent_skills(
"sandbox-1",
thread_id="thread-1",
user_id="user-1",
projection=projection,
)
except BaseException as exc: # pragma: no cover - surfaced below
errors.append(exc)
first = threading.Thread(target=sync, args=(projections[0],))
second = threading.Thread(
target=sync,
args=(projections[1],),
kwargs={"started": second_sync_started},
)
first.start()
assert first_upload_started.wait(timeout=5)
second.start()
try:
assert second_sync_started.wait(timeout=5)
assert not second_reset_started.wait(timeout=0.2)
finally:
allow_first_upload_to_finish.set()
first.join(timeout=5)
second.join(timeout=5)
assert not first.is_alive()
assert not second.is_alive()
assert errors == []
assert second_reset_started.is_set()
@pytest.mark.parametrize(
"container_path",
[
"skills",
"/",
"//mnt/skills",
"/mnt//skills",
"/mnt/skills/.",
"/mnt/skills/..",
"/mnt",
"/mnt/user-data",
"/mnt/acp-workspace",
"/home",
"/home/user",
"/bin",
"/boot",
"/dev",
"/etc",
"/etc/deerflow-skills",
"/lib",
"/lib32",
"/lib64",
"/libx32",
"/lost+found",
"/media",
"/opt",
"/proc",
"/root",
"/run",
"/sbin",
"/snap",
"/srv",
"/sys",
"/tmp",
"/usr",
"/usr/local/deerflow-skills",
"/var",
"/var/lib/deerflow-skills",
],
)
def test_sync_agent_skills_rejects_unsafe_reset_roots_before_remote_access(
tmp_path,
container_path,
):
from deerflow.skills.projection import SkillProjectionPaths
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
root = tmp_path / "skills-view"
projection = SkillProjectionPaths(
public=root / "public",
custom=root / "custom",
legacy=root / "legacy",
integrations=root / "integrations",
)
client = FakeClient(sandbox_id="sandbox-1")
provider = _make_provider()
provider._config["skills_container_path"] = container_path
provider._sandboxes["sandbox-1"] = mod.E2BSandbox(
id="sandbox-1",
client=client,
home_dir="/home/user",
)
with pytest.raises(ValueError, match="safe E2B skills reset target"):
provider.sync_agent_skills(
"sandbox-1",
thread_id="thread-1",
user_id="user-1",
projection=projection,
)
assert client.files.read_calls == []
assert client.commands.calls == []
@pytest.mark.parametrize(
("container_path", "expected"),
[
("/mnt/skills", "/mnt/skills"),
("/mnt/skills/", "/mnt/skills"),
("/home/user/skills", "/home/user/skills"),
("/custom-skills", "/custom-skills"),
("/custom/skills", "/custom/skills"),
],
)
def test_validate_skills_reset_root_accepts_isolated_directories(
container_path,
expected,
):
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
assert mod._validate_skills_reset_root(container_path, home_dir="/home/user") == expected
@pytest.mark.parametrize(
("home_dir", "container_path"),
[
("/opt/e2b-home", "/opt/e2b-home/skills"),
("/tmp/e2b-home", "/tmp/e2b-home/skills"),
],
)
def test_validate_skills_reset_root_accepts_isolated_custom_home_subtree(
home_dir,
container_path,
):
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
assert (
mod._validate_skills_reset_root(
container_path,
home_dir=home_dir,
)
== container_path
)
def test_sync_agent_skills_rejects_symlinked_remote_root_before_deleting(
monkeypatch,
tmp_path,
):
from deerflow.skills.projection import SkillProjectionPaths
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
root = tmp_path / "skills-view"
projection = SkillProjectionPaths(
public=root / "public",
custom=root / "custom",
legacy=root / "legacy",
integrations=root / "integrations",
)
for category in (
projection.public,
projection.custom,
projection.legacy,
projection.integrations,
):
category.mkdir(parents=True, exist_ok=True)
manifest_path = root / ".projection-manifest.json"
manifest_path.write_text("{}", encoding="utf-8")
legacy_signature = hashlib.sha256(manifest_path.read_bytes()).hexdigest()
monkeypatch.setattr(
mod,
"get_app_config",
lambda: SimpleNamespace(skills=SimpleNamespace(container_path="/mnt/skills")),
)
def reject_symlinked_root(command: str):
assert "if [ -L /mnt/skills ]" in command
return SimpleNamespace(
stdout="",
stderr="Refusing symlinked skills root\n",
exit_code=2,
)
client = FakeClient(
sandbox_id="sandbox-1",
commands=FakeCommandsAPI([reject_symlinked_root]),
files=FakeFilesAPI(
{
"/mnt/skills/.deerflow-projection-signature": legacy_signature.encode(),
}
),
)
provider = _make_provider()
provider._sandboxes["sandbox-1"] = mod.E2BSandbox(
id="sandbox-1",
client=client,
home_dir="/home/user",
)
monkeypatch.setattr(
provider,
"_upload_tree",
lambda *_args, **_kwargs: (_ for _ in ()).throw(AssertionError("upload must not start after a rejected reset")),
)
with pytest.raises(RuntimeError, match="Failed to reset E2B skill projection"):
provider.sync_agent_skills(
"sandbox-1",
thread_id="thread-1",
user_id="user-1",
projection=projection,
)
assert client.files.read_calls == []
assert client.files.write_calls == []
def test_sync_agent_skills_leaves_no_signature_after_upload_failure(
monkeypatch,
tmp_path,
):
from deerflow.skills.projection import SkillProjectionPaths
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
root = tmp_path / "skills-view"
projection = SkillProjectionPaths(
public=root / "public",
custom=root / "custom",
legacy=root / "legacy",
integrations=root / "integrations",
)
for category in (
projection.public,
projection.custom,
projection.legacy,
projection.integrations,
):
category.mkdir(parents=True, exist_ok=True)
(root / ".projection-manifest.json").write_text("{}", encoding="utf-8")
monkeypatch.setattr(
mod,
"get_app_config",
lambda: SimpleNamespace(skills=SimpleNamespace(container_path="/mnt/skills")),
)
commands = FakeCommandsAPI([SimpleNamespace(stdout="SKILLS_RESET_OK\n", stderr="", exit_code=0)])
client = FakeClient(sandbox_id="sandbox-1", commands=commands)
provider = _make_provider()
provider._sandboxes["sandbox-1"] = mod.E2BSandbox(
id="sandbox-1",
client=client,
home_dir="/home/user",
)
monkeypatch.setattr(
provider,
"_upload_tree",
lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("upload failed")),
)
with pytest.raises(RuntimeError, match="upload failed"):
provider.sync_agent_skills(
"sandbox-1",
thread_id="thread-1",
user_id="user-1",
projection=projection,
)
assert "/mnt/skills/.deerflow-projection-signature" not in client.files.store
def test_upload_tree_streams_file_contents(tmp_path):
source = tmp_path / "large.bin"
source.write_bytes(b"mount content")
@ -831,6 +1306,7 @@ def test_load_config_clamps_invalid_mount_upload_deadline(monkeypatch, caplog, r
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
class FakeConfig:
skills = SimpleNamespace(container_path="/mnt/skills")
sandbox = SimpleNamespace(
model_extra={"mount_upload_deadline_seconds": raw},
api_key="test-key",
@ -1046,9 +1522,9 @@ def _make_sandbox(client: FakeClient, *, sandbox_id: str | None = None) -> Any:
)
def test_thread_key_returns_user_thread_tuple():
def test_thread_key_includes_the_provider_skills_root():
p = _make_provider()
assert p._thread_key("t1", "u1") == ("u1", "t1")
assert p._thread_key("t1", "u1") == ("u1", "t1", "/mnt/skills")
def test_sandbox_id_falls_back_when_client_id_is_none():
@ -1060,6 +1536,7 @@ def test_sandbox_id_falls_back_when_client_id_is_none():
def test_stable_seed_is_deterministic_and_user_scoped():
p = _make_provider()
custom_root = _make_provider(skills_container_path="/custom-skills")
s_a = p._stable_seed("t1", "u1")
s_b = p._stable_seed("t1", "u1")
s_other_user = p._stable_seed("t1", "u2")
@ -1067,6 +1544,7 @@ def test_stable_seed_is_deterministic_and_user_scoped():
assert s_a == s_b
assert s_a != s_other_user
assert s_a != s_other_thread
assert s_a != custom_root._stable_seed("t1", "u1")
def test_is_sandbox_gone_error_matches_known_signatures():
@ -1285,7 +1763,7 @@ def test_refresh_owned_leases_reclaims_lapsed_lease():
client = FakeClient(sandbox_id="sb-lapsed")
sandbox = _make_sandbox(client, sandbox_id="sb-lapsed")
p._sandboxes["sb-lapsed"] = sandbox
p._thread_sandboxes[("u1", "t1")] = "sb-lapsed"
p._thread_sandboxes[p._thread_key("t1", "u1")] = "sb-lapsed"
p._owned_sandbox_ids.add("sb-lapsed")
p._refresh_owned_leases()
@ -1300,7 +1778,8 @@ def test_refresh_owned_leases_forgets_peer_owned_sandbox():
client = FakeClient(sandbox_id="sb-lost")
sandbox = _make_sandbox(client, sandbox_id="sb-lost")
p._sandboxes["sb-lost"] = sandbox
p._thread_sandboxes[("u1", "t1")] = "sb-lost"
key = p._thread_key("t1", "u1")
p._thread_sandboxes[key] = "sb-lost"
p._owned_sandbox_ids.add("sb-lost")
p._ownership = FakeOwnershipStore(
{"sb-lost": ("owner-peer", "own")},
@ -1310,7 +1789,7 @@ def test_refresh_owned_leases_forgets_peer_owned_sandbox():
p._refresh_owned_leases()
assert p.get("sb-lost") is None
assert ("u1", "t1") not in p._thread_sandboxes
assert key not in p._thread_sandboxes
assert "sb-lost" not in p._owned_sandbox_ids
assert client.closed is True
@ -1320,7 +1799,7 @@ def test_reuse_in_process_sandbox_returns_cached_id_on_healthy_reuse():
client = FakeClient()
sb = _make_sandbox(client, sandbox_id="sb-1")
p._sandboxes["sb-1"] = sb
p._thread_sandboxes[("u1", "t1")] = "sb-1"
p._thread_sandboxes[p._thread_key("t1", "u1")] = "sb-1"
sid = p._reuse_in_process_sandbox("t1", user_id="u1")
assert sid == "sb-1"
@ -1333,12 +1812,13 @@ def test_reuse_in_process_sandbox_evicts_dead_sandbox():
sb = _make_sandbox(client, sandbox_id="sb-dead")
sb._dead = True
p._sandboxes["sb-dead"] = sb
p._thread_sandboxes[("u1", "t1")] = "sb-dead"
key = p._thread_key("t1", "u1")
p._thread_sandboxes[key] = "sb-dead"
sid = p._reuse_in_process_sandbox("t1", user_id="u1")
assert sid is None
assert "sb-dead" not in p._sandboxes
assert ("u1", "t1") not in p._thread_sandboxes
assert key not in p._thread_sandboxes
def test_reuse_in_process_sandbox_evicts_when_ping_fails():
@ -1346,7 +1826,7 @@ def test_reuse_in_process_sandbox_evicts_when_ping_fails():
client = FakeClient(commands=FakeCommandsAPI([FakeCommandsAPI.GONE]))
sb = _make_sandbox(client, sandbox_id="sb-stale")
p._sandboxes["sb-stale"] = sb
p._thread_sandboxes[("u1", "t1")] = "sb-stale"
p._thread_sandboxes[p._thread_key("t1", "u1")] = "sb-stale"
sid = p._reuse_in_process_sandbox("t1", user_id="u1")
assert sid is None
@ -1356,10 +1836,11 @@ def test_reuse_in_process_sandbox_evicts_when_ping_fails():
def test_reuse_in_process_sandbox_cleans_dangling_mapping():
p = _make_provider()
p._thread_sandboxes[("u1", "t1")] = "ghost"
key = p._thread_key("t1", "u1")
p._thread_sandboxes[key] = "ghost"
sid = p._reuse_in_process_sandbox("t1", user_id="u1")
assert sid is None
assert ("u1", "t1") not in p._thread_sandboxes
assert key not in p._thread_sandboxes
def test_reuse_in_process_sandbox_returns_none_when_no_mapping():
@ -1376,7 +1857,7 @@ def test_reclaim_warm_pool_sandbox_happy_path(monkeypatch):
sid = p._reclaim_warm_pool_sandbox("t1", user_id="u1")
assert sid == "sb-warm"
assert "sb-warm" in p._sandboxes
assert p._thread_sandboxes[("u1", "t1")] == "sb-warm"
assert p._thread_sandboxes[p._thread_key("t1", "u1")] == "sb-warm"
assert "sb-warm" not in p._warm_pool
assert [c[0] for c in fake_cls.connect_calls] == ["sb-warm"]
@ -1465,17 +1946,42 @@ class _FakePaginator:
return page
def _info(sandbox_id: str, user_id: str, thread_id: str):
def _info(
sandbox_id: str,
user_id: str,
thread_id: str,
*,
skills_container_path: str = "/mnt/skills",
):
return SimpleNamespace(
sandbox_id=sandbox_id,
metadata={
"deer_flow_provider": "e2b_sandbox_provider",
"deer_flow_user": user_id,
"deer_flow_thread": thread_id,
"deer_flow_skills_root": skills_container_path,
},
)
def test_discover_remote_sandbox_rejects_a_different_skills_root(monkeypatch):
p = _make_provider(skills_container_path="/custom-skills")
fake_cls = _install_fake_sdk(monkeypatch, p)
fake_cls.list_return = [_info("sb-old-root", "u1", "t1")]
assert p._discover_remote_sandbox("t1", user_id="u1") is None
assert fake_cls.connect_calls == []
def test_create_metadata_records_the_snapshotted_skills_root(monkeypatch):
provider = _make_provider(skills_container_path="/custom-skills")
fake_cls = _install_fake_sdk(monkeypatch, provider)
provider.acquire("t1", user_id="u1")
assert fake_cls.create_calls[0]["metadata"]["deer_flow_skills_root"] == "/custom-skills"
def test_discover_remote_sandbox_walks_paginator(monkeypatch):
p = _make_provider()
fake_cls = _install_fake_sdk(monkeypatch, p)
@ -1488,7 +1994,7 @@ def test_discover_remote_sandbox_walks_paginator(monkeypatch):
sid = p._discover_remote_sandbox("t1", user_id="u1")
assert sid == "sb-match"
assert p._thread_sandboxes[("u1", "t1")] == "sb-match"
assert p._thread_sandboxes[p._thread_key("t1", "u1")] == "sb-match"
def test_discover_remote_sandbox_accepts_legacy_list(monkeypatch):
@ -1511,7 +2017,7 @@ def test_discover_remote_sandbox_skips_dead_candidate(monkeypatch):
fake_cls.connect_factory = lambda _sid, **_kw: client
assert p._discover_remote_sandbox("t1", user_id="u1") is None
assert ("u1", "t1") not in p._thread_sandboxes
assert p._thread_key("t1", "u1") not in p._thread_sandboxes
assert client.closed is True
@ -1531,7 +2037,7 @@ def test_discover_remote_sandbox_tries_later_candidate_when_first_is_dead(monkey
assert p._discover_remote_sandbox("t1", user_id="u1") == "sb-b-live"
assert dead.closed is True
assert p._thread_sandboxes[("u1", "t1")] == "sb-b-live"
assert p._thread_sandboxes[p._thread_key("t1", "u1")] == "sb-b-live"
assert "sb-b-live" in p._owned_sandbox_ids
@ -1677,7 +2183,7 @@ def test_reconcile_adopts_canonical_after_restart_loses_local_state(monkeypatch)
stats = p._reconcile_remote_sandboxes(now=100.0)
assert stats.adopted == 1
assert p._thread_sandboxes[("u1", "t1")] == "sb-existing"
assert p._thread_sandboxes[p._thread_key("t1", "u1")] == "sb-existing"
assert "sb-existing" in p._owned_sandbox_ids
@ -1748,6 +2254,33 @@ def test_reconcile_kills_metadata_orphan_only_after_ttl(monkeypatch):
assert client.killed is True
def test_reconcile_never_adopts_an_old_skills_root_and_reaps_it_after_grace(
monkeypatch,
):
provider = _make_provider(skills_container_path="/custom-skills")
fake_cls = _install_fake_sdk(monkeypatch, provider)
fake_cls.list_return = [
_info(
"sb-old-root",
"u1",
"t1",
skills_container_path="/mnt/skills",
)
]
client = FakeClient(sandbox_id="sb-old-root")
fake_cls.connect_factory = lambda _sid, **_kw: client
provider._config["reconciliation_grace_seconds"] = 5.0
first = provider._reconcile_remote_sandboxes(now=100.0)
second = provider._reconcile_remote_sandboxes(now=106.0)
assert first.adopted == 0
assert first.deferred == 1
assert provider._thread_key("t1", "u1") not in provider._thread_sandboxes
assert second.killed == 1
assert client.killed is True
def test_discover_remote_sandbox_discards_candidate_when_bootstrap_fails(monkeypatch):
p = _make_provider()
fake_cls = _install_fake_sdk(monkeypatch, p)
@ -1766,7 +2299,7 @@ def test_discover_remote_sandbox_discards_candidate_when_bootstrap_fails(monkeyp
assert p._discover_remote_sandbox("t1", user_id="u1") is None
assert client.killed is True
assert client.closed is True
assert ("u1", "t1") not in p._thread_sandboxes
assert p._thread_key("t1", "u1") not in p._thread_sandboxes
def test_discovery_claims_ownership_before_bootstrap_cleanup(monkeypatch):
@ -1900,7 +2433,14 @@ def test_e2b_config_accepts_documented_reconciliation_fields(monkeypatch, caplog
reconciliation_max_seconds=15,
)
provider = mod.E2BSandboxProvider.__new__(mod.E2BSandboxProvider)
monkeypatch.setattr(mod, "get_app_config", lambda: SimpleNamespace(sandbox=config))
monkeypatch.setattr(
mod,
"get_app_config",
lambda: SimpleNamespace(
sandbox=config,
skills=SimpleNamespace(container_path="/mnt/skills"),
),
)
with caplog.at_level("WARNING"):
provider._load_config()
@ -1916,7 +2456,14 @@ def test_e2b_config_warns_about_unknown_fields(monkeypatch, caplog):
overflo_policy="reject",
)
provider = mod.E2BSandboxProvider.__new__(mod.E2BSandboxProvider)
monkeypatch.setattr(mod, "get_app_config", lambda: SimpleNamespace(sandbox=config))
monkeypatch.setattr(
mod,
"get_app_config",
lambda: SimpleNamespace(
sandbox=config,
skills=SimpleNamespace(container_path="/mnt/skills"),
),
)
with caplog.at_level("WARNING"):
provider._load_config()
@ -2108,13 +2655,14 @@ def test_release_dead_sandbox_skips_warm_pool(monkeypatch):
sb = _make_sandbox(client, sandbox_id="sb-dead")
sb._dead = True
p._sandboxes["sb-dead"] = sb
p._thread_sandboxes[("u1", "t1")] = "sb-dead"
key = p._thread_key("t1", "u1")
p._thread_sandboxes[key] = "sb-dead"
p.release("sb-dead")
assert "sb-dead" not in p._warm_pool, "dead sandbox must not be parked"
assert "sb-dead" not in p._sandboxes
assert ("u1", "t1") not in p._thread_sandboxes
assert key not in p._thread_sandboxes
assert client.killed is True, "release of dead sandbox must kill the remote VM"
@ -2125,7 +2673,7 @@ def test_release_healthy_sandbox_parks_in_warm_pool(monkeypatch, tmp_path):
client = FakeClient(commands=cmds)
sb = _make_sandbox(client, sandbox_id="sb-warm-1")
p._sandboxes["sb-warm-1"] = sb
p._thread_sandboxes[("u1", "t1")] = "sb-warm-1"
p._thread_sandboxes[p._thread_key("t1", "u1")] = "sb-warm-1"
p.release("sb-warm-1")
@ -2142,7 +2690,7 @@ def test_acquire_waits_for_same_thread_release_transition(monkeypatch):
client = FakeClient(sandbox_id="sb-release-race")
sandbox = _make_sandbox(client)
provider._sandboxes[sandbox.id] = sandbox
provider._thread_sandboxes[("user-1", "thread-1")] = sandbox.id
provider._thread_sandboxes[provider._thread_key("thread-1", "user-1")] = sandbox.id
sync_started = threading.Event()
allow_sync_to_finish = threading.Event()
@ -2190,7 +2738,7 @@ def test_release_skips_warm_pool_when_sync_reveals_dead_vm(monkeypatch, tmp_path
client = FakeClient(commands=FakeCommandsAPI([FakeCommandsAPI.GONE]))
sb = _make_sandbox(client, sandbox_id="sb-died-during-sync")
p._sandboxes["sb-died-during-sync"] = sb
p._thread_sandboxes[("u1", "t1")] = "sb-died-during-sync"
p._thread_sandboxes[p._thread_key("t1", "u1")] = "sb-died-during-sync"
p.release("sb-died-during-sync")
@ -2987,10 +3535,19 @@ def test_discovery_uses_sdk_query_and_tracks_without_reserving(monkeypatch) -> N
"deer_flow_provider": "e2b_sandbox_provider",
"deer_flow_user": "user-a",
"deer_flow_thread": "thread-a",
"deer_flow_skills_root": "/mnt/skills",
"deer_flow_capacity_ledger": store.key,
},
)
expected_query = {key: entry.metadata[key] for key in ("deer_flow_provider", "deer_flow_user", "deer_flow_thread")}
expected_query = {
key: entry.metadata[key]
for key in (
"deer_flow_provider",
"deer_flow_user",
"deer_flow_thread",
"deer_flow_skills_root",
)
}
sdk.list_return = SimpleNamespace(
has_next=False,
next_items=lambda: [entry] if sdk.list_calls[-1]["query"].metadata == expected_query else [],
@ -4134,6 +4691,7 @@ def test_discovery_reports_busy_capacity_without_killing_remote_vm(monkeypatch):
"deer_flow_provider": "e2b_sandbox_provider",
"deer_flow_user": "u2",
"deer_flow_thread": "t2",
"deer_flow_skills_root": "/mnt/skills",
},
)
]
@ -4159,6 +4717,7 @@ def test_discovery_reports_shutdown_without_killing_remote_vm(monkeypatch, caplo
"deer_flow_provider": "e2b_sandbox_provider",
"deer_flow_user": "u1",
"deer_flow_thread": "t1",
"deer_flow_skills_root": "/mnt/skills",
},
)
]
@ -4196,6 +4755,7 @@ def test_discovery_bootstrap_kill_failure_retains_reserved_slot(monkeypatch):
"deer_flow_provider": "e2b_sandbox_provider",
"deer_flow_user": "u1",
"deer_flow_thread": "t1",
"deer_flow_skills_root": "/mnt/skills",
},
)
]
@ -4230,6 +4790,7 @@ def test_shutdown_does_not_retry_kill_for_unowned_discovery_vm(monkeypatch):
"deer_flow_provider": "e2b_sandbox_provider",
"deer_flow_user": "u1",
"deer_flow_thread": "t1",
"deer_flow_skills_root": "/mnt/skills",
},
)
]
@ -4274,6 +4835,7 @@ def test_shutdown_during_discovery_does_not_kill_unowned_vm(monkeypatch):
"deer_flow_provider": "e2b_sandbox_provider",
"deer_flow_user": "u1",
"deer_flow_thread": "t1",
"deer_flow_skills_root": "/mnt/skills",
},
)
]
@ -4321,5 +4883,10 @@ def test_shutdown_during_discovery_does_not_kill_unowned_vm(monkeypatch):
def test_stable_seed_matches_shared_identity():
from deerflow.sandbox.identity import derive_sandbox_scope_token
mod = importlib.import_module("deerflow.community.e2b_sandbox.e2b_sandbox_provider")
assert mod.E2BSandboxProvider._stable_seed("t-1", "u-1") == derive_sandbox_scope_token(user_id="u-1", thread_id="t-1")
provider = _make_provider(skills_container_path="/custom-skills")
base_scope = derive_sandbox_scope_token(user_id="u-1", thread_id="t-1")
expected = hashlib.sha256(
f"{base_scope}\0/custom-skills".encode(),
).hexdigest()[:16]
assert provider._stable_seed("t-1", "u-1") == expected

View File

@ -205,6 +205,8 @@ def test_make_lead_agent_scopes_bootstrap_middlewares_to_custom_agent(monkeypatc
assert len(middleware_calls) == 1
assert middleware_calls[0]["agent_name"] == "game"
assert middleware_calls[0]["available_skills"] == {"bootstrap"}
assert middleware_calls[0]["owns_agent_skill_projection"] is False
assert len(prompt_calls) == 1

View File

@ -565,6 +565,21 @@ class TestMultipleMounts:
class TestLocalSandboxProviderMounts:
def test_skill_isolation_capability_fails_closed_when_host_bash_is_enabled(self):
provider = LocalSandboxProvider.__new__(LocalSandboxProvider)
with patch(
"deerflow.sandbox.local.local_sandbox_provider.is_host_bash_allowed",
return_value=False,
):
assert provider.supports_agent_skill_isolation is True
with patch(
"deerflow.sandbox.local.local_sandbox_provider.is_host_bash_allowed",
return_value=True,
):
assert provider.supports_agent_skill_isolation is False
def test_thread_mappings_mount_per_user_integration_projections(self, tmp_path):
from deerflow.config.paths import Paths

View File

@ -1,4 +1,4 @@
"""Keep Gateway/provisioner extra-mount allowlists in lockstep."""
"""Keep Gateway/provisioner fixed and dynamic mount contracts in lockstep."""
from __future__ import annotations
@ -22,8 +22,15 @@ def test_gateway_and_provisioner_extra_mount_contracts_match() -> None:
gateway_paths = _literal_assignment(gateway_path, "_PROVISIONER_EXTRA_MOUNT_PATHS")
provisioner_paths = _literal_assignment(provisioner_path, "ALLOWED_EXTRA_MOUNT_PATHS")
gateway_categories = _literal_assignment(gateway_path, "_MANAGED_SKILL_CATEGORY_NAMES")
provisioner_categories = _literal_assignment(provisioner_path, "MANAGED_SKILL_CATEGORY_NAMES")
gateway_reserved = _literal_assignment(gateway_path, "_RESERVED_SANDBOX_MOUNT_PATHS")
provisioner_reserved = _literal_assignment(provisioner_path, "RESERVED_SANDBOX_MOUNT_PATHS")
assert gateway_paths == provisioner_paths
assert gateway_categories == provisioner_categories
assert gateway_reserved == provisioner_reserved
assert _literal_assignment(provisioner_path, "DEFAULT_SKILLS_CONTAINER_PATH") == "/mnt/skills"
assert "/mnt/integrations/lark-cli/runtime" in gateway_paths
assert "/mnt/integrations/lark-cli/config/locks" in gateway_paths
assert _literal_assignment(provisioner_path, "MAX_EXTRA_MOUNTS") == 10

View File

@ -2,6 +2,21 @@
import pytest
def _thread_skill_mounts(
provisioner_module,
skills_container_path="/mnt/skills",
):
return [
provisioner_module.ExtraMount(
host_path=f"/state/users/alice/threads/thread-1/skills_view/{category}",
container_path=f"{skills_container_path}/{category}",
read_only=True,
)
for category in ("public", "custom", "legacy", "integrations")
]
# ── _build_volumes ─────────────────────────────────────────────────────
@ -176,6 +191,72 @@ class TestBuildVolumes:
assert exc_info.value.status_code == 400
def test_thread_skill_mounts_replace_hostpath_skill_volumes(
self,
provisioner_module,
):
provisioner_module.SKILLS_PVC_NAME = ""
provisioner_module.USERDATA_PVC_NAME = ""
provisioner_module.DEER_FLOW_HOST_BASE_DIR = "/state"
volumes = provisioner_module._build_volumes(
"thread-1",
user_id="alice",
extra_mounts=_thread_skill_mounts(provisioner_module),
)
names = {volume.name for volume in volumes}
assert not {"skills-public", "skills-custom", "skills-legacy"} & names
assert names == {"user-data", "extra-0", "extra-1", "extra-2", "extra-3"}
def test_custom_root_thread_mounts_replace_hostpath_skill_volumes(
self,
provisioner_module,
):
provisioner_module.SKILLS_PVC_NAME = ""
provisioner_module.USERDATA_PVC_NAME = ""
provisioner_module.DEER_FLOW_HOST_BASE_DIR = "/state"
skills_root = "/custom-skills"
volumes = provisioner_module._build_volumes(
"thread-1",
user_id="alice",
extra_mounts=_thread_skill_mounts(
provisioner_module,
skills_root,
),
skills_container_path=skills_root,
)
names = {volume.name for volume in volumes}
assert not {"skills-public", "skills-custom", "skills-legacy"} & names
assert names == {
"user-data",
"extra-0",
"extra-1",
"extra-2",
"extra-3",
}
def test_thread_skill_mounts_replace_skills_pvc_with_userdata_pvc_categories(
self,
provisioner_module,
):
provisioner_module.SKILLS_PVC_NAME = "skills-pvc"
provisioner_module.USERDATA_PVC_NAME = "userdata-pvc"
provisioner_module.DEER_FLOW_HOST_BASE_DIR = "/state"
volumes = provisioner_module._build_volumes(
"thread-1",
user_id="alice",
extra_mounts=_thread_skill_mounts(provisioner_module),
)
assert all(volume.name != "skills" for volume in volumes)
extra_volumes = [volume for volume in volumes if volume.name.startswith("extra-")]
assert len(extra_volumes) == 4
assert all(volume.persistent_volume_claim.claim_name == "userdata-pvc" for volume in extra_volumes)
# ── _build_volume_mounts ───────────────────────────────────────────────
@ -348,6 +429,119 @@ class TestBuildVolumeMounts:
assert exc_info.value.status_code == 400
def test_thread_skill_category_mounts_are_unique_in_hostpath_mode(
self,
provisioner_module,
):
provisioner_module.SKILLS_PVC_NAME = ""
provisioner_module.USERDATA_PVC_NAME = ""
provisioner_module.DEER_FLOW_HOST_BASE_DIR = "/state"
mounts = provisioner_module._build_volume_mounts(
"thread-1",
user_id="alice",
extra_mounts=_thread_skill_mounts(provisioner_module),
)
mount_paths = [mount.mount_path for mount in mounts]
assert len(mount_paths) == len(set(mount_paths))
assert set(mount_paths) == {
"/mnt/user-data",
"/mnt/skills/public",
"/mnt/skills/custom",
"/mnt/skills/legacy",
"/mnt/skills/integrations",
}
def test_thread_skill_category_mounts_use_userdata_pvc_subpaths(
self,
provisioner_module,
):
provisioner_module.SKILLS_PVC_NAME = "skills-pvc"
provisioner_module.USERDATA_PVC_NAME = "userdata-pvc"
provisioner_module.DEER_FLOW_HOST_BASE_DIR = "/state"
mounts = provisioner_module._build_volume_mounts(
"thread-1",
user_id="alice",
extra_mounts=_thread_skill_mounts(provisioner_module),
)
skill_mounts = [mount for mount in mounts if mount.mount_path.startswith("/mnt/skills/")]
assert len(skill_mounts) == 4
assert all(mount.name != "skills" for mount in mounts)
assert {mount.sub_path for mount in skill_mounts} == {f"deer-flow/users/alice/threads/thread-1/skills_view/{category}" for category in ("public", "custom", "legacy", "integrations")}
@pytest.mark.parametrize("use_userdata_pvc", [False, True])
def test_custom_root_thread_skill_mounts_replace_every_default_path(
self,
provisioner_module,
use_userdata_pvc,
):
provisioner_module.SKILLS_PVC_NAME = "skills-pvc" if use_userdata_pvc else ""
provisioner_module.USERDATA_PVC_NAME = "userdata-pvc" if use_userdata_pvc else ""
provisioner_module.DEER_FLOW_HOST_BASE_DIR = "/state"
skills_root = "/custom-skills"
mounts = provisioner_module._build_volume_mounts(
"thread-1",
user_id="alice",
extra_mounts=_thread_skill_mounts(
provisioner_module,
skills_root,
),
skills_container_path=skills_root,
)
mount_paths = {mount.mount_path for mount in mounts}
assert not any(path.startswith("/mnt/skills") for path in mount_paths)
assert {f"{skills_root}/{category}" for category in ("public", "custom", "legacy", "integrations")} <= mount_paths
def test_custom_root_is_used_for_unrestricted_skills_mounts(
self,
provisioner_module,
):
provisioner_module.SKILLS_PVC_NAME = ""
provisioner_module.USERDATA_PVC_NAME = ""
mounts = provisioner_module._build_volume_mounts(
"thread-1",
skills_container_path="/custom-skills",
)
assert {mount.mount_path for mount in mounts if mount.name.startswith("skills-")} == {
"/custom-skills/public",
"/custom-skills/custom",
"/custom-skills/legacy",
}
@pytest.mark.parametrize(
"skills_root",
[
"/",
"relative-skills",
"//custom-skills",
"/custom//skills",
"/custom/../skills",
"/mnt",
"/mnt/user-data/skills",
"/mnt/acp-workspace/skills",
"/mnt/integrations/lark-cli/skills",
],
)
def test_rejects_unsafe_skills_container_roots(
self,
provisioner_module,
skills_root,
):
with pytest.raises(provisioner_module.HTTPException) as exc_info:
provisioner_module._build_volume_mounts(
"thread-1",
skills_container_path=skills_root,
)
assert exc_info.value.status_code == 400
# ── _build_pod integration ─────────────────────────────────────────────
@ -669,6 +863,41 @@ class TestLarkCliBrokerSidecar:
env = {e.name: e.value for e in (sandbox.env or [])}
assert env.get("DEERFLOW_LARK_BROKER_URL") == provisioner_module.LARK_BROKER_URL
def test_custom_skills_root_is_compatible_with_broker_credentials(
self,
provisioner_module,
):
provisioner_module.SKILLS_PVC_NAME = ""
provisioner_module.USERDATA_PVC_NAME = ""
provisioner_module.DEER_FLOW_HOST_BASE_DIR = "/state"
provisioner_module.LARK_CLI_BROKER_IMAGE = "deer-flow/lark-cli-broker:v1.0.65"
skills_root = "/custom-skills"
pod = provisioner_module._build_pod(
"sandbox-1",
"thread-1",
user_id="alice",
extra_mounts=[
*_thread_skill_mounts(
provisioner_module,
skills_root,
),
*self._credential_mounts(provisioner_module),
],
skills_container_path=skills_root,
provision_lark_cli_broker=True,
)
sandbox_mount_paths = {mount.mount_path for mount in pod.spec.containers[0].volume_mounts}
assert not any(path.startswith("/mnt/skills") for path in sandbox_mount_paths)
assert {f"{skills_root}/{category}" for category in ("public", "custom", "legacy", "integrations")} <= sandbox_mount_paths
sidecar = next(container for container in pod.spec.containers if container.name == "lark-cli-broker")
assert {mount.mount_path for mount in sidecar.volume_mounts} == {
provisioner_module.LARK_BROKER_SIDECAR_CONFIG_PATH,
provisioner_module.LARK_BROKER_SIDECAR_LOCKS_PATH,
provisioner_module.LARK_BROKER_SIDECAR_DATA_PATH,
}
def test_broker_supersedes_init_container(self, provisioner_module):
"""Both images set + both flags on → broker wins (shim init, sidecar)."""
provisioner_module.SKILLS_PVC_NAME = ""

View File

@ -42,6 +42,14 @@ def test_provisioner_accepts_canonical_thread_ids(provisioner_module, thread_id:
assert request.thread_id == thread_id
def test_provisioner_request_defaults_skills_container_path(provisioner_module) -> None:
request = provisioner_module.CreateSandboxRequest(
sandbox_id="sandbox-validation",
)
assert request.skills_container_path == "/mnt/skills"
class _RecordingCoreV1:
def __init__(
self,
@ -232,6 +240,41 @@ def test_create_sandbox_route_builds_expected_skills_mount_layout(
assert mount_names == expected_mount_names
def test_create_sandbox_route_threads_custom_skills_root_into_pod(
monkeypatch: pytest.MonkeyPatch,
provisioner_module,
) -> None:
fake_core_v1 = _RecordingCoreV1(
event_loop_thread_id=-1,
ready_after_service_reads={"sandbox-custom-skills": 1},
)
monkeypatch.setattr(provisioner_module, "core_v1", fake_core_v1)
categories = ("public", "custom", "legacy", "integrations")
response = provisioner_module.create_sandbox(
provisioner_module.CreateSandboxRequest(
sandbox_id="sandbox-custom-skills",
thread_id="thread-1",
user_id="alice",
skills_container_path="/custom-skills",
extra_mounts=[
provisioner_module.ExtraMount(
host_path=(f"/.deer-flow/users/alice/threads/thread-1/skills_view/{category}"),
container_path=f"/custom-skills/{category}",
read_only=True,
)
for category in categories
],
)
)
assert response.status == "Running"
pod = fake_core_v1.created_pod_specs["sandbox-custom-skills"]
mount_paths = {mount.mount_path for mount in pod.spec.containers[0].volume_mounts}
assert not any(path.startswith("/mnt/skills") for path in mount_paths)
assert {f"/custom-skills/{category}" for category in categories} <= mount_paths
def test_create_sandbox_retries_transient_service_read_errors(monkeypatch: pytest.MonkeyPatch, provisioner_module) -> None:
fake_core_v1 = _RecordingCoreV1(
event_loop_thread_id=-1,

View File

@ -20,6 +20,7 @@ from deerflow.config.reload_boundary import (
is_startup_only_field,
iter_startup_only_field_paths,
)
from deerflow.config.skills_config import SkillsConfig
def test_registry_has_a_reason_for_every_field():
@ -115,6 +116,15 @@ def test_appconfig_schema_marks_registered_fields_with_prefix():
assert description.startswith(STARTUP_ONLY_PREFIX), f"AppConfig.{field_path} should have Field(description=) starting with {STARTUP_ONLY_PREFIX!r}, got {description!r}"
def test_skills_container_path_is_registered_as_startup_only():
"""AIO and E2B snapshot the mount root when their provider starts."""
field_path = "skills.container_path"
assert field_path in STARTUP_ONLY_FIELDS
description = SkillsConfig.model_fields["container_path"].description or ""
assert description.startswith(STARTUP_ONLY_PREFIX)
assert STARTUP_ONLY_FIELDS[field_path] in description
def test_no_appconfig_field_uses_prefix_without_registration():
"""Reverse drift check: if a future schema edit adds the
``startup-only:`` prefix to a new field, the registry must list it.

View File

@ -34,6 +34,31 @@ class _StubResponse:
return self._payload
@pytest.mark.parametrize(
"container_path",
[
"/",
"relative-skills",
"//custom-skills",
"/custom//skills",
"/custom/../skills",
"/mnt",
"/mnt/user-data/skills",
"/mnt/acp-workspace/skills",
"/mnt/integrations/lark-cli/skills",
],
)
def test_skills_container_path_rejects_unsafe_or_noncanonical_roots(
container_path,
):
with pytest.raises(ValueError):
remote_backend_mod._normalize_skills_container_path(container_path)
def test_skills_container_path_accepts_isolated_custom_root():
assert remote_backend_mod._normalize_skills_container_path("/custom-skills") == "/custom-skills"
def test_list_running_delegates_to_provisioner_list(monkeypatch):
backend = RemoteSandboxBackend("http://provisioner:8002")
sandbox_info = SandboxInfo(sandbox_id="test-id", sandbox_url="http://localhost:8080")
@ -165,11 +190,21 @@ def test_create_delegates_to_provisioner_create(monkeypatch, expected_user_id):
backend = RemoteSandboxBackend("http://provisioner:8002")
expected = SandboxInfo(sandbox_id="abc123", sandbox_url="http://k3s:31001")
def mock_create(thread_id: str, sandbox_id: str, extra_mounts=None, *, user_id=None, provision_lark_cli_runtime=False, provision_lark_cli_broker=False):
def mock_create(
thread_id: str,
sandbox_id: str,
extra_mounts=None,
*,
user_id=None,
skills_container_path="/mnt/skills",
provision_lark_cli_runtime=False,
provision_lark_cli_broker=False,
):
assert thread_id == "thread-1"
assert sandbox_id == "abc123"
assert extra_mounts == [("/host", "/container", False)]
assert user_id == expected_user_id
assert skills_container_path == "/mnt/skills"
assert provision_lark_cli_runtime is True
assert provision_lark_cli_broker is False
return expected
@ -197,6 +232,7 @@ def test_provisioner_create_returns_sandbox_info(monkeypatch):
"thread_id": "thread-1",
"user_id": "test-user-autouse",
"include_legacy_skills": True,
"skills_container_path": "/mnt/skills",
"provision_lark_cli_runtime": False,
"provision_lark_cli_broker": False,
}
@ -247,6 +283,48 @@ def test_provisioner_create_forwards_supported_extra_mounts(monkeypatch):
)
def test_provisioner_create_forwards_custom_skills_root_and_all_policy_mounts(
monkeypatch,
):
backend = RemoteSandboxBackend("http://provisioner:8002")
monkeypatch.setattr(
remote_backend_mod,
"user_should_see_legacy_skills",
lambda user_id: False,
)
captured: dict = {}
def mock_post(url: str, json: dict, timeout: int, headers=None):
captured.update(json)
return _StubResponse(
payload={
"sandbox_id": "abc123",
"sandbox_url": "http://k3s:31001",
}
)
monkeypatch.setattr(requests, "post", mock_post)
categories = ("public", "custom", "legacy", "integrations")
backend._provisioner_create(
"thread-1",
"abc123",
extra_mounts=[
(
f"/state/users/alice/threads/thread-1/skills_view/{category}",
f"/custom-skills/{category}",
True,
)
for category in categories
],
user_id="alice",
skills_container_path="/custom-skills",
)
assert captured["skills_container_path"] == "/custom-skills"
assert {mount["container_path"] for mount in captured["extra_mounts"]} == {f"/custom-skills/{category}" for category in categories}
def test_provisioner_create_strips_runtime_mount_when_init_container_enabled(monkeypatch):
backend = RemoteSandboxBackend("http://provisioner:8002")
monkeypatch.setattr(remote_backend_mod, "user_should_see_legacy_skills", lambda user_id: False)
@ -318,6 +396,7 @@ def test_provisioner_create_accepts_anonymous_thread_id(monkeypatch):
"thread_id": None,
"user_id": "test-user-autouse",
"include_legacy_skills": False,
"skills_container_path": "/mnt/skills",
"provision_lark_cli_runtime": False,
"provision_lark_cli_broker": False,
}

View File

@ -12,6 +12,7 @@ from langgraph.runtime import Runtime
from langgraph.types import Command, Overwrite
from deerflow.agents.thread_state import ThreadState
from deerflow.sandbox.exceptions import SandboxAuthorizationError, SandboxRuntimeError
from deerflow.sandbox.middleware import SandboxMiddleware, SandboxMiddlewareState
from deerflow.sandbox.sandbox import Sandbox
from deerflow.sandbox.sandbox_provider import SandboxProvider, reset_sandbox_provider, set_sandbox_provider
@ -36,6 +37,24 @@ class _SyncProvider(SandboxProvider):
return None
class _AgentSkillSyncProvider(_SyncProvider):
supports_agent_skill_isolation = True
def __init__(self) -> None:
super().__init__()
self.skill_syncs: list[tuple[str, str, str, object]] = []
def sync_agent_skills(
self,
sandbox_id: str,
*,
thread_id: str,
user_id: str,
projection,
) -> None:
self.skill_syncs.append((sandbox_id, thread_id, user_id, projection))
class _SandboxStub(Sandbox):
def execute_command(
self,
@ -151,6 +170,129 @@ async def test_abefore_agent_uses_async_provider_acquire() -> None:
assert provider.user_ids == ["owner-2"]
def test_explicit_skill_policy_eagerly_acquires_and_syncs_existing_thread(
monkeypatch: pytest.MonkeyPatch,
) -> None:
provider = _AgentSkillSyncProvider()
projection = object()
middleware = SandboxMiddleware(lazy_init=True, available_skills=set())
monkeypatch.setattr(
middleware,
"_prepare_agent_skill_projection",
lambda *_args, **_kwargs: projection,
)
set_sandbox_provider(provider)
try:
result = middleware.before_agent(
{"sandbox": {"sandbox_id": "shared-view-sandbox"}},
Runtime(context={"thread_id": "thread-policy", "user_id": "owner-policy"}),
)
finally:
reset_sandbox_provider()
assert result is not None
assert isinstance(result["sandbox"], Overwrite)
assert result["sandbox"].value == {"sandbox_id": "sync-sandbox"}
assert provider.thread_ids == ["thread-policy"]
assert provider.user_ids == ["owner-policy"]
assert provider.skill_syncs == [("sync-sandbox", "thread-policy", "owner-policy", projection)]
def test_explicit_skill_policy_fails_closed_for_unsupported_provider(
monkeypatch: pytest.MonkeyPatch,
) -> None:
provider = _SyncProvider()
middleware = SandboxMiddleware(lazy_init=True, available_skills={"allowed"})
monkeypatch.setattr(
middleware,
"_prepare_agent_skill_projection",
lambda *_args, **_kwargs: object(),
)
set_sandbox_provider(provider)
try:
with pytest.raises(
SandboxRuntimeError,
match="cannot enforce per-Agent skill filesystem isolation",
):
middleware.before_agent(
{},
Runtime(context={"thread_id": "thread-policy", "user_id": "owner-policy"}),
)
finally:
reset_sandbox_provider()
assert provider.thread_ids == []
def test_non_owner_skill_policy_preserves_lazy_init_without_projection_or_acquire(
monkeypatch: pytest.MonkeyPatch,
) -> None:
provider = _SyncProvider()
middleware = SandboxMiddleware(
lazy_init=True,
available_skills={"bootstrap"},
owns_agent_skill_projection=False,
)
prepare_calls: list[tuple[str, str]] = []
original_prepare = middleware._prepare_agent_skill_projection
def _prepare(thread_id: str, *, user_id: str):
prepare_calls.append((thread_id, user_id))
return original_prepare(thread_id, user_id=user_id)
monkeypatch.setattr(middleware, "_prepare_agent_skill_projection", _prepare)
set_sandbox_provider(provider)
try:
result = middleware.before_agent(
{},
Runtime(
context={
"thread_id": "thread-bootstrap",
"user_id": "owner-bootstrap",
}
),
)
finally:
reset_sandbox_provider()
assert result is None
assert prepare_calls == [("thread-bootstrap", "owner-bootstrap")]
assert provider.thread_ids == []
def test_explicit_skill_policy_does_not_reuse_checkpointed_sandbox_after_auth_denial(
monkeypatch: pytest.MonkeyPatch,
) -> None:
provider = _AgentSkillSyncProvider()
middleware = SandboxMiddleware(lazy_init=True, available_skills=set())
monkeypatch.setattr(
middleware,
"_prepare_agent_skill_projection",
lambda *_args, **_kwargs: object(),
)
monkeypatch.setattr(
"deerflow.sandbox.middleware.authorize_sandbox_execution",
lambda **_kwargs: (_ for _ in ()).throw(SandboxAuthorizationError("denied")),
)
set_sandbox_provider(provider)
try:
with pytest.raises(SandboxAuthorizationError, match="denied"):
middleware.before_agent(
{"sandbox": {"sandbox_id": "shared-view-sandbox"}},
Runtime(
context={
"thread_id": "thread-policy",
"user_id": "owner-policy",
}
),
)
finally:
reset_sandbox_provider()
assert provider.thread_ids == []
assert provider.skill_syncs == []
@pytest.mark.anyio
@pytest.mark.parametrize(
("middleware", "state", "runtime"),

View File

@ -14,7 +14,14 @@ import pytest
from deerflow.config.extensions_config import ExtensionsConfig, SkillStateConfig
from deerflow.config.paths import Paths
from deerflow.skills.projection import ensure_public_skill_projection, ensure_skill_projections, rebuild_skill_projections, skill_projection_mutation
from deerflow.sandbox.middleware import SandboxMiddleware
from deerflow.skills.projection import (
ensure_public_skill_projection,
ensure_skill_projections,
ensure_thread_skill_projection,
rebuild_skill_projections,
skill_projection_mutation,
)
from deerflow.skills.storage.user_scoped_skill_storage import UserScopedSkillStorage
@ -227,6 +234,242 @@ def test_managed_integration_projection_is_filtered_per_user(projection_env) ->
assert (bob_projection.integrations / "lark-cli" / "lark-doc" / "SKILL.md").is_file()
def test_thread_projection_enforces_agent_allowlist_across_skill_categories(projection_env) -> None:
env = projection_env
_write_skill(env.skills_root / "public", "public-allowed")
_write_skill(env.skills_root / "public", "public-denied")
env.storage.write_custom_skill("custom-allowed", "SKILL.md", _skill_content("custom-allowed"))
env.storage.write_custom_skill("custom-denied", "SKILL.md", _skill_content("custom-denied"))
integration_root = env.paths.integration_skills_dir() / "provider"
_write_skill(integration_root, "integration-allowed")
_write_skill(integration_root, "integration-denied")
projected = ensure_thread_skill_projection(
env.storage,
"thread-1",
{"public-allowed", "custom-allowed", "integration-allowed"},
)
assert projected is not None
assert (projected.public / "public-allowed" / "SKILL.md").is_file()
assert not (projected.public / "public-denied").exists()
assert (projected.custom / "custom-allowed" / "SKILL.md").is_file()
assert not (projected.custom / "custom-denied").exists()
assert (projected.integrations / "provider" / "integration-allowed" / "SKILL.md").is_file()
assert not (projected.integrations / "provider" / "integration-denied").exists()
def test_thread_projection_rejects_symlink_escape_to_omitted_skill(
projection_env,
) -> None:
env = projection_env
allowed_file = _write_skill(env.skills_root / "public", "public-allowed")
_write_skill(env.skills_root / "public", "public-denied", "DENIED_MARKER")
leak = allowed_file.parent / "leak.md"
try:
leak.symlink_to(Path("..") / "public-denied" / "SKILL.md")
except OSError as exc:
if getattr(exc, "winerror", None) == 1314:
pytest.skip("Windows symlink creation requires SeCreateSymbolicLinkPrivilege")
raise
with pytest.raises(ValueError, match="symlink escapes its package"):
ensure_thread_skill_projection(
env.storage,
"thread-symlink-escape",
{"public-allowed"},
)
root = env.paths.thread_skills_view_dir(
"thread-symlink-escape",
user_id="alice",
)
assert all(list((root / category).iterdir()) == [] for category in ("public", "custom", "legacy", "integrations"))
assert not (root / ".projection-manifest.json").exists()
def test_thread_projection_exposes_only_effective_same_name_winner(
projection_env,
) -> None:
env = projection_env
_write_skill(env.skills_root / "public", "shadowed-skill", "public copy")
env.storage.write_custom_skill(
"shadowed-skill",
"SKILL.md",
_skill_content("shadowed-skill", "custom copy"),
)
projected = ensure_thread_skill_projection(
env.storage,
"thread-shadowing",
{"shadowed-skill"},
)
assert projected is not None
assert not (projected.public / "shadowed-skill").exists()
custom_file = projected.custom / "shadowed-skill" / "SKILL.md"
assert custom_file.is_file()
assert "custom copy" in custom_file.read_text(encoding="utf-8")
def test_thread_projection_enforces_allowlist_for_legacy_skills(
projection_env,
) -> None:
env = projection_env
_write_skill(env.skills_root / "custom", "legacy-allowed")
_write_skill(env.skills_root / "custom", "legacy-denied")
projected = ensure_thread_skill_projection(
env.storage,
"thread-legacy",
{"legacy-allowed"},
)
assert projected is not None
assert (projected.legacy / "legacy-allowed" / "SKILL.md").is_file()
assert not (projected.legacy / "legacy-denied").exists()
def test_thread_projection_intersects_agent_allowlist_with_deployment_state(
projection_env,
) -> None:
env = projection_env
_write_skill(env.skills_root / "public", "deployment-disabled")
env.extensions.skills["deployment-disabled"] = SkillStateConfig(enabled=False)
projected = ensure_thread_skill_projection(
env.storage,
"thread-disabled",
{"deployment-disabled"},
)
assert projected is not None
assert not (projected.public / "deployment-disabled").exists()
def test_empty_agent_allowlist_projects_no_business_skills(projection_env) -> None:
env = projection_env
_write_skill(env.skills_root / "public", "public-skill")
env.storage.write_custom_skill("custom-skill", "SKILL.md", _skill_content("custom-skill"))
projected = ensure_thread_skill_projection(env.storage, "thread-empty", set())
assert projected is not None
for category in (projected.public, projected.custom, projected.legacy, projected.integrations):
assert list(category.iterdir()) == []
def test_unrestricted_thread_without_policy_keeps_shared_projection(
projection_env,
) -> None:
env = projection_env
projected = ensure_thread_skill_projection(
env.storage,
"thread-unrestricted",
None,
)
assert projected is None
assert not env.paths.thread_skills_view_dir(
"thread-unrestricted",
user_id="alice",
).exists()
def test_unrestricted_run_reuses_existing_thread_mount_with_full_enabled_view(projection_env) -> None:
env = projection_env
_write_skill(env.skills_root / "public", "alpha")
_write_skill(env.skills_root / "public", "beta")
projected = ensure_thread_skill_projection(env.storage, "thread-switch", {"alpha"})
assert projected is not None
category_inodes = {category: getattr(projected, category).stat().st_ino for category in ("public", "custom", "legacy", "integrations")}
assert (projected.public / "alpha" / "SKILL.md").is_file()
assert not (projected.public / "beta").exists()
unrestricted = ensure_thread_skill_projection(env.storage, "thread-switch", None)
assert unrestricted == projected
assert (projected.public / "alpha" / "SKILL.md").is_file()
assert (projected.public / "beta" / "SKILL.md").is_file()
assert {category: getattr(projected, category).stat().st_ino for category in ("public", "custom", "legacy", "integrations")} == category_inodes
def test_subagent_non_owner_preserves_restricted_lead_projection(
projection_env,
monkeypatch,
) -> None:
env = projection_env
_write_skill(env.skills_root / "public", "allowed")
_write_skill(env.skills_root / "public", "omitted")
provider = SimpleNamespace(supports_agent_skill_isolation=True)
monkeypatch.setattr(
"deerflow.sandbox.middleware.get_sandbox_provider",
lambda: provider,
)
monkeypatch.setattr("deerflow.config.get_app_config", lambda: env.config)
monkeypatch.setattr(
"deerflow.skills.storage.get_or_new_user_skill_storage",
lambda *_args, **_kwargs: env.storage,
)
lead = SandboxMiddleware(available_skills={"allowed"})
projected = lead._prepare_agent_skill_projection(
"thread-delegation",
user_id="alice",
)
assert projected is not None
assert (projected.public / "allowed" / "SKILL.md").is_file()
assert not (projected.public / "omitted").exists()
subagent = SandboxMiddleware(
available_skills=None,
owns_agent_skill_projection=False,
)
assert (
subagent._prepare_agent_skill_projection(
"thread-delegation",
user_id="alice",
)
is None
)
assert (projected.public / "allowed" / "SKILL.md").is_file()
assert not (projected.public / "omitted").exists()
def test_thread_projection_revokes_removed_skill_before_repopulation(projection_env, monkeypatch) -> None:
env = projection_env
_write_skill(env.skills_root / "public", "alpha")
_write_skill(env.skills_root / "public", "beta")
projected = ensure_thread_skill_projection(env.storage, "thread-update", {"alpha", "beta"})
assert projected is not None
from deerflow.skills import projection as projection_module
real_stage_skill = projection_module._stage_skill
staging_started = Event()
release_staging = Event()
def _delayed_stage_skill(*args, **kwargs):
if not staging_started.is_set():
staging_started.set()
assert release_staging.wait(timeout=5)
return real_stage_skill(*args, **kwargs)
monkeypatch.setattr(projection_module, "_stage_skill", _delayed_stage_skill)
with ThreadPoolExecutor(max_workers=1) as executor:
rebuild = executor.submit(ensure_thread_skill_projection, env.storage, "thread-update", {"alpha"})
assert staging_started.wait(timeout=5)
beta_was_revoked = not (projected.public / "beta").exists()
release_staging.set()
rebuild.result(timeout=5)
assert beta_was_revoked
assert (projected.public / "alpha" / "SKILL.md").is_file()
assert not (projected.public / "beta").exists()
def test_disabling_custom_skill_hides_only_target_while_rebuilding(projection_env, monkeypatch) -> None:
env = projection_env
env.storage.write_custom_skill("alpha", "SKILL.md", _skill_content("alpha"))
@ -505,7 +748,10 @@ def test_rebuild_failure_clears_old_projection(projection_env, monkeypatch) -> N
replacement = source.with_suffix(".replacement")
replacement.write_text(_skill_content("demo-skill", "after"), encoding="utf-8")
replacement.replace(source)
monkeypatch.setattr("deerflow.skills.projection._stage_skill", lambda *_args: (_ for _ in ()).throw(OSError("disk full")))
monkeypatch.setattr(
"deerflow.skills.projection._stage_skill",
lambda *_args, **_kwargs: (_ for _ in ()).throw(OSError("disk full")),
)
with pytest.raises(OSError, match="disk full"):
ensure_skill_projections(env.storage)

View File

@ -21,7 +21,10 @@ from deerflow.config.paths import Paths
from deerflow.sandbox.local.local_sandbox import PathMapping
from deerflow.sandbox.local.local_sandbox_provider import LocalSandboxProvider
from deerflow.sandbox.tools import read_file_tool
from deerflow.skills.projection import rebuild_skill_projections
from deerflow.skills.projection import (
ensure_thread_skill_projection,
rebuild_skill_projections,
)
from deerflow.skills.storage import reset_user_skill_storage
from deerflow.skills.storage.user_scoped_skill_storage import UserScopedSkillStorage
from deerflow.skills.types import SKILL_MD_FILE, Skill, SkillCategory
@ -375,6 +378,146 @@ class TestThreeWayMountEndToEnd:
assert "SECRET_PROCEDURE" not in structured_disabled_again
assert "disabled" in structured_disabled_again.lower()
def test_local_agent_allowlist_is_enforced_by_every_filesystem_path(
self,
tmp_path,
):
skills_root = tmp_path / "skills"
_write_skill(skills_root / "public", "allowed-skill", "ALLOWED_MARKER")
_write_skill(skills_root / "public", "excluded-skill", "EXCLUDED_MARKER")
(skills_root / "custom").mkdir(parents=True)
paths = Paths(base_dir=tmp_path)
cfg = _build_config(skills_root)
extensions = ExtensionsConfig()
with (
patch("deerflow.config.get_app_config", return_value=cfg),
patch("deerflow.config.paths.get_paths", return_value=paths),
patch(
"deerflow.config.extensions_config.ExtensionsConfig.from_file",
return_value=extensions,
),
patch(
"deerflow.config.extensions_config.get_extensions_config",
return_value=extensions,
),
):
storage = UserScopedSkillStorage(
"user-1",
host_path=str(skills_root),
app_config=cfg,
)
projection = ensure_thread_skill_projection(
storage,
"thread-policy",
{"allowed-skill"},
)
assert projection is not None
provider = LocalSandboxProvider()
sandbox_id = provider.acquire("thread-policy", user_id="user-1")
sandbox = provider.get(sandbox_id)
assert sandbox is not None
mappings = {mapping.container_path: mapping for mapping in sandbox.path_mappings}
assert set(path for path in mappings if path.startswith("/mnt/skills")) == {"/mnt/skills"}
assert Path(mappings["/mnt/skills"].local_path) == projection.public.parent
listing = "\n".join(sandbox.list_dir("/mnt/skills", max_depth=4))
assert "allowed-skill" in listing
assert "excluded-skill" not in listing
assert "EXCLUDED_MARKER" not in sandbox.execute_command("find /mnt/skills -name SKILL.md -print -exec cat {} \\;")
excluded_path = "/mnt/skills/public/excluded-skill/SKILL.md"
with pytest.raises(FileNotFoundError):
sandbox.read_file(excluded_path)
globbed, _ = sandbox.glob("/mnt/skills", "**/SKILL.md")
assert any("allowed-skill" in path for path in globbed)
assert all("excluded-skill" not in path for path in globbed)
grepped, _ = sandbox.grep(
"/mnt/skills",
"EXCLUDED_MARKER",
literal=True,
)
assert grepped == []
absolute_read = sandbox.execute_command(f"cat {excluded_path}")
relative_read = sandbox.execute_command("cd /mnt/skills/public && cat excluded-skill/SKILL.md")
python_read = sandbox.execute_command("python3 -c \"from pathlib import Path; print(Path('/mnt/skills/public/excluded-skill/SKILL.md').read_text())\"")
symlink_read = sandbox.execute_command("ln -s /mnt/skills/public/excluded-skill /mnt/skills/public/excluded-link && cat /mnt/skills/public/excluded-link/SKILL.md")
for result in (
absolute_read,
relative_read,
python_read,
symlink_read,
):
assert "EXCLUDED_MARKER" not in result
assert "Exit Code:" in result
def test_local_empty_agent_allowlist_exposes_no_business_skill(
self,
tmp_path,
):
skills_root = tmp_path / "skills"
_write_skill(skills_root / "public", "public-skill", "PUBLIC_MARKER")
(skills_root / "custom").mkdir(parents=True)
paths = Paths(base_dir=tmp_path)
cfg = _build_config(skills_root)
extensions = ExtensionsConfig()
with (
patch("deerflow.config.get_app_config", return_value=cfg),
patch("deerflow.config.paths.get_paths", return_value=paths),
patch(
"deerflow.config.extensions_config.ExtensionsConfig.from_file",
return_value=extensions,
),
patch(
"deerflow.config.extensions_config.get_extensions_config",
return_value=extensions,
),
):
storage = UserScopedSkillStorage(
"user-1",
host_path=str(skills_root),
app_config=cfg,
)
storage.write_custom_skill(
"custom-skill",
"SKILL.md",
"---\nname: custom-skill\ndescription: CUSTOM_MARKER\n---\n",
)
projection = ensure_thread_skill_projection(
storage,
"thread-empty-policy",
set(),
)
assert projection is not None
provider = LocalSandboxProvider()
sandbox_id = provider.acquire(
"thread-empty-policy",
user_id="user-1",
)
sandbox = provider.get(sandbox_id)
assert sandbox is not None
listing = "\n".join(sandbox.list_dir("/mnt/skills", max_depth=4))
assert "public-skill" not in listing
assert "custom-skill" not in listing
shell_listing = sandbox.execute_command("ls -R /mnt/skills")
assert "public-skill" not in shell_listing
assert "custom-skill" not in shell_listing
assert "MARKER" not in sandbox.execute_command("find /mnt/skills -name SKILL.md -print -exec cat {} \\;")
with pytest.raises(FileNotFoundError):
sandbox.read_file("/mnt/skills/public/public-skill/SKILL.md")
globbed, _ = sandbox.glob("/mnt/skills", "**/SKILL.md")
assert globbed == []
grepped, _ = sandbox.grep(
"/mnt/skills",
"MARKER",
literal=True,
)
assert grepped == []
# ── AioSandboxProvider ──────────────────────────────────────────────
def test_aio_public_skill_mount(self, skills_fs, aio_mod):

View File

@ -203,6 +203,20 @@ def test_build_subagent_runtime_middlewares_threads_app_config_to_llm_middleware
assert policy_idx < durable_idx < date_idx == len(middlewares) - 2
def test_subagent_runtime_sandbox_does_not_own_lead_skill_projection() -> None:
from deerflow.extensions.registry import ExtensionRegistry
from deerflow.sandbox.middleware import SandboxMiddleware
middlewares = build_subagent_runtime_middlewares(
app_config=_make_app_config(),
available_skills={"allowed"},
extensions=ExtensionRegistry().build(),
)
sandbox_middleware = next(middleware for middleware in middlewares if isinstance(middleware, SandboxMiddleware))
assert sandbox_middleware._owns_agent_skill_projection is False
def test_tool_progress_middleware_is_outer_relative_to_error_handling(monkeypatch: pytest.MonkeyPatch):
# ToolProgressMiddleware must have a lower index than ToolErrorHandlingMiddleware
# so that the framework's "first in list = outermost" rule makes it outer.
@ -289,6 +303,63 @@ def test_lead_runtime_middlewares_thread_app_config_to_tool_error_handling(monke
assert tool_middleware._app_config is app_config
def test_lead_runtime_middlewares_pass_agent_skills_to_sandbox(
monkeypatch: pytest.MonkeyPatch,
):
monkeypatch.setitem(
sys.modules,
"deerflow.agents.middlewares.input_sanitization_middleware",
_module(
"deerflow.agents.middlewares.input_sanitization_middleware",
InputSanitizationMiddleware=object,
neutralize_untrusted_tags=lambda value: value,
),
)
app_config = _make_app_config()
_stub_runtime_middleware_imports(monkeypatch)
middlewares = build_lead_runtime_middlewares(
app_config=app_config,
available_skills={"allowed-skill"},
)
sandbox_middleware = next(middleware for middleware in middlewares if getattr(middleware, "kwargs", {}).get("available_skills") == {"allowed-skill"})
assert sandbox_middleware.kwargs == {
"lazy_init": True,
"available_skills": {"allowed-skill"},
"owns_agent_skill_projection": True,
}
def test_lead_runtime_middlewares_can_delegate_skill_projection_ownership(
monkeypatch: pytest.MonkeyPatch,
):
monkeypatch.setitem(
sys.modules,
"deerflow.agents.middlewares.input_sanitization_middleware",
_module(
"deerflow.agents.middlewares.input_sanitization_middleware",
InputSanitizationMiddleware=object,
neutralize_untrusted_tags=lambda value: value,
),
)
app_config = _make_app_config()
_stub_runtime_middleware_imports(monkeypatch)
middlewares = build_lead_runtime_middlewares(
app_config=app_config,
available_skills={"bootstrap"},
owns_agent_skill_projection=False,
)
sandbox_middleware = next(middleware for middleware in middlewares if getattr(middleware, "kwargs", {}).get("available_skills") == {"bootstrap"})
assert sandbox_middleware.kwargs == {
"lazy_init": True,
"available_skills": {"bootstrap"},
"owns_agent_skill_projection": False,
}
def test_build_lead_runtime_middlewares_orders_thread_data_before_uploads():
"""ThreadDataMiddleware must run before UploadsMiddleware so the uploads
directory is guaranteed to exist when UploadsMiddleware scans it under

View File

@ -1688,6 +1688,10 @@ skills:
# Path where skills are mounted in the sandbox container
# This is used by the agent to access skills in both local and Docker sandbox
# AIO/provisioner and E2B modes require a canonical absolute non-root path
# that does not overlap reserved platform mounts. Providers snapshot this
# path for identity, mounts, metadata, and synchronization, so restart the
# Gateway after changing it.
# Default: /mnt/skills
container_path: /mnt/skills

View File

@ -20,12 +20,13 @@ The **Sandbox Provisioner** is a FastAPI service that dynamically manages sandbo
### How It Works
1. **Backend Request**: When the backend needs to execute code, it sends a `POST /api/sandboxes` request with a `sandbox_id`, `thread_id`, and optional `user_id`.
1. **Backend Request**: When the backend needs to execute code, it sends a `POST /api/sandboxes` request with a `sandbox_id`, `thread_id`, optional `user_id`, and the configured `skills_container_path` (default: `/mnt/skills`).
2. **Pod Creation**: The provisioner creates a dedicated Pod in the `deer-flow` namespace with:
- The sandbox container image (all-in-one-sandbox)
- HostPath volumes mounted for:
- `/mnt/skills/{public,custom,legacy}` → Read-only enabled-only skill projections
- `{skills_container_path}/{public,custom,legacy}` → Default read-only skill projections
- `{skills_container_path}/integrations` → Optional read-only managed-integration projection supplied by the Gateway
- `/mnt/user-data` → Read-write access to thread-specific data
- Resource limits (CPU, memory, ephemeral storage)
- Readiness/liveness probes
@ -208,7 +209,7 @@ PYTHONPATH=. python scripts/migrate_user_isolation.py --user-id <target-user-id>
This moves legacy `threads/{thread_id}/user-data` data under `users/<target-user-id>/threads/{thread_id}/user-data`, which matches the new provisioner PVC subPath when the gateway base directory is mounted at `deer-flow/` on the PVC. Use `default` as the target user only when the legacy data should remain in the default no-auth user namespace. Run the migration while no gateway or sandbox Pods are writing to those paths.
In hostPath mode, the gateway materializes enabled-only views under `skills_view/public` and `users/{user_id}/skills_view/{custom,legacy}` beneath `DEER_FLOW_HOST_BASE_DIR`; the provisioner mounts those stable directories. When skills are materialized per thread on the same PVC, set `SKILLS_PVC_NAME` to that PVC and configure `SKILLS_PVC_SUBPATH_TEMPLATE=deer-flow/users/{user_id}/threads/{thread_id}/skills`. Leaving the template empty preserves the legacy behavior of mounting the skills PVC root at `/mnt/skills`. The gateway does not yet populate that PVC layout dynamically, so PVC-backed skills do not receive hostPath projection updates.
In hostPath mode, the gateway materializes enabled-only views under `skills_view/public` and `users/{user_id}/skills_view/{custom,legacy}` beneath `DEER_FLOW_HOST_BASE_DIR`; the provisioner mounts those stable directories. A lead Agent with an explicit skills policy supplies all four category mounts from `users/{user_id}/threads/{thread_id}/skills_view`; those overrides replace the default hostPath or root skills-PVC mount. When `USERDATA_PVC_NAME` is configured, the provisioner mounts these thread categories from that PVC using `deer-flow/users/{user_id}/threads/{thread_id}/skills_view/{category}` subpaths. For unrestricted threads, operators can still set `SKILLS_PVC_NAME` and optionally configure `SKILLS_PVC_SUBPATH_TEMPLATE`; leaving the template empty mounts the skills PVC root unchanged. The gateway does not populate the unrestricted `SKILLS_PVC_SUBPATH_TEMPLATE` layout dynamically.
**hostPath skills volumes require the gateway and the K8s node to see the same `DEER_FLOW_HOST_BASE_DIR`** (single-node deployment, or NFS/shared storage mounted at that path on every node). The gateway writes the projection there before every sandbox acquire, so as long as that path is shared, the directory the provisioner mounts always exists by the time the Pod is scheduled — even a boot-time rebuild failure for one user self-heals on their next acquire, before the provisioner is called. `skills-custom` and `skills-legacy` use hostPath type `Directory` (not `DirectoryOrCreate`): if the shared-storage assumption is violated — the gateway wrote to a different node than the one the Pod lands on — Pod creation now fails visibly instead of silently mounting an empty directory. Use `SKILLS_PVC_NAME` instead of hostPath for genuinely multi-node clusters without shared storage.

View File

@ -36,6 +36,7 @@ import re
import secrets
import time
from contextlib import asynccontextmanager
from pathlib import PurePosixPath
import urllib3
from fastapi import FastAPI, HTTPException, Request, Response
@ -107,16 +108,26 @@ if SANDBOX_SERVICE_TYPE not in {"NodePort", "ClusterIP"}:
SAFE_THREAD_ID_PATTERN = r"^[A-Za-z0-9_-]{1,64}$"
SAFE_USER_ID_PATTERN = r"^[A-Za-z0-9_\-]+$"
DEFAULT_USER_ID = "default"
DEFAULT_SKILLS_CONTAINER_PATH = "/mnt/skills"
MAX_EXTRA_MOUNTS = 10
ALLOWED_EXTRA_MOUNT_PATHS = {
"/mnt/acp-workspace",
"/mnt/skills/custom",
"/mnt/skills/integrations",
"/mnt/integrations/lark-cli/config",
"/mnt/integrations/lark-cli/config/locks",
"/mnt/integrations/lark-cli/data",
"/mnt/integrations/lark-cli/runtime",
}
MANAGED_SKILL_CATEGORY_NAMES = (
"public",
"custom",
"legacy",
"integrations",
)
RESERVED_SANDBOX_MOUNT_PATHS = (
"/mnt/user-data",
"/mnt/acp-workspace",
"/mnt/integrations/lark-cli",
)
# Path to the kubeconfig *inside* the provisioner container.
# Typically the host's ~/.kube/config is mounted here.
@ -171,16 +182,57 @@ def _is_path_under_base(path: str, base: str) -> bool:
return False
def _normalize_extra_mount_container_path(container_path: str) -> str:
def _normalize_skills_container_path(container_path: str) -> str:
"""Return a canonical skills root that cannot overlap platform mounts."""
if not container_path or not container_path.startswith("/") or container_path.startswith("//"):
raise HTTPException(status_code=400, detail="The skills container path must be an absolute non-root path")
normalized = posixpath.normpath(container_path)
if normalized == "/" or normalized != container_path:
raise HTTPException(
status_code=400,
detail="The skills container path must not be root or contain redundant separators, '.' or '..'",
)
root = PurePosixPath(normalized)
for reserved_path in RESERVED_SANDBOX_MOUNT_PATHS:
reserved = PurePosixPath(reserved_path)
if root == reserved or root.is_relative_to(reserved) or reserved.is_relative_to(root):
raise HTTPException(
status_code=400,
detail=f"The skills container path {normalized!r} overlaps reserved sandbox path {reserved_path!r}",
)
return normalized
def _managed_skill_category_mount_paths(
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
) -> set[str]:
root = _normalize_skills_container_path(skills_container_path)
return {posixpath.join(root, category) for category in MANAGED_SKILL_CATEGORY_NAMES}
def _normalize_extra_mount_container_path(
container_path: str,
*,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
) -> str:
normalized = posixpath.normpath(container_path)
if not normalized.startswith("/"):
raise HTTPException(status_code=400, detail=f"Extra mount path must be absolute: {container_path}")
if normalized not in ALLOWED_EXTRA_MOUNT_PATHS:
allowed_paths = ALLOWED_EXTRA_MOUNT_PATHS | _managed_skill_category_mount_paths(
skills_container_path
)
if normalized not in allowed_paths:
raise HTTPException(status_code=400, detail=f"Unsupported extra mount path: {container_path}")
return normalized
def _validated_extra_mounts(extra_mounts: list["ExtraMount"] | None) -> list["ExtraMount"]:
def _validated_extra_mounts(
extra_mounts: list["ExtraMount"] | None,
*,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
) -> list["ExtraMount"]:
"""Validate extra mounts before converting them into K8s hostPath/PVC mounts."""
if not extra_mounts:
return []
@ -197,7 +249,10 @@ def _validated_extra_mounts(extra_mounts: list["ExtraMount"] | None) -> list["Ex
if not _is_path_under_base(host_path, host_base_dir):
raise HTTPException(status_code=400, detail=f"Extra mount host path is outside DeerFlow state: {mount.host_path}")
container_path = _normalize_extra_mount_container_path(mount.container_path)
container_path = _normalize_extra_mount_container_path(
mount.container_path,
skills_container_path=skills_container_path,
)
if container_path in seen_container_paths:
raise HTTPException(status_code=400, detail=f"Duplicate extra mount path: {container_path}")
seen_container_paths.add(container_path)
@ -259,14 +314,21 @@ def _runtime_provided_extra_mounts(
return [mount for mount in extra_mounts if posixpath.normpath(mount.container_path) not in dropped]
def _lark_broker_credential_mounts(extra_mounts: list["ExtraMount"] | None) -> dict[str, "ExtraMount"]:
def _lark_broker_credential_mounts(
extra_mounts: list["ExtraMount"] | None,
*,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
) -> dict[str, "ExtraMount"]:
"""Extract the config/locks/data mounts the broker sidecar needs.
Keyed by container path so the caller can wire each into the sidecar's fixed
``/var/lark/{config,config/locks,data}`` paths.
"""
result: dict[str, ExtraMount] = {}
for mount in _validated_extra_mounts(extra_mounts):
for mount in _validated_extra_mounts(
extra_mounts,
skills_container_path=skills_container_path,
):
normalized = posixpath.normpath(mount.container_path)
if normalized in (
LARK_CLI_CONFIG_CONTAINER_PATH,
@ -409,6 +471,8 @@ class CreateSandboxRequest(BaseModel):
user_id: str = Field(default=DEFAULT_USER_ID, pattern=SAFE_USER_ID_PATTERN)
extra_mounts: list[ExtraMount] = Field(default_factory=list)
include_legacy_skills: bool = False
# Sent explicitly by new Gateways; the default keeps old Gateways compatible.
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH
# When true (and LARK_CLI_INIT_IMAGE is configured), provision the sandbox
# lark-cli runtime via an init container + emptyDir instead of a runtime
# hostPath/PVC extra mount.
@ -447,9 +511,18 @@ def _sandbox_url(sandbox_id: str, node_port: int | None = None) -> str:
return f"http://{NODE_HOST}:{node_port}"
def _build_extra_volumes(extra_mounts: list[ExtraMount] | None = None) -> list[k8s_client.V1Volume]:
def _build_extra_volumes(
extra_mounts: list[ExtraMount] | None = None,
*,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
) -> list[k8s_client.V1Volume]:
volumes: list[k8s_client.V1Volume] = []
for index, mount in enumerate(_validated_extra_mounts(extra_mounts)):
for index, mount in enumerate(
_validated_extra_mounts(
extra_mounts,
skills_container_path=skills_container_path,
)
):
if USERDATA_PVC_NAME:
volumes.append(
k8s_client.V1Volume(
@ -473,9 +546,18 @@ def _build_extra_volumes(extra_mounts: list[ExtraMount] | None = None) -> list[k
return volumes
def _build_extra_volume_mounts(extra_mounts: list[ExtraMount] | None = None) -> list[k8s_client.V1VolumeMount]:
def _build_extra_volume_mounts(
extra_mounts: list[ExtraMount] | None = None,
*,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
) -> list[k8s_client.V1VolumeMount]:
mounts: list[k8s_client.V1VolumeMount] = []
for index, mount in enumerate(_validated_extra_mounts(extra_mounts)):
for index, mount in enumerate(
_validated_extra_mounts(
extra_mounts,
skills_container_path=skills_container_path,
)
):
volume_mount = k8s_client.V1VolumeMount(
name=_extra_mount_volume_name(index),
mount_path=mount.container_path,
@ -493,25 +575,42 @@ def _build_volumes(
*,
include_legacy_skills: bool = False,
extra_mounts: list[ExtraMount] | None = None,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
provision_lark_cli_runtime: bool = False,
provision_lark_cli_broker: bool = False,
) -> list[k8s_client.V1Volume]:
"""Build volume list: PVC when configured, otherwise hostPath.
Skills are split into public, per-user custom, and legacy (global-custom)
volumes so that ``/mnt/skills/{public,custom,legacy}/`` paths resolve
volumes so that ``<skills-root>/{public,custom,legacy}/`` paths resolve
correctly inside the sandbox matching the hostPath layout produced by
``LocalSandboxProvider`` and ``AioSandboxProvider``.
"""
volumes: list[k8s_client.V1Volume] = []
del include_legacy_skills # retained for request compatibility
skills_root = _normalize_skills_container_path(skills_container_path)
managed_skill_paths = _managed_skill_category_mount_paths(skills_root)
validated_extra_mounts = _validated_extra_mounts(
extra_mounts,
skills_container_path=skills_root,
)
skill_overrides = {
posixpath.normpath(mount.container_path)
for mount in validated_extra_mounts
if posixpath.normpath(mount.container_path)
in managed_skill_paths
}
all_skill_categories_overridden = managed_skill_paths <= skill_overrides
# ── Skills volumes ────────────────────────────────────────────────
if SKILLS_PVC_NAME:
# PVC mode: three-way subPath not yet supported; fall back to
# single-volume mount for backward compatibility.
logger.warning("SKILLS_PVC_NAME is set — three-way skills layout is not supported in PVC mode yet; falling back to single /mnt/skills mount")
if SKILLS_PVC_NAME and not all_skill_categories_overridden:
# An unrestricted thread keeps the operator-provided skills PVC root.
logger.warning(
"SKILLS_PVC_NAME is set — three-way skills layout is not supported in PVC mode yet; "
"falling back to single %s mount",
skills_root,
)
volumes.append(
k8s_client.V1Volume(
name="skills",
@ -521,18 +620,19 @@ def _build_volumes(
),
)
)
else:
elif not SKILLS_PVC_NAME:
# hostPath mode: three-way layout
public_path = join_host_path(DEER_FLOW_HOST_BASE_DIR, "skills_view", "public")
volumes.append(
k8s_client.V1Volume(
name="skills-public",
host_path=k8s_client.V1HostPathVolumeSource(
path=public_path,
type="Directory",
),
if posixpath.join(skills_root, "public") not in skill_overrides:
volumes.append(
k8s_client.V1Volume(
name="skills-public",
host_path=k8s_client.V1HostPathVolumeSource(
path=public_path,
type="Directory",
),
)
)
)
user_custom_path = join_host_path(
DEER_FLOW_HOST_BASE_DIR,
@ -541,28 +641,30 @@ def _build_volumes(
"skills_view",
"custom",
)
volumes.append(
k8s_client.V1Volume(
name="skills-custom",
host_path=k8s_client.V1HostPathVolumeSource(
path=user_custom_path,
type="Directory",
),
if posixpath.join(skills_root, "custom") not in skill_overrides:
volumes.append(
k8s_client.V1Volume(
name="skills-custom",
host_path=k8s_client.V1HostPathVolumeSource(
path=user_custom_path,
type="Directory",
),
)
)
)
legacy_path = join_host_path(
DEER_FLOW_HOST_BASE_DIR, "users", user_id, "skills_view", "legacy"
)
volumes.append(
k8s_client.V1Volume(
name="skills-legacy",
host_path=k8s_client.V1HostPathVolumeSource(
path=legacy_path,
type="Directory",
),
if posixpath.join(skills_root, "legacy") not in skill_overrides:
volumes.append(
k8s_client.V1Volume(
name="skills-legacy",
host_path=k8s_client.V1HostPathVolumeSource(
path=legacy_path,
type="Directory",
),
)
)
)
# ── User-data volume ──────────────────────────────────────────────
@ -586,10 +688,11 @@ def _build_volumes(
volumes.extend(
_build_extra_volumes(
_runtime_provided_extra_mounts(
extra_mounts,
validated_extra_mounts,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
)
),
skills_container_path=skills_root,
)
)
# The runtime emptyDir is shared by the init container (writer) and the
@ -603,7 +706,10 @@ def _build_volumes(
)
# Pattern B: config/locks/data volumes go to the broker sidecar only.
if _lark_cli_broker_enabled(provision_lark_cli_broker):
credential_mounts = _lark_broker_credential_mounts(extra_mounts)
credential_mounts = _lark_broker_credential_mounts(
validated_extra_mounts,
skills_container_path=skills_root,
)
for container_path, volume_name in (
(LARK_CLI_CONFIG_CONTAINER_PATH, LARK_BROKER_CONFIG_VOLUME_NAME),
(LARK_CLI_LOCKS_CONTAINER_PATH, LARK_BROKER_LOCKS_VOLUME_NAME),
@ -640,23 +746,37 @@ def _build_volume_mounts(
*,
include_legacy_skills: bool = False,
extra_mounts: list[ExtraMount] | None = None,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
provision_lark_cli_runtime: bool = False,
provision_lark_cli_broker: bool = False,
) -> list[k8s_client.V1VolumeMount]:
"""Build volume mount list, mirroring three-way skills layout.
Skills are mounted to ``/mnt/skills/{public,custom,legacy}/`` so that
Skills are mounted to ``<skills-root>/{public,custom,legacy}/`` so that
category-aware ``Skill.get_container_path()`` paths resolve correctly.
PVC mode falls back to a single ``/mnt/skills`` mount and can optionally
PVC mode falls back to a single ``<skills-root>`` mount and can optionally
scope that mount with ``SKILLS_PVC_SUBPATH_TEMPLATE``.
"""
mounts: list[k8s_client.V1VolumeMount] = []
del include_legacy_skills # retained for request compatibility
skills_root = _normalize_skills_container_path(skills_container_path)
managed_skill_paths = _managed_skill_category_mount_paths(skills_root)
validated_extra_mounts = _validated_extra_mounts(
extra_mounts,
skills_container_path=skills_root,
)
skill_overrides = {
posixpath.normpath(mount.container_path)
for mount in validated_extra_mounts
if posixpath.normpath(mount.container_path)
in managed_skill_paths
}
all_skill_categories_overridden = managed_skill_paths <= skill_overrides
if SKILLS_PVC_NAME:
if SKILLS_PVC_NAME and not all_skill_categories_overridden:
skills_mount = k8s_client.V1VolumeMount(
name="skills",
mount_path="/mnt/skills",
mount_path=skills_root,
read_only=True,
)
if SKILLS_PVC_SUBPATH_TEMPLATE:
@ -665,26 +785,25 @@ def _build_volume_mounts(
thread_id=thread_id,
)
mounts.append(skills_mount)
else:
mounts.extend(
[
k8s_client.V1VolumeMount(
name="skills-public",
mount_path="/mnt/skills/public",
read_only=True,
),
k8s_client.V1VolumeMount(
name="skills-custom",
mount_path="/mnt/skills/custom",
read_only=True,
),
k8s_client.V1VolumeMount(
name="skills-legacy",
mount_path="/mnt/skills/legacy",
read_only=True,
),
]
)
elif not SKILLS_PVC_NAME:
default_skill_mounts = [
k8s_client.V1VolumeMount(
name="skills-public",
mount_path=posixpath.join(skills_root, "public"),
read_only=True,
),
k8s_client.V1VolumeMount(
name="skills-custom",
mount_path=posixpath.join(skills_root, "custom"),
read_only=True,
),
k8s_client.V1VolumeMount(
name="skills-legacy",
mount_path=posixpath.join(skills_root, "legacy"),
read_only=True,
),
]
mounts.extend(mount for mount in default_skill_mounts if mount.mount_path not in skill_overrides)
userdata_mount = k8s_client.V1VolumeMount(
name="user-data",
@ -697,10 +816,11 @@ def _build_volume_mounts(
mounts.extend(
_build_extra_volume_mounts(
_runtime_provided_extra_mounts(
extra_mounts,
validated_extra_mounts,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
)
),
skills_container_path=skills_root,
)
)
# Sandbox reads the runtime dir (real binary in Pattern A, shim in Pattern B).
@ -766,6 +886,8 @@ def _build_lark_cli_init_containers(
def _build_lark_cli_broker_sidecars(
provision_lark_cli_broker: bool,
extra_mounts: list[ExtraMount] | None,
*,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
) -> list[k8s_client.V1Container]:
"""Broker sidecar that holds lark-cli + the per-user credentials (Pattern B).
@ -776,7 +898,10 @@ def _build_lark_cli_broker_sidecars(
"""
if not _lark_cli_broker_enabled(provision_lark_cli_broker):
return []
credential_mounts = _lark_broker_credential_mounts(extra_mounts)
credential_mounts = _lark_broker_credential_mounts(
extra_mounts,
skills_container_path=skills_container_path,
)
volume_mounts: list[k8s_client.V1VolumeMount] = []
for container_path, volume_name, sidecar_path in (
(LARK_CLI_CONFIG_CONTAINER_PATH, LARK_BROKER_CONFIG_VOLUME_NAME, LARK_BROKER_SIDECAR_CONFIG_PATH),
@ -830,6 +955,7 @@ def _build_pod(
*,
include_legacy_skills: bool = False,
extra_mounts: list[ExtraMount] | None = None,
skills_container_path: str = DEFAULT_SKILLS_CONTAINER_PATH,
provision_lark_cli_runtime: bool = False,
provision_lark_cli_broker: bool = False,
) -> k8s_client.V1Pod:
@ -903,6 +1029,7 @@ def _build_pod(
user_id=user_id,
include_legacy_skills=include_legacy_skills,
extra_mounts=extra_mounts,
skills_container_path=skills_container_path,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
),
@ -911,7 +1038,11 @@ def _build_pod(
allow_privilege_escalation=True,
),
),
*_build_lark_cli_broker_sidecars(provision_lark_cli_broker, extra_mounts),
*_build_lark_cli_broker_sidecars(
provision_lark_cli_broker,
extra_mounts,
skills_container_path=skills_container_path,
),
],
init_containers=init_containers,
volumes=_build_volumes(
@ -919,6 +1050,7 @@ def _build_pod(
user_id=user_id,
include_legacy_skills=include_legacy_skills,
extra_mounts=extra_mounts,
skills_container_path=skills_container_path,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
),
@ -1032,15 +1164,19 @@ def create_sandbox(req: CreateSandboxRequest):
thread_id = req.thread_id or sandbox_id
user_id = req.user_id
include_legacy_skills = req.include_legacy_skills
skills_container_path = _normalize_skills_container_path(
req.skills_container_path
)
provision_lark_cli_runtime = req.provision_lark_cli_runtime
provision_lark_cli_broker = req.provision_lark_cli_broker
logger.info(
"Received request to create sandbox '%s' for thread '%s' user '%s' include_legacy_skills=%s provision_lark_cli_runtime=%s provision_lark_cli_broker=%s",
"Received request to create sandbox '%s' for thread '%s' user '%s' include_legacy_skills=%s skills_container_path=%s provision_lark_cli_runtime=%s provision_lark_cli_broker=%s",
sandbox_id,
thread_id,
user_id,
include_legacy_skills,
skills_container_path,
_lark_cli_runtime_enabled(provision_lark_cli_runtime),
_lark_cli_broker_enabled(provision_lark_cli_broker),
)
@ -1064,6 +1200,7 @@ def create_sandbox(req: CreateSandboxRequest):
user_id=user_id,
include_legacy_skills=include_legacy_skills,
extra_mounts=req.extra_mounts,
skills_container_path=skills_container_path,
provision_lark_cli_runtime=provision_lark_cli_runtime,
provision_lark_cli_broker=provision_lark_cli_broker,
),

View File

@ -133,6 +133,16 @@ skills:
- **Empty list `[]`**: the agent has no skills.
- **Named list**: the agent loads only those specific skills.
For a lead custom Agent, an explicit list is enforced in both discovery and the
sandbox filesystem. `/mnt/skills` contains only the intersection of globally
enabled, user-visible, and Agent-allowed skills. AIO and E2B enforce that view
for shell commands and file tools. Local enforces it through managed virtual
paths only while host bash is disabled (the default); its host filesystem is
not a security boundary, and enabling host bash makes an explicit Agent policy
fail closed. A different sandbox provider must advertise the same contract or
an explicit policy fails closed before the run acquires a sandbox. Agent
configuration changes take effect on the thread's next run.
## Skill evolution
DeerFlow includes an optional **skill evolution** feature that allows the agent to autonomously create and improve skills in the `skills/custom/` directory:

View File

@ -127,6 +127,13 @@ skills:
- **空列表 `[]`**Agent 没有技能。
- **命名列表**Agent 只加载那些特定技能。
对于作为主 Agent 运行的自定义 Agent显式列表会同时约束技能发现和沙箱文件系统。
`/mnt/skills` 只包含“全局已启用、当前用户可见、Agent 已允许”三者的交集。AIO 和
E2B 会对 shell 命令及文件工具强制执行该视图。Local 仅在 host bash 保持默认禁用时,
通过受管理的虚拟路径执行该视图;主机文件系统并不是安全边界,启用 host bash 后,
显式 Agent 策略会失败关闭。其他 sandbox provider 若未声明支持,显式策略会在获取
沙箱前失败关闭。Agent 配置变更会在该线程的下一次运行生效。
## 技能进化
DeerFlow 包含一个可选的**技能进化**功能,允许 Agent 在 `skills/custom/` 目录中自主创建和改进技能: