fix(sandbox): derive the AIO sandbox ID off the event loop in async acquire (#7507)

* fix(sandbox): derive the AIO sandbox ID off the event loop in async acquire

_acquire_internal_async offloads every acquire layer except the sandbox ID
derivation, which ran inline. _sandbox_id_for_thread checks the thread's
skills_view directory with exists(), and resolving it goes through
Paths.base_dir, so each async thread acquire made filesystem syscalls on the
event loop. The strict blocking-I/O gate raises on it.

Run the derivation through the same owned-worker helper as the reuse and
warm-pool layers around it, and anchor it in the blocking-I/O suite for both
the default and the policy-scoped ID.

* docs(changelog): reference #7507 for the AIO async acquire sandbox ID fix

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

CHANGELOG.md

@@ -1397,6 +1397,12 @@ As of 2026-10-21, the milestone includes **829 merged pull requests**.
 
 ### Fixed
 
+- **sandbox:** The AIO provider's async acquire no longer checks the thread's
+  skill directory on the event loop. Picking the sandbox ID ran a filesystem
+  `stat` and path resolution inline on every thread acquire, which stalls the
+  loop when `DEER_FLOW_HOME` is on slow or network storage. It now runs in a
+  worker like the rest of acquire. ([#7507])
+
 - **threads:** `POST /api/threads` now rejects an `assistant_id` that cannot
   name an agent with `422`. It used to store the value, and every later read
   of the thread (`GET` thread, state, history) answered
@@ -13084,3 +13090,4 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
 [#7410]: https://github.com/bytedance/deer-flow/pull/7410
 [#7445]: https://github.com/bytedance/deer-flow/pull/7445
 [#7449]: https://github.com/bytedance/deer-flow/pull/7449
+[#7507]: https://github.com/bytedance/deer-flow/pull/7507

CHANGELOG_zh.md

@@ -1112,6 +1112,11 @@
 
 ### 修复
 
+- **沙箱：** AIO 提供方的异步 acquire 不再在事件循环上检查线程的技能目录。此前每次
+  线程 acquire 选择沙箱 ID 时，都会在事件循环上直接执行文件系统 `stat` 和路径解析；
+  `DEER_FLOW_HOME` 位于较慢或网络存储上时会阻塞事件循环。现在这一步与 acquire 的
+  其他步骤一样在工作线程中执行。([#7507])
+
 - **线程：** `POST /api/threads` 现在会以 `422` 拒绝无法对应任何智能体的
   `assistant_id`。此前该值会被直接保存，之后读取该线程（`GET` 线程、状态、历史）
   都会返回 `500 Internal Server Error`，直到删除线程。修复前已这样保存的线程
@@ -10445,3 +10450,4 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写，核心包含子
 [#7410]: https://github.com/bytedance/deer-flow/pull/7410
 [#7445]: https://github.com/bytedance/deer-flow/pull/7445
 [#7449]: https://github.com/bytedance/deer-flow/pull/7449
+[#7507]: https://github.com/bytedance/deer-flow/pull/7507

backend/packages/harness/deerflow/community/aio_sandbox/aio_sandbox_provider.py

@@ -2319,8 +2319,15 @@ class AioSandboxProvider(WarmPoolLifecycleMixin[SandboxInfo], SandboxProvider):
         if cached_id is not None:
             return cached_id
 
-        # Deterministic ID for thread-specific, random for anonymous
-        sandbox_id = self._sandbox_id_for_thread(thread_id, user_id)
+        # Deterministic ID for thread-specific, random for anonymous. Deriving
+        # it checks the thread's skills_view on disk through Paths.base_dir, so
+        # it runs in a worker like the layers around it.
+        sandbox_id = await _run_started_acquire_worker(
+            self._acquire_worker_executor,
+            self._sandbox_id_for_thread,
+            thread_id,
+            user_id,
+        )
         if thread_id:
             key = self._thread_key(thread_id, user_id)
             with self._lock:

backend/packages/harness/deerflow/sandbox/AGENTS.md

@@ -29,7 +29,7 @@ cancellation before propagating it.
 **Interface**: `Sandbox`: `execute_command(command, env=None)`, additive `execute_command_in_scope(..., scope_id=...)` / `release_command_scope(scope_id)`, `read_file`, `write_file`, `list_dir`, `glob`, `grep`. Scoped hooks default to pass-through without server-side sessions, preserving third-party subclasses. `grep` accepts a text file or directory tree. Per-call `env` injects secrets: `LocalSandbox` merges into the subprocess environment; `AioSandbox` uses fresh `bash.exec(env=...)` sessions. `list_dir`: missing path → `FileNotFoundError`; command/client failure → `OSError`, never `[]` (`ls_tool`: `(empty)`). Remote `glob`/`grep` share it via `sandbox/remote_search.py`: missing root → `FileNotFoundError`, failed search → `OSError`; only a genuine no-match returns `[]`. The parser takes the command's `limit` and reports `truncated` when output passed it, which providers return after Python-side filtering; tools call an empty truncated result incomplete. Remote `grep(glob=...)` scopes like `glob()` (root-relative `path_matches`), never by basename alone. Remotes use `sandbox/remote_list_dir.py`: capture `find`'s status, not `| head`'s (`sh -lc` lacks `pipefail`); missing binary → `OSError`; truncation SIGPIPE → success.
 **Bounded downloads**: `sandbox/download.py` owns `DEFAULT_DOWNLOAD_LIMIT` (100 MiB) and `MAX_IMAGE_BYTES` (20 MiB), shared by providers and vision readers. `download_file(path, max_bytes=...)` enforces the smaller of the caller and provider limits during reading. Streaming readers close on every exit and discard the first overflowing chunk without consuming later chunks; SDK chunk sizing still determines over-read. Local/BoxLite readers probe one extra byte. Oversize is `OSError(EFBIG)`, never a successful prefix. Explicit E2B limits require streaming; custom providers must implement the keyword for vision callers, which fail closed without an unbounded retry. AIO holds its transfer lock through stream cleanup so `close()` cannot close the shared HTTP client mid-read. Keep cancellation draining before lease release. Tests: `test_image_download_limit.py` and image cancellation tests.
 
-**Provider Pattern**: `SandboxProvider`: `acquire`, `acquire_async`, `get`, `get_scoped`, `release`. `get_scoped` checks the identity-scoped cache without blocking; a miss uses canonical `acquire(user_id, thread_id)`, never a checkpoint id. AIO reads active identity maps. Async hooks keep Docker creation/discovery, cross-process locking, readiness checks, and release off-loop. Set `supports_agent_skill_isolation=True` only if every tool enforces explicit lead Agent policy: prepared thread-root bind mounts plus upload `sync_agent_skills`. Host-backed providers report false when enabled shells bypass path mappings; middleware rejects them before explicit-policy acquire.
+**Provider Pattern**: `SandboxProvider`: `acquire`, `acquire_async`, `get`, `get_scoped`, `release`. `get_scoped` checks the identity-scoped cache without blocking; a miss uses canonical `acquire(user_id, thread_id)`, never a checkpoint id. AIO reads active identity maps. Async hooks keep Docker creation/discovery, sandbox ID derivation, cross-process locking, readiness checks, and release off-loop. Set `supports_agent_skill_isolation=True` only if every tool enforces explicit lead Agent policy: prepared thread-root bind mounts plus upload `sync_agent_skills`. Host-backed providers report false when enabled shells bypass path mappings; middleware rejects them before explicit-policy acquire.
 **Shared components** (RFC #4741): remote IDs use `derive_sandbox_scope_token` (`sandbox/identity.py`); preserve its keyword-only SHA-256/16-hex contract to avoid orphaning containers. `AcquireSerializer` (`sandbox/acquire_serialization.py`) serializes selected acquire/release transitions with a bounded, refcounted per-key `threading.Lock` table and dedicated bounded executor (no event-loop/default-executor blocking). Workers own cancellation cleanup without waiting for cancelled tasks to resume; provider `shutdown()`/`reset()` calls idempotent `close()`. Keys: AIO `(user_id, thread_id)`, E2B `(user_id, thread_id, skills_root)`, BoxLite/Tenki/OpenSandbox derived id. Random-UUID `thread_id=None` acquires bypass serialization.
 **Execution leases** (`sandbox/lease.py`, #5128): cross-instance ownership selects which Gateway may reap a container; process-local `SandboxLeaseManager` tracks a client's concurrent lead, subagent, Gateway-request, and channel-upload users. Runs have ephemeral owners. Restore persisted sandboxes only after `get_scoped(user_id, thread_id)` succeeds; otherwise canonical `acquire` replaces the id. Last holder performs pending `SandboxProvider.release`. External run/state boundaries reject server-owned `sandbox`. Outer lifecycle fences repeat idempotent release after each graph/tool/request batch; per-tool terminal `Command` wrappers never release while siblings run. Fork-restored children and upload syncs hold without requesting release: server-created forks may borrow the parent client; ordinary checkpoint ids stay identity-scoped. Serialize persisted lookup and retention by `(user_id, thread_id)`; stale or mismatched bindings reacquire, and a post-acquire miss rolls back then raises. Provider I/O runs outside the metadata lock. Repeated cancellation cannot interrupt acquire/rollback/release reconciliation or let `to_thread` work outlive its execution/request holder; failures are logged without replacing the original cancellation. Lease/scope context IDs are server-owned: Gateway/worker scrub caller values; only internal subagents assign task IDs. Managers key by provider object identity (not hash/equality), so unhashable custom providers work. Subagent owners double as `sandbox_command_scope_id`: AIO gives each scope an ordered persistent shell, replaces it after `ErrorObservation` or a missing-session 404, and cleans it on lease release. Revalidate registry identity after each scope-lock wait so queued commands cannot resurrect released sessions. Env-bearing commands use fresh `bash.exec` sessions so secrets do not persist.
 **Authorization gate** (`sandbox:execute`, RFC #4063 Phase 3): every sandbox-backed tool call passes through the gate in `deerflow/authz/sandbox_authz.py` - a binary `authorize(principal, "sandbox", "execute", target="*")` check before either reusing a persisted sandbox id or calling `provider.acquire`. Rechecking reuse is required because authorization config and user roles can change while the sandbox remains cached. Sync tool invocations call `authorize_sandbox_execution`; async tool invocations await `authorize_sandbox_execution_async` exactly once. A task-local `ContextVar` scopes that single decision across the complete composed tool invocation, including `ReadBeforeWriteMiddleware`'s pre-write inspection, tool body, and post-read mark; the value is copied into `asyncio.to_thread` workers. Authorization denial is converted to the normal error `ToolMessage` at the composed middleware boundary and is explicitly excluded from the gate's generic fail-open handlers. Async config loading and provider class discovery/import are offloaded before `aauthorize()` so reused sandbox calls do not hash config files or import custom modules on the event loop; provider construction remains on the running event loop because async providers may initialize loop-affine clients. The gate lives at the single tool initialization entry point (`ensure_sandbox_initialized` / `ensure_sandbox_initialized_async` in `tools.py`), while `SandboxMiddleware.before_agent` / `abefore_agent` apply the matching sync/async check to eager acquisition. Deny raises `SandboxAuthorizationError` (`sandbox/exceptions.py`), which propagates out of ordinary tool execution as a friendly error `ToolMessage` ("sandbox execution is not permitted for your role") - the eager path catches it and skips acquisition instead, deferring the deny to the first sandbox-touching tool call so both paths share the same semantics. Provider errors (authorization calls and provider resolution) follow `authorization.fail_closed` / `fail_open`; no readable `config.yaml` or `authorization.enabled: false` makes the gate a no-op (`safe_app_config` tolerates missing config). Gateway upload/artifact sync calls `try_acquire_sandbox_for_request` (`app/gateway/authz.py`), which gates, returns a request lease, and skips sync on deny while preserving the primary operation. Callers release after their last sandbox operation; artifacts request normal parking, uploads do not. Tests: `tests/test_sandbox_authorization.py` and `tests/blocking_io/test_sandbox_authorization.py`.

backend/tests/blocking_io/test_aio_sandbox_get.py

@@ -19,6 +19,7 @@ If ownership work is put back on this path, this test fails.
 
 from __future__ import annotations
 
+import asyncio
 import threading
 from pathlib import Path
 from unittest.mock import MagicMock
@@ -199,3 +200,66 @@ async def test_blocking_probe_thread_dir_actually_trips_the_gate(tmp_path, monke
 
     with pytest.raises(BlockingError):
         get_paths().thread_dir("aio-sandbox-lock-wait")
+
+
+@pytest.mark.parametrize("skills_view", [False, True], ids=["default-id", "policy-scoped-id"])
+async def test_async_acquire_derives_sandbox_id_off_the_loop(tmp_path, monkeypatch, skills_view) -> None:
+    """Deriving the sandbox ID checks the thread's skills_view on disk.
+
+    ``_sandbox_id_for_thread`` stats ``skills_view`` through ``Paths.base_dir``
+    to pick the policy-scoped or the historic deterministic ID. Every layer
+    around it in ``_acquire_internal_async`` already runs in a worker; the ID
+    derivation ran inline and blocked the loop on each thread acquire.
+    """
+    from concurrent.futures import ThreadPoolExecutor
+
+    from deerflow.config.paths import get_paths
+
+    provider = _make_provider(tmp_path)
+    provider._acquire_worker_executor = ThreadPoolExecutor(thread_name_prefix="aio-sandbox-owned-worker")
+    monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path))
+    monkeypatch.setattr("deerflow.config.paths._paths", None)
+    # Already offloaded with asyncio.to_thread; stubbed so the anchor does not
+    # build a real skills projection.
+    monkeypatch.setattr(provider, "_ensure_skills_projection", lambda user_id: None)
+    if skills_view:
+        await asyncio.to_thread(lambda: get_paths().thread_skills_view_dir("t-id", user_id="u1").mkdir(parents=True))
+    expected_id = await asyncio.to_thread(provider._sandbox_id_for_thread, "t-id", "u1")
+
+    reclaimed: list[str] = []
+
+    class _ReachedWarmPool(Exception):
+        pass
+
+    def _stop_at_warm_pool(thread_id, sandbox_id, *, user_id=None):
+        reclaimed.append(sandbox_id)
+        raise _ReachedWarmPool
+
+    monkeypatch.setattr(provider, "_reclaim_warm_pool_sandbox", _stop_at_warm_pool)
+
+    try:
+        # Red before the fix: the gate raises BlockingError from the inline
+        # ``_sandbox_id_for_thread`` before the warm-pool layer is reached.
+        with pytest.raises(_ReachedWarmPool):
+            await provider._acquire_internal_async("t-id", user_id="u1")
+    finally:
+        provider._acquire_worker_executor.shutdown(wait=True)
+
+    assert reclaimed == [expected_id]
+
+
+async def test_blocking_probe_sandbox_id_actually_trips_the_gate(tmp_path, monkeypatch) -> None:
+    """Meta-check: deriving a thread's sandbox ID inline on the loop is blocking.
+
+    Keeps the anchor above honest: if the gate stopped seeing the syscalls
+    behind ``thread_skills_view_dir(...).exists()``, the anchor would pass with
+    the inline call restored.
+    """
+    from blockbuster import BlockingError
+
+    provider = _make_provider(tmp_path)
+    monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path))
+    monkeypatch.setattr("deerflow.config.paths._paths", None)
+
+    with pytest.raises(BlockingError):
+        provider._sandbox_id_for_thread("t-id", "u1")