mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-11 06:28:58 +00:00
bb75f8d736
* feat(sandbox): share sandbox identity derivation and acquire serialization (#4741) Remote providers (AIO, E2B, BoxLite, Tenki, OpenSandbox) each inlined the same sha256(user:thread)[:16] sandbox-id expression and kept per-scope lock dicts that grew unboundedly until shutdown. This extracts both mechanisms into shared components without changing provider lifecycle, ids, capacity semantics, or public tool behavior: - sandbox/identity.py: keyword-only derive_sandbox_scope_token (byte-pinned compatibility contract) + is_sandbox_scope_token; per-provider golden vectors pin current behavior including BoxLite's raw-None quirk and each provider's private user_id resolution. - sandbox/acquire_serialization.py: AcquireSerializer — per-key lock table with holder/waiter refcount reclamation, bounded dedicated executor (async waits off both the event loop and the default executor), worker-owned cancellation cleanup (no event-loop callback dependency), idempotent close(). - Each provider adopts both components; AIO/E2B key by (user_id, thread_id) with acquire and (E2B) release serialized; BoxLite/Tenki/OpenSandbox key by derived sandbox id and offload the whole sync acquire to the serializer's executor so a cancelled awaiter cannot overlap a retried same-scope body (leaked-remote-VM regression caught in review). - thread_id=None acquires stay unserialized; provider shutdown()/reset() close the serializer; E2B capacity/ledger/reconciliation and AIO ownership/flock machinery untouched. - blocking-IO anchor proves contended OpenSandbox acquire_async stays off the event loop (teeth verified red/green); AGENTS.md documents the shared components. * refactor(sandbox): address review on acquire serialization (#5089) - Replace unreachable checkin branch with an assertion: run() returns False only after abandon(), which the except handler always re-raises; the old _checkin would have double-decremented the refcount. - Document the task.cancelling() == 0 assumption in hold_async. - Drop unused thread_id/user_id kwargs from BoxLite and Tenki _acquire_scope_locked (OpenSandbox still forwards them). * fix(sandbox): preserve request ContextVars in acquire executor bridge (#5089) loop.run_in_executor() does not copy contextvars, unlike the inherited SandboxProvider.acquire_async() which used asyncio.to_thread(). The BoxLite/OpenSandbox/Tenki acquire_async bridges introduced in this PR therefore dropped the request trace id (logged as trace_id=-). Add AcquireSerializer.run_on_executor(), which copies the calling context and runs the callable through ctx.run, and route all three providers through it. Add regression tests binding request_trace_context and verifying the worker thread observes it.
44 lines
1.7 KiB
Python
44 lines
1.7 KiB
Python
"""Shared sandbox scope-token derivation (RFC #4741).
|
|
|
|
COMPATIBILITY CONTRACT: this derivation is a durable identity boundary. AIO,
|
|
E2B, BoxLite, Tenki, and OpenSandbox locate existing containers and VMs
|
|
through this token. Changing any property — separator, encoding, digest,
|
|
casing, or truncation length — is a breaking migration: existing remote
|
|
resources would no longer be found and would be cold-started.
|
|
|
|
Providers resolve ``user_id`` before calling this function, and their
|
|
resolutions differ (effective-user lookup, ``""`` substitution, or raw
|
|
pass-through). That resolution stays provider-private; this module only pins
|
|
what happens to the resolved strings.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import hashlib
|
|
import re
|
|
|
|
SANDBOX_ID_VERSION = 1
|
|
|
|
_TOKEN_HEX_LEN = 16
|
|
_TOKEN_RE = re.compile(r"[0-9a-f]{16}")
|
|
|
|
|
|
def derive_sandbox_scope_token(*, user_id: str, thread_id: str) -> str:
|
|
"""Return the durable 16-lowercase-hex sandbox scope token.
|
|
|
|
Keyword-only: every provider helper this replaces takes
|
|
``(thread_id, user_id)`` positionally — the reverse order — and both are
|
|
plain ``str``; keyword-only call sites eliminate silent argument-order
|
|
mistakes during and after the migration.
|
|
|
|
WARNING: changing the separator, encoding, digest, casing, or truncation
|
|
length is a breaking migration — existing containers and VMs would no
|
|
longer be found. See module docstring.
|
|
"""
|
|
return hashlib.sha256(f"{user_id}:{thread_id}".encode()).hexdigest()[:_TOKEN_HEX_LEN]
|
|
|
|
|
|
def is_sandbox_scope_token(value: object) -> bool:
|
|
"""Validate token shape only; a truncated hash is not reversible."""
|
|
return isinstance(value, str) and _TOKEN_RE.fullmatch(value) is not None
|