mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-15 17:18:38 +00:00
* fix(sandbox): harden local Docker sandbox containers and port binding Root causes (security audit SBX-1/SBX-2) in the local container backend: - _resolve_docker_bind_host published sandbox ports on 0.0.0.0 whenever DEER_FLOW_SANDBOX_HOST was non-loopback (docker-compose defaults to host.docker.internal), exposing the unauthenticated /v1/shell/* exec API on every host interface. - _start_container ran every sandbox with seccomp=unconfined and no capability, privilege-escalation, or resource limits, so untrusted model-authored code could exhaust the host, escalate privileges, and reach internal networks / cloud metadata endpoints directly. Hardening changes and defaults: - Port binding: non-loopback sandbox hosts now bind the Docker default bridge gateway instead of 0.0.0.0, discovered dynamically via `docker network inspect bridge` with a static 172.17.0.1 fallback. host.docker.internal resolves to that gateway through host-gateway, so DooD gateways and the Docker host still reach the sandbox while external interfaces no longer see the port. DEER_FLOW_SANDBOX_BIND_HOST=0.0.0.0 restores the legacy broad bind. - seccomp=unconfined is no longer unconditional: sandboxes run with Docker's default seccomp profile; opt back in with DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED=1, only when the sandbox image is verified to require syscalls the default profile blocks. - Add --cap-drop=ALL and --security-opt no-new-privileges (Docker only; the Apple Container CLI does not support these flags). - Bounded resources with env overrides: --memory 2g (DEER_FLOW_SANDBOX_MEMORY), --cpus 2 (DEER_FLOW_SANDBOX_CPUS), --pids-limit 512 (DEER_FLOW_SANDBOX_PIDS_LIMIT); each also accepts "0"/"none" to disable the limit. - No --user is forced by default (the default AIO sandbox image's user is upstream-controlled and unverified), but DEER_FLOW_SANDBOX_CONTAINER_USER passes one through for deployments that know their image. - DEER_FLOW_SANDBOX_NETWORK passes --network so sandboxes can be attached to a dedicated egress-controlled network; default networking is unchanged. backend/docs/CONFIGURATION.md documents the new bind behavior and every override; tests cover each default and escape hatch. * fix(sandbox): follow host-gateway mapping for binds; keep image-required seccomp default Review follow-ups on the hardening change: - Bind: resolve the sandbox host itself and bind that address, instead of assuming the default bridge IPv4. host.docker.internal follows the daemon host-gateway-ip mapping (customizable, possibly IPv6), so the resolved address is exactly where the gateway connects — the published port and advertised URL always match. IPv6 is bracketed for docker -p, zone ids stripped, wildcard resolutions ignored; unresolved hosts fall back to the bridge gateway with a warning pointing at DEER_FLOW_SANDBOX_BIND_HOST. - seccomp: the shipped AIO image needs seccomp=unconfined for its Chromium browser (upstream quick-start always passes it; the upstream FAQ documents the browser failing under Docker default profile), so that option returns as the default. Tightening stays possible via DEER_FLOW_SANDBOX_SECCOMP_PROFILE=<path to a restricted, Chromium-compatible profile> or DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED=0 for images verified to work with Docker's default profile. - cap-drop/no-new-privileges and the resource limits are unchanged. - Tests updated for both behaviors; 37 pass. * fix(sandbox): bracket bare IPv6 bind overrides; state seccomp default accurately DEER_FLOW_SANDBOX_BIND_HOST was returned verbatim, so a bare IPv6 literal like fd00::1 produced an invalid publish spec (fd00::1:port:8080); Docker requires the bracketed form. Normalize raw and already-bracketed IPv6 literals (IPv4/hostnames untouched), with resolver-level and argv-level tests covering the explicit IPv6 override. The CONFIGURATION.md overview claimed Docker's default seccomp profile stays active, contradicting the seccomp=unconfined default the table (and the code) actually ship for the Chromium-based image; spell out the relaxed default and where to change it. * style(sandbox): apply ruff format to local_backend * fix(sandbox): reject host networking, force builtin seccomp opt-out, resolve hostname binds Review follow-up on #4986 (willem-bd): - P1: DEER_FLOW_SANDBOX_NETWORK=host (and container:<name>) now raise a RuntimeError at start instead of silently voiding the hardened port bind — Docker discards -p/--publish in host mode and shares the network namespace for container:<name>, which would re-expose the unauthenticated exec API on the host's interfaces. Two regression tests cover both rejections. - P2: the seccomp opt-out now passes seccomp=builtin explicitly instead of omitting the option, so a daemon configured with an unconfined or custom default cannot weaken the documented opt-out; the test asserts the flag. - P2: hostname values in DEER_FLOW_SANDBOX_BIND_HOST resolve to an address before use (Docker publish specs require an IP literal as the host part, so host.docker.internal previously produced an invalid spec that prevented every sandbox from starting); unresolvable names raise a clear configuration error. Tests cover resolution and rejection; CONFIGURATION.md updated for all three behaviors. 43/43 pass in tests/test_aio_sandbox_local_backend.py; ruff check + format clean. * fix(sandbox): reject DEER_FLOW_SANDBOX_NETWORK=none (loopback-only, breaks published API port) * fix(sandbox): validate the effective Docker network target; normalize IPv6 sandbox hosts once name=host / name=none dodge raw-string checks but attach like the bare words; strip name= prefixes and validate the effective target (network IDs keep passing). Bracketed IPv6 sandbox hosts now resolve for the bind and bare IPv6 hosts produce bracketed URL authorities — both input forms give identical bind and URL addresses. * fix(sandbox): parse the full Docker network long syntax before validating Docker accepts comma-separated key=value fields in any order (name=, gw-priority=, alias=, ...); a name=host field hides the host network behind surrounding fields. Parse the CSV and validate the parsed name= target (last occurrence wins, fields lowercased, mirroring opts/network.go); no-name values fall through like Docker's own rejection. * fix(sandbox): keep CHOWN/SETUID/SETGID through cap-drop=ALL for the default image The shipped image's entrypoint starts as root, creates the gem user, chowns /opt/jupyter and drops to that user via su; without those three capabilities the set -e script dies before the readiness endpoint exists. no-new-privileges stays (it blocks gaining privileges via exec, not using the added caps). Adds a docker-gated real-image startup smoke test. * fix(sandbox): let pre-initialized non-root images drop the startup capabilities The CHOWN/SETUID/SETGID re-add only exists for the shipped image's root entrypoint handoff. A custom image that never runs as root gets an explicit opt-out (DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS=0) so those capabilities are not left available to sandboxed code (chown on bind mounts, UID/GID impersonation). * test(sandbox): gate the real-image smoke test behind the live marker The default offline suite (make test = -m 'not live') must not depend on a third-party registry: mark the smoke test live, probe the daemon inside the test body (never at collection time), and allow pinning the image reference via DEER_FLOW_SANDBOX_SMOKE_IMAGE for a dedicated integration job. * test/docs: isolate DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS in tests; add table row; split custom-image guidance _clear_hardening_env now clears the new knob so a developer shell or .env preset cannot flip the default-path tests. CONFIGURATION.md gains the table row, and the custom-image guidance becomes its own paragraph with the no-new-privileges scope stated correctly (it does not mitigate the retained CAP_SETUID/SETGID risk). * test(sandbox): make the live smoke test diagnosable 300s readiness budget (cold pull + cold start must not be conflated with broken capabilities) and dump the container's last 40 log lines on failure so the next live run tells us whether the capability set is incomplete (chown/useradd/su errors) or the services are merely slow. * test(ci): align the smoke test with the 60s provider deadline; add a dedicated live smoke workflow Single-source the readiness deadline as SANDBOX_LOCAL_PROVIDER_READY_TIMEOUT (used by both provider paths and the smoke test) so the validation cannot drift from the production contract again. New sandbox-image-smoke.yml runs the live test on a dedicated job, with the image reference pinnable via the SANDBOX_SMOKE_IMAGE repository variable (digest resolved and recorded in the job summary when falling back to :latest). * test(sandbox): pull the failing program's own logs on smoke failure supervisord only surfaces exit codes in docker logs; nginx's stderr lands in files inside the container. Dump supervisor program logs, nginx -t, and the nginx error log on failure so the next run names the exact broken line. * ci(sandbox): export an immutable repo@digest reference for the smoke run docker pull once on the runner platform, resolve RepoDigests[0], and pass that immutable reference to the test via GITHUB_ENV — the recorded and executed images can no longer diverge when the tag moves, and platform selection is left to the daemon instead of jq over the manifest index. * fix(sandbox): add DAC_OVERRIDE — the root nginx master writes gem-owned logs The image's root nginx master opens /var/log/nginx/{access,error}.log, which belong to the gem user, for the container's lifetime; without CAP_DAC_OVERRIDE it dies with 'open() failed (13: Permission denied)' on every start (FATAL under supervisord) and readiness never arrives. Four capabilities now: CHOWN/SETUID/SETGID for the entrypoint handoff plus this runtime log-write need.
1003 lines
38 KiB
Python
1003 lines
38 KiB
Python
import logging
|
|
import os
|
|
import socket
|
|
import subprocess
|
|
from types import SimpleNamespace
|
|
|
|
import pytest
|
|
|
|
from deerflow.community.aio_sandbox.local_backend import (
|
|
LocalContainerBackend,
|
|
_format_container_command_for_log,
|
|
_format_container_mount,
|
|
_redact_container_command_for_log,
|
|
_resolve_docker_bind_host,
|
|
)
|
|
|
|
|
|
def test_format_container_mount_uses_mount_syntax_for_docker_windows_paths():
|
|
args = _format_container_mount("docker", "D:/deer-flow/backend/.deer-flow/threads", "/mnt/threads", False)
|
|
|
|
assert args == [
|
|
"--mount",
|
|
"type=bind,src=D:/deer-flow/backend/.deer-flow/threads,dst=/mnt/threads",
|
|
]
|
|
|
|
|
|
def test_format_container_mount_marks_docker_readonly_mounts():
|
|
args = _format_container_mount("docker", "/host/path", "/mnt/path", True)
|
|
|
|
assert args == [
|
|
"--mount",
|
|
"type=bind,src=/host/path,dst=/mnt/path,readonly",
|
|
]
|
|
|
|
|
|
def test_format_container_mount_keeps_volume_syntax_for_apple_container():
|
|
args = _format_container_mount("container", "/host/path", "/mnt/path", True)
|
|
|
|
assert args == [
|
|
"-v",
|
|
"/host/path:/mnt/path:ro",
|
|
]
|
|
|
|
|
|
def test_redact_container_command_for_log_redacts_env_values():
|
|
redacted = _redact_container_command_for_log(
|
|
[
|
|
"docker",
|
|
"run",
|
|
"-e",
|
|
"API_KEY=secret-value",
|
|
"--env=TOKEN=token-value",
|
|
"--name",
|
|
"sandbox",
|
|
"image",
|
|
]
|
|
)
|
|
|
|
assert "API_KEY=<redacted>" in redacted
|
|
assert "--env=TOKEN=<redacted>" in redacted
|
|
assert "secret-value" not in " ".join(redacted)
|
|
assert "token-value" not in " ".join(redacted)
|
|
|
|
|
|
def test_redact_container_command_for_log_keeps_inherited_env_names():
|
|
redacted = _redact_container_command_for_log(
|
|
[
|
|
"docker",
|
|
"run",
|
|
"-e",
|
|
"API_KEY",
|
|
"--env=TOKEN",
|
|
"--name",
|
|
"sandbox",
|
|
"image",
|
|
]
|
|
)
|
|
|
|
assert redacted == [
|
|
"docker",
|
|
"run",
|
|
"-e",
|
|
"API_KEY",
|
|
"--env=TOKEN",
|
|
"--name",
|
|
"sandbox",
|
|
"image",
|
|
]
|
|
|
|
|
|
def test_format_container_command_for_log_uses_windows_quoting(monkeypatch):
|
|
monkeypatch.setattr(os, "name", "nt")
|
|
|
|
command = _format_container_command_for_log(["docker", "run", "--name", "sandbox one", "image"])
|
|
|
|
assert command == 'docker run --name "sandbox one" image'
|
|
|
|
|
|
def test_start_container_logs_redacted_env_values(monkeypatch, caplog):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={"API_KEY": "secret-value", "NORMAL": "visible-value"},
|
|
)
|
|
monkeypatch.setattr(backend, "_runtime", "docker")
|
|
|
|
captured_cmd: list[str] = []
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
captured_cmd.extend(cmd)
|
|
return SimpleNamespace(stdout="container-id\n", stderr="", returncode=0)
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
with caplog.at_level(logging.INFO, logger="deerflow.community.aio_sandbox.local_backend"):
|
|
backend._start_container("sandbox-test", 18080)
|
|
|
|
joined_cmd = " ".join(captured_cmd)
|
|
assert "API_KEY=secret-value" in joined_cmd
|
|
assert "NORMAL=visible-value" in joined_cmd
|
|
|
|
log_output = "\n".join(record.getMessage() for record in caplog.records)
|
|
assert "API_KEY=<redacted>" in log_output
|
|
assert "NORMAL=<redacted>" in log_output
|
|
assert "secret-value" not in log_output
|
|
assert "visible-value" not in log_output
|
|
|
|
|
|
def _capture_start_container_command(monkeypatch, backend: LocalContainerBackend, runtime: str = "docker") -> list[str]:
|
|
monkeypatch.setattr(backend, "_runtime", runtime)
|
|
captured_cmd: list[str] = []
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
captured_cmd.extend(cmd)
|
|
return SimpleNamespace(stdout="container-id\n", stderr="", returncode=0)
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
backend._start_container("sandbox-test", 18080)
|
|
return captured_cmd
|
|
|
|
|
|
def test_resolve_docker_bind_host_defaults_loopback_for_localhost(monkeypatch):
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_HOST", raising=False)
|
|
|
|
assert _resolve_docker_bind_host() == "127.0.0.1"
|
|
|
|
|
|
def test_resolve_docker_bind_host_follows_host_gateway_mapping_for_dood(monkeypatch):
|
|
"""The bind follows what host.docker.internal actually resolves to."""
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "host.docker.internal")
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._resolve_sandbox_host_address",
|
|
lambda host: "192.168.64.1",
|
|
)
|
|
|
|
assert _resolve_docker_bind_host() == "192.168.64.1"
|
|
|
|
|
|
def test_resolve_docker_bind_host_brackets_ipv6_host_gateway(monkeypatch):
|
|
"""An IPv6 host-gateway mapping binds the bracketed IPv6 address."""
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "host.docker.internal")
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._resolve_sandbox_host_address",
|
|
lambda host: "[fd00::1]",
|
|
)
|
|
|
|
assert _resolve_docker_bind_host() == "[fd00::1]"
|
|
|
|
|
|
def test_resolve_docker_bind_host_brackets_bare_ipv6_override(monkeypatch):
|
|
"""A bare IPv6 literal in the override becomes a valid Docker publish host.
|
|
|
|
Docker's ``-p`` syntax requires bracketed IPv6 literals
|
|
(``[fd00::1]:port:8080``); operators writing the escape hatch naturally
|
|
give the bare address, so it must be normalized before use.
|
|
"""
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "fd00::1")
|
|
assert _resolve_docker_bind_host() == "[fd00::1]"
|
|
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "[fd00::1]")
|
|
assert _resolve_docker_bind_host() == "[fd00::1]"
|
|
|
|
# IPv4 literals pass through unchanged.
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "192.168.64.1")
|
|
assert _resolve_docker_bind_host() == "192.168.64.1"
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "0.0.0.0")
|
|
assert _resolve_docker_bind_host() == "0.0.0.0"
|
|
|
|
|
|
def test_resolve_docker_bind_host_resolves_hostname_override(monkeypatch):
|
|
"""-p requires an IP literal as the host part, so a hostname override
|
|
resolves to the address the daemon actually maps before use."""
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "host.docker.internal")
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._resolve_sandbox_host_address",
|
|
lambda host: "192.168.64.1" if host == "host.docker.internal" else None,
|
|
)
|
|
assert _resolve_docker_bind_host() == "192.168.64.1"
|
|
|
|
|
|
def test_resolve_docker_bind_host_rejects_unresolvable_hostname_override(monkeypatch):
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "not-a-resolvable-host.invalid")
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._resolve_sandbox_host_address",
|
|
lambda host: None,
|
|
)
|
|
with pytest.raises(RuntimeError, match="DEER_FLOW_SANDBOX_BIND_HOST"):
|
|
_resolve_docker_bind_host()
|
|
|
|
|
|
def test_resolve_docker_bind_host_uses_discovered_bridge_gateway_when_resolution_fails(monkeypatch):
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "host.docker.internal")
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._resolve_sandbox_host_address",
|
|
lambda host: None,
|
|
)
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._docker_bridge_gateway_ip",
|
|
lambda: "192.168.64.1",
|
|
)
|
|
|
|
assert _resolve_docker_bind_host() == "192.168.64.1"
|
|
|
|
|
|
def test_resolve_docker_bind_host_falls_back_to_static_bridge_gateway(monkeypatch):
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "host.docker.internal")
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._resolve_sandbox_host_address",
|
|
lambda host: None,
|
|
)
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._docker_bridge_gateway_ip",
|
|
lambda: None,
|
|
)
|
|
|
|
assert _resolve_docker_bind_host() == "172.17.0.1"
|
|
|
|
|
|
def test_resolve_docker_bind_host_uses_ipv6_loopback_for_ipv6_sandbox_host(monkeypatch):
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "[::1]")
|
|
|
|
assert _resolve_docker_bind_host() == "[::1]"
|
|
|
|
|
|
def test_resolve_docker_bind_host_logs_selected_bind_reason(caplog):
|
|
with caplog.at_level(logging.DEBUG, logger="deerflow.community.aio_sandbox.local_backend"):
|
|
assert _resolve_docker_bind_host(sandbox_host="localhost", bind_host="") == "127.0.0.1"
|
|
|
|
messages = "\n".join(record.getMessage() for record in caplog.records)
|
|
assert "Docker sandbox bind: 127.0.0.1 (loopback default)" in messages
|
|
|
|
|
|
def test_resolve_docker_bind_host_allows_explicit_override(monkeypatch):
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "localhost")
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "192.0.2.10")
|
|
|
|
assert _resolve_docker_bind_host() == "192.0.2.10"
|
|
|
|
|
|
def test_resolve_docker_bind_host_allows_restoring_legacy_broad_bind(monkeypatch):
|
|
"""DEER_FLOW_SANDBOX_BIND_HOST=0.0.0.0 restores the pre-hardening bind."""
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "host.docker.internal")
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "0.0.0.0")
|
|
|
|
assert _resolve_docker_bind_host() == "0.0.0.0"
|
|
|
|
|
|
def _clear_hardening_env(monkeypatch):
|
|
for var in (
|
|
"DEER_FLOW_SANDBOX_HOST",
|
|
"DEER_FLOW_SANDBOX_BIND_HOST",
|
|
"DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED",
|
|
"DEER_FLOW_SANDBOX_SECCOMP_PROFILE",
|
|
"DEER_FLOW_SANDBOX_MEMORY",
|
|
"DEER_FLOW_SANDBOX_CPUS",
|
|
"DEER_FLOW_SANDBOX_PIDS_LIMIT",
|
|
"DEER_FLOW_SANDBOX_CONTAINER_USER",
|
|
"DEER_FLOW_SANDBOX_NETWORK",
|
|
"DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS",
|
|
):
|
|
monkeypatch.delenv(var, raising=False)
|
|
|
|
|
|
def test_start_container_binds_local_docker_port_to_loopback_by_default(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_HOST", raising=False)
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert captured_cmd[captured_cmd.index("-p") + 1] == "127.0.0.1:18080:8080"
|
|
|
|
|
|
def test_start_container_brackets_bare_ipv6_bind_override(monkeypatch):
|
|
"""A bare IPv6 override reaches -p as a bracketed, valid publish host."""
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "host.docker.internal")
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "fd00::1")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert captured_cmd[captured_cmd.index("-p") + 1] == "[fd00::1]:18080:8080"
|
|
|
|
|
|
def test_start_container_binds_dood_port_to_bridge_gateway(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "host.docker.internal")
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
monkeypatch.setattr(
|
|
"deerflow.community.aio_sandbox.local_backend._docker_bridge_gateway_ip",
|
|
lambda: "172.17.0.1",
|
|
)
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert captured_cmd[captured_cmd.index("-p") + 1] == "172.17.0.1:18080:8080"
|
|
|
|
|
|
def test_start_container_binds_ipv6_sandbox_host_to_ipv6_loopback(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", "[::1]")
|
|
monkeypatch.delenv("DEER_FLOW_SANDBOX_BIND_HOST", raising=False)
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert captured_cmd[captured_cmd.index("-p") + 1] == "[::1]:18080:8080"
|
|
|
|
|
|
def test_start_container_keeps_apple_container_port_format(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "127.0.0.1")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend, runtime="container")
|
|
|
|
assert captured_cmd[captured_cmd.index("-p") + 1] == "18080:8080"
|
|
|
|
|
|
def test_start_container_hardens_docker_run_by_default(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert "--cap-drop=ALL" in captured_cmd
|
|
# The shipped image's entrypoint starts as root, creates the gem user,
|
|
# chowns /opt/jupyter, and drops to that user via su — CHOWN/SETUID/SETGID
|
|
# must survive the drop or the container exits before readiness. The root
|
|
# nginx master also writes gem-owned logs under /var/log/nginx for the
|
|
# container's lifetime, which needs DAC_OVERRIDE.
|
|
cap_adds = [arg.split("=", 1)[1] for arg in captured_cmd if arg.startswith("--cap-add=")]
|
|
assert cap_adds == ["CHOWN", "SETUID", "SETGID", "DAC_OVERRIDE"]
|
|
security_opts = [captured_cmd[i + 1] for i, arg in enumerate(captured_cmd) if arg == "--security-opt"]
|
|
assert "no-new-privileges" in security_opts
|
|
# The shipped AIO image needs seccomp=unconfined for its Chromium
|
|
# browser (upstream FAQ), so that option stays the default; the
|
|
# hardening that does not break the shipped image is kept.
|
|
assert "seccomp=unconfined" in security_opts
|
|
assert captured_cmd[captured_cmd.index("--memory") + 1] == "2g"
|
|
assert captured_cmd[captured_cmd.index("--cpus") + 1] == "2"
|
|
assert captured_cmd[captured_cmd.index("--pids-limit") + 1] == "512"
|
|
# Opt-in-only knobs stay absent unless explicitly configured.
|
|
assert "--user" not in captured_cmd
|
|
assert "--network" not in captured_cmd
|
|
|
|
|
|
def test_start_container_seccomp_can_opt_out_to_default_profile(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED", "0")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
security_opts = [captured_cmd[i + 1] for i, arg in enumerate(captured_cmd) if arg == "--security-opt"]
|
|
assert "seccomp=unconfined" not in security_opts
|
|
# The opt-out must select the built-in profile explicitly: omitting the
|
|
# option would inherit the daemon's (possibly unconfined) default.
|
|
assert "seccomp=builtin" in security_opts
|
|
assert "no-new-privileges" in security_opts
|
|
|
|
|
|
def test_start_container_seccomp_profile_env_selects_custom_profile(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_SECCOMP_PROFILE", "/etc/docker/chromium-seccomp.json")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
security_opts = [captured_cmd[i + 1] for i, arg in enumerate(captured_cmd) if arg == "--security-opt"]
|
|
assert "seccomp=/etc/docker/chromium-seccomp.json" in security_opts
|
|
assert "seccomp=unconfined" not in security_opts
|
|
assert "no-new-privileges" in security_opts
|
|
|
|
|
|
def test_resolve_sandbox_host_address_formats_and_filters(monkeypatch):
|
|
import socket as socket_module
|
|
|
|
def fake_getaddrinfo(host, port):
|
|
if host == "v4host":
|
|
return [(socket_module.AF_INET, None, None, "", ("203.0.113.7", 0))]
|
|
if host == "v6host":
|
|
return [(socket_module.AF_INET6, None, None, "", ("fd00::1%eth0", 0, 0, 0))]
|
|
if host == "wildcard":
|
|
return [(socket_module.AF_INET, None, None, "", ("0.0.0.0", 0))]
|
|
raise OSError("no such host")
|
|
|
|
monkeypatch.setattr("deerflow.community.aio_sandbox.local_backend.socket.getaddrinfo", fake_getaddrinfo)
|
|
|
|
from deerflow.community.aio_sandbox.local_backend import _resolve_sandbox_host_address
|
|
|
|
assert _resolve_sandbox_host_address("v4host") == "203.0.113.7"
|
|
# zone ids are stripped and IPv6 is bracketed for docker -p syntax
|
|
assert _resolve_sandbox_host_address("v6host") == "[fd00::1]"
|
|
# wildcard resolutions are not bindable choices
|
|
assert _resolve_sandbox_host_address("wildcard") is None
|
|
assert _resolve_sandbox_host_address("unknown.invalid") is None
|
|
|
|
|
|
def test_start_container_resource_limits_env_override(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_MEMORY", "4g")
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_CPUS", "4")
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_PIDS_LIMIT", "1024")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert captured_cmd[captured_cmd.index("--memory") + 1] == "4g"
|
|
assert captured_cmd[captured_cmd.index("--cpus") + 1] == "4"
|
|
assert captured_cmd[captured_cmd.index("--pids-limit") + 1] == "1024"
|
|
|
|
|
|
def test_start_container_resource_limits_can_be_disabled(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_MEMORY", "0")
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_CPUS", "none")
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_PIDS_LIMIT", "0")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert "--memory" not in captured_cmd
|
|
assert "--cpus" not in captured_cmd
|
|
assert "--pids-limit" not in captured_cmd
|
|
|
|
|
|
def test_start_container_passes_through_user_and_network(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_CONTAINER_USER", "1000:1000")
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "deer-flow-sandbox-egress")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert captured_cmd[captured_cmd.index("--user") + 1] == "1000:1000"
|
|
assert captured_cmd[captured_cmd.index("--network") + 1] == "deer-flow-sandbox-egress"
|
|
|
|
|
|
def test_start_container_rejects_host_networking(monkeypatch):
|
|
"""host mode discards -p/--publish, voiding the hardened bind and
|
|
re-exposing the unauthenticated exec API on the host's interfaces."""
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "host")
|
|
|
|
with pytest.raises(RuntimeError, match="DEER_FLOW_SANDBOX_NETWORK"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_rejects_shared_container_network_namespace(monkeypatch):
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "container:gateway")
|
|
|
|
with pytest.raises(RuntimeError, match="DEER_FLOW_SANDBOX_NETWORK"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_rejects_none_network(monkeypatch):
|
|
"""The none driver leaves the container loopback-only, so the published
|
|
sandbox API port cannot receive traffic: readiness would time out and
|
|
every acquisition would fail. Fail fast at start-up instead."""
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "none")
|
|
|
|
with pytest.raises(RuntimeError, match="loopback-only"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_does_not_add_docker_hardening_to_apple_container(monkeypatch):
|
|
"""Apple Container's CLI does not support the Docker hardening flags."""
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_BIND_HOST", "127.0.0.1")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend, runtime="container")
|
|
|
|
assert "--cap-drop=ALL" not in captured_cmd
|
|
assert "--security-opt" not in captured_cmd
|
|
assert "--memory" not in captured_cmd
|
|
assert "--cpus" not in captured_cmd
|
|
assert "--pids-limit" not in captured_cmd
|
|
|
|
|
|
def _backend_for_inspect_tests() -> LocalContainerBackend:
|
|
backend = LocalContainerBackend(
|
|
image="sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
backend._runtime = "docker"
|
|
return backend
|
|
|
|
|
|
def test_is_container_running_false_when_container_missing(monkeypatch):
|
|
backend = _backend_for_inspect_tests()
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
return SimpleNamespace(stdout="", stderr="Error: No such object: sandbox-missing", returncode=1)
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
assert backend._is_container_running("sandbox-missing") is False
|
|
|
|
|
|
def test_is_container_running_raises_on_runtime_error(monkeypatch):
|
|
backend = _backend_for_inspect_tests()
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
return SimpleNamespace(stdout="", stderr="Cannot connect to the Docker daemon", returncode=1)
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
with pytest.raises(RuntimeError, match="Failed to inspect container sandbox-busy"):
|
|
backend._is_container_running("sandbox-busy")
|
|
|
|
|
|
def test_is_container_running_raises_on_timeout(monkeypatch):
|
|
backend = _backend_for_inspect_tests()
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
raise subprocess.TimeoutExpired(cmd=cmd, timeout=kwargs["timeout"])
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
with pytest.raises(RuntimeError, match="Timed out checking container sandbox-timeout"):
|
|
backend._is_container_running("sandbox-timeout")
|
|
|
|
|
|
def test_discover_returns_none_when_runtime_check_fails(monkeypatch):
|
|
"""A transient daemon error during discovery must fall through to create, not fail acquire."""
|
|
backend = _backend_for_inspect_tests()
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
return SimpleNamespace(stdout="", stderr="Cannot connect to the Docker daemon", returncode=1)
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
assert backend.discover("sandbox-blip") is None
|
|
|
|
|
|
def test_discover_returns_none_when_runtime_check_times_out(monkeypatch):
|
|
"""An inspect timeout during discovery must not propagate out of discover()."""
|
|
backend = _backend_for_inspect_tests()
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
raise subprocess.TimeoutExpired(cmd=cmd, timeout=kwargs["timeout"])
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
assert backend.discover("sandbox-timeout") is None
|
|
|
|
|
|
def test_is_container_running_false_on_apple_container_not_found(monkeypatch):
|
|
"""Apple Container's generic "not found" is trusted when it names the container."""
|
|
backend = _backend_for_inspect_tests()
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
return SimpleNamespace(stdout="", stderr='Error: not found: "sandbox-apple"', returncode=1)
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
assert backend._is_container_running("sandbox-apple") is False
|
|
|
|
|
|
def test_is_container_running_raises_on_unrelated_not_found_error(monkeypatch):
|
|
"""Transient errors whose text contains "not found" must not be misread as a dead container."""
|
|
backend = _backend_for_inspect_tests()
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
return SimpleNamespace(stdout="", stderr="Error: credential helper not found in $PATH", returncode=1)
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
with pytest.raises(RuntimeError, match="Failed to inspect container sandbox-busy"):
|
|
backend._is_container_running("sandbox-busy")
|
|
|
|
|
|
def test_stop_container_passes_a_timeout(monkeypatch):
|
|
"""An unbounded `stop` can outlive the teardown lease that guards it.
|
|
|
|
The `del:` marker keeps a peer from re-acquiring the container during the
|
|
stop, but a lease can lapse (a store outage longer than the TTL) while a
|
|
wedged daemon leaves `docker stop` blocked forever — and the stop then lands
|
|
on a container the peer has since been handed. Bounding the call caps that
|
|
exposure independently of the ownership layer.
|
|
"""
|
|
backend = _backend_for_inspect_tests()
|
|
seen = {}
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
seen.update(kwargs)
|
|
return SimpleNamespace(stdout="", stderr="", returncode=0)
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
backend._stop_container("sandbox-slow")
|
|
|
|
assert seen.get("timeout") == backend._STOP_TIMEOUT_SECONDS
|
|
|
|
|
|
def test_stop_container_propagates_a_timeout_instead_of_reporting_success(monkeypatch):
|
|
"""A timed-out stop must not be swallowed like a failed one.
|
|
|
|
`CalledProcessError` means the runtime answered "I could not stop it"; a
|
|
timeout means we do not know, and the container is probably still running.
|
|
Returning normally would let `_destroy_warm_entry` report a clean stop and
|
|
drop the warm entry, leaking a running container nothing tracks.
|
|
"""
|
|
backend = _backend_for_inspect_tests()
|
|
|
|
def fake_run(cmd, **kwargs):
|
|
raise subprocess.TimeoutExpired(cmd=cmd, timeout=kwargs["timeout"])
|
|
|
|
monkeypatch.setattr("subprocess.run", fake_run)
|
|
|
|
with pytest.raises(subprocess.TimeoutExpired):
|
|
backend._stop_container("sandbox-wedged")
|
|
|
|
|
|
# ── Extended network syntax and IPv6 host normalization ──────────────────────
|
|
|
|
|
|
def test_start_container_rejects_extended_network_syntax_host(monkeypatch):
|
|
"""name=host attaches the host network namespace exactly like `host`;
|
|
the raw string must not dodge the rejection."""
|
|
backend = _backend_for_inspect_tests()
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "name=host")
|
|
|
|
with pytest.raises(RuntimeError, match="DEER_FLOW_SANDBOX_NETWORK"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_rejects_extended_network_syntax_none(monkeypatch):
|
|
backend = _backend_for_inspect_tests()
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "name=none")
|
|
|
|
with pytest.raises(RuntimeError, match="loopback-only"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_rejects_extended_network_syntax_container(monkeypatch):
|
|
backend = _backend_for_inspect_tests()
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "name=container:gateway")
|
|
|
|
with pytest.raises(RuntimeError, match="DEER_FLOW_SANDBOX_NETWORK"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_passes_extended_network_syntax_for_custom_networks(monkeypatch):
|
|
"""The legit name=<custom-net> long form (and network IDs) keep working."""
|
|
backend = _backend_for_inspect_tests()
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "name=deer-flow-sandbox-egress")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert captured_cmd[captured_cmd.index("--network") + 1] == "name=deer-flow-sandbox-egress"
|
|
|
|
|
|
@pytest.mark.parametrize("sandbox_host", ["fd00::1", "[fd00::1]"])
|
|
def test_discover_brackets_ipv6_sandbox_host_for_url(monkeypatch, sandbox_host):
|
|
"""Both IPv6 input forms must yield the same bracketed URL authority:
|
|
the bare form used to produce the malformed http://fd00::1:<port>."""
|
|
backend = _backend_for_inspect_tests()
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", sandbox_host)
|
|
monkeypatch.setattr(backend, "_is_container_running", lambda name: True)
|
|
monkeypatch.setattr(backend, "_get_container_port", lambda name: 18081)
|
|
|
|
seen_urls = []
|
|
|
|
def fake_ready(url, timeout):
|
|
seen_urls.append(url)
|
|
return True
|
|
|
|
monkeypatch.setattr("deerflow.community.aio_sandbox.local_backend.wait_for_sandbox_ready", fake_ready)
|
|
|
|
info = backend.discover("sbx-ipv6")
|
|
|
|
assert info.sandbox_url == "http://[fd00::1]:18081"
|
|
assert seen_urls == ["http://[fd00::1]:18081"]
|
|
|
|
|
|
@pytest.mark.parametrize("sandbox_host", ["fd00::1", "[fd00::1]"])
|
|
def test_create_brackets_ipv6_sandbox_host_for_url(monkeypatch, sandbox_host):
|
|
backend = _backend_for_inspect_tests()
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_HOST", sandbox_host)
|
|
monkeypatch.setattr(backend, "_start_container", lambda name, port, mounts=None: "container-id")
|
|
monkeypatch.setattr("deerflow.community.aio_sandbox.local_backend.get_free_port", lambda start_port=None: 18082)
|
|
|
|
info = backend.create(thread_id="t", sandbox_id="sbx-ipv6")
|
|
|
|
assert info.sandbox_url == "http://[fd00::1]:18082"
|
|
|
|
|
|
def test_resolve_sandbox_host_address_accepts_bracketed_ipv6(monkeypatch):
|
|
"""The bracketed form must resolve (unbracketed for getaddrinfo) instead
|
|
of failing through to the IPv4 bridge fallback."""
|
|
from deerflow.community.aio_sandbox.local_backend import _resolve_sandbox_host_address
|
|
|
|
infos = [(socket.AF_INET6, socket.SOCK_STREAM, 0, "", ("fd00::1", 0, 0, 0))]
|
|
|
|
def fake_getaddrinfo(host, port, *args, **kwargs):
|
|
assert host == "fd00::1", f"getaddrinfo must receive the unbracketed form, got {host!r}"
|
|
return infos
|
|
|
|
monkeypatch.setattr("deerflow.community.aio_sandbox.local_backend.socket.getaddrinfo", fake_getaddrinfo)
|
|
|
|
assert _resolve_sandbox_host_address("[fd00::1]") == "[fd00::1]"
|
|
|
|
|
|
# ── Long-syntax fields in any position (Docker opts/network.go semantics) ────
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"network",
|
|
[
|
|
"name=host,gw-priority=0",
|
|
"gw-priority=0,name=host",
|
|
"name=host,alias=sbx",
|
|
],
|
|
)
|
|
def test_start_container_rejects_host_with_additional_long_syntax_fields(monkeypatch, network):
|
|
"""Docker accepts comma-separated fields in any order; both orderings
|
|
select the host network and must not dodge the rejection."""
|
|
backend = _backend_for_inspect_tests()
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", network)
|
|
|
|
with pytest.raises(RuntimeError, match="DEER_FLOW_SANDBOX_NETWORK"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_rejects_none_with_additional_long_syntax_fields(monkeypatch):
|
|
backend = _backend_for_inspect_tests()
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "gw-priority=0,name=none")
|
|
|
|
with pytest.raises(RuntimeError, match="loopback-only"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_rejects_container_mode_with_additional_long_syntax_fields(monkeypatch):
|
|
backend = _backend_for_inspect_tests()
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "name=container:gateway,gw-priority=0")
|
|
|
|
with pytest.raises(RuntimeError, match="DEER_FLOW_SANDBOX_NETWORK"):
|
|
_capture_start_container_command(monkeypatch, backend)
|
|
|
|
|
|
def test_start_container_passes_long_syntax_custom_network_with_fields(monkeypatch):
|
|
"""A legit long-syntax value with extra fields keeps passing through verbatim."""
|
|
backend = _backend_for_inspect_tests()
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_NETWORK", "name=egressnet,gw-priority=1")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert captured_cmd[captured_cmd.index("--network") + 1] == "name=egressnet,gw-priority=1"
|
|
|
|
|
|
def test_effective_network_target_last_name_field_wins():
|
|
"""Docker's parser lets a later name= field overwrite an earlier one."""
|
|
from deerflow.community.aio_sandbox.local_backend import _effective_docker_network_target as target
|
|
|
|
assert target("name=host,name=egressnet") == "egressnet"
|
|
assert target("name=egressnet,name=host") == "host"
|
|
assert target("gw-priority=0") == "gw-priority=0" # no name= field: Docker errors itself
|
|
assert target("bridge") == "bridge"
|
|
assert target("1f2a" * 16) == "1f2a" * 16 # network ID passes through
|
|
|
|
|
|
# ── Real-image startup smoke test (docker-gated) ─────────────────────────────
|
|
# Keep in sync with aio_sandbox_provider.DEFAULT_IMAGE.
|
|
_DEFAULT_AIO_IMAGE = "enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest"
|
|
|
|
|
|
def _docker_daemon_available() -> bool:
|
|
try:
|
|
subprocess.run(["docker", "info"], capture_output=True, timeout=30, check=True)
|
|
return True
|
|
except Exception:
|
|
return False
|
|
|
|
|
|
# `live`: pulls and runs a mutable external image, so the default offline
|
|
# suite (`make test` = `-m "not live"`) never touches the network. The
|
|
# daemon probe happens inside the test body — never at collection time.
|
|
@pytest.mark.live
|
|
def test_default_image_starts_under_hardened_capabilities(monkeypatch):
|
|
"""Real smoke test against the shipped default image — no subprocess mock.
|
|
|
|
The image's entrypoint (/opt/gem/run.sh) starts as root, creates the gem
|
|
account at runtime, chown -R's /opt/jupyter, and drops to that user via
|
|
su before starting the services. Under the default hardened argv
|
|
(--cap-drop=ALL + no-new-privileges) that initialization needs
|
|
CHOWN/SETUID/SETGID to be re-added, or the container exits (set -e)
|
|
before the readiness endpoint exists. Reaching readiness through the
|
|
real docker run proves the whole startup chain survives the hardening.
|
|
"""
|
|
from deerflow.community.aio_sandbox.backend import SANDBOX_LOCAL_PROVIDER_READY_TIMEOUT
|
|
from deerflow.community.aio_sandbox.local_backend import wait_for_sandbox_ready
|
|
|
|
if not _docker_daemon_available():
|
|
pytest.skip("requires a running Docker daemon")
|
|
|
|
backend = LocalContainerBackend(
|
|
# Pin via this override when wiring a dedicated integration job, so
|
|
# the run does not depend on a mutable :latest tag.
|
|
image=os.environ.get("DEER_FLOW_SANDBOX_SMOKE_IMAGE", _DEFAULT_AIO_IMAGE),
|
|
base_port=18210,
|
|
container_prefix="sandbox-smoke",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
|
|
info = backend.create(thread_id="smoke", sandbox_id="caps-smoke")
|
|
try:
|
|
# The production deadline, single-sourced: the sync and async
|
|
# provider paths destroy the container after exactly this budget, so
|
|
# a longer one here could pass while every real acquisition fails.
|
|
# (create() completes docker run — including the image pull — before
|
|
# this timer starts, so the pull is not part of the budget.)
|
|
ready = wait_for_sandbox_ready(info.sandbox_url, timeout=SANDBOX_LOCAL_PROVIDER_READY_TIMEOUT)
|
|
if not ready:
|
|
# Fail diagnosably: the entrypoint's own log tells us whether the
|
|
# capability set is still incomplete (chown/useradd/su errors) or
|
|
# the services are merely slow.
|
|
logs = subprocess.run(
|
|
["docker", "logs", info.container_name],
|
|
capture_output=True,
|
|
text=True,
|
|
timeout=30,
|
|
)
|
|
# supervisord only reports exit codes in the container log; the
|
|
# failing program's own stderr goes to files inside the container.
|
|
# Pull the usual suspects so the failure is actionable in CI.
|
|
prog_logs = subprocess.run(
|
|
[
|
|
"docker",
|
|
"exec",
|
|
info.container_name,
|
|
"sh",
|
|
"-c",
|
|
"cat /var/log/supervisor/* 2>/dev/null | tail -n 40; echo '--- nginx -t ---'; nginx -t 2>&1; echo '--- nginx error.log ---'; tail -n 20 /var/log/nginx/error.log 2>/dev/null",
|
|
],
|
|
capture_output=True,
|
|
text=True,
|
|
timeout=30,
|
|
)
|
|
tail = "\n".join((logs.stdout + logs.stderr).splitlines()[-40:]) + "\n" + (prog_logs.stdout or "")
|
|
pytest.fail(f"default image never became ready under the hardened capabilities: {info.sandbox_url}\n--- last 40 container log lines ---\n{tail}")
|
|
assert backend.is_alive(info)
|
|
finally:
|
|
backend.destroy(info)
|
|
|
|
|
|
def test_start_container_preinitialized_image_can_drop_startup_caps(monkeypatch):
|
|
"""A custom, pre-initialized non-root image never runs the root handoff,
|
|
so CHOWN/SETUID/SETGID must not stay available for the container's
|
|
lifetime (chown on bind mounts, UID/GID impersonation). Opting out with
|
|
DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS=0 drops every capability."""
|
|
backend = LocalContainerBackend(
|
|
image="my-preinitialized-sandbox:latest",
|
|
base_port=8080,
|
|
container_prefix="sandbox",
|
|
config_mounts=[],
|
|
environment={},
|
|
)
|
|
_clear_hardening_env(monkeypatch)
|
|
monkeypatch.setenv("DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS", "0")
|
|
|
|
captured_cmd = _capture_start_container_command(monkeypatch, backend)
|
|
|
|
assert "--cap-drop=ALL" in captured_cmd
|
|
assert not [arg for arg in captured_cmd if arg.startswith("--cap-add=")]
|
|
security_opts = [captured_cmd[i + 1] for i, arg in enumerate(captured_cmd) if arg == "--security-opt"]
|
|
assert "no-new-privileges" in security_opts
|