fix(docker): make dev provisioner opt-in via compose profile (#6948)

* fix(docker): make dev provisioner opt-in via compose profile

Gate the Kubernetes provisioner behind a Compose profile and have the
wrapper enable it only when config selects provisioner sandbox mode.
This lets the default Docker Compose stack start without kubeconfig.

Fixes #3232

* test(docker): render every compose profile in the env-file regression test

main added backend/tests/test_compose_default_bind_host.py, whose _render helper asserted that the dev provisioner still loads ../.env. `docker compose config` without a profile omits profile-gated services, so that assertion failed once the provisioner became opt-in.

Pass `--profile *` from the helper: the test is about an existing .env reaching every service, so it has to render the services a profile gates rather than only the ones `up` starts by default.

* docs(docker): restore script mode and localize direct Compose guidance

The dev provisioner is now opt-in, but the earlier edit also flipped scripts/docker.sh from 100755 to 100644 even though README.md still documents direct `./scripts/docker.sh` invocation. Restore the executable bit.

Also add the direct-Compose guidance to the translated READMEs so readers of README_zh.md, README_ja.md, and README_fr.md learn that `--profile provisioner` selects the Kubernetes mode and the default stack does not need Kubernetes.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

CONTRIBUTING.md

@@ -51,11 +51,14 @@ Docker provides a consistent, isolated environment with all dependencies pre-con
    version, creates the missing `.env` files, and exports `DEER_FLOW_ROOT`.
 
    If you do run Compose directly, run it **from the repository root** and set
-   `DEER_FLOW_ROOT` to the absolute path of your checkout. Compose interpolates
-   that variable into host-side paths (`DEER_FLOW_HOST_BASE_DIR`,
-   `THREADS_HOST_PATH`) that the AIO and provisioner sandbox modes bind-mount;
-   leaving it unset renders them as `/backend/.deer-flow`, so those mounts
-   silently miss your checkout instead of failing:
+   `DEER_FLOW_ROOT` to the absolute path of your checkout. The default command
+   starts the local/AIO development stack without Kubernetes; add
+   `--profile provisioner` only when you need provisioner/Kubernetes sandbox
+   mode. Compose interpolates `DEER_FLOW_ROOT` into host-side paths
+   (`DEER_FLOW_HOST_BASE_DIR`, `THREADS_HOST_PATH`) that the AIO and provisioner
+   sandbox modes bind-mount; leaving it unset renders them as
+   `/backend/.deer-flow`, so those mounts silently miss your checkout instead of
+   failing:
 
    ```bash
    # macOS / Linux

README.md

@@ -494,7 +494,7 @@ make docker-start   # Start services (auto-detects sandbox mode from config.yaml
 make docker-logs    # View logs
 ```
 
-`make docker-start` starts `provisioner` only when `config.yaml` uses provisioner mode (`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider` with `provisioner_url`).
+`make docker-start` starts `provisioner` only when `config.yaml` uses provisioner mode (`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider` with `provisioner_url`). Direct Compose users can add `--profile provisioner` to the dev command for the same mode; the default stack does not require Kubernetes.
 
 Docker builds use the upstream `uv` registry by default. The gateway image installs with `uv sync --locked`, and `uv.lock` pins `https://pypi.org/simple`, so `UV_INDEX_URL` has to keep that value: exporting a mirror such as `https://pypi.tuna.tsinghua.edu.cn/simple` fails the build with ``The lockfile at `uv.lock` needs to be updated, but `--locked` was provided`` (reproduce in an already-synced environment with `cd backend && UV_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple uv sync --locked --check`; if the environment itself is stale, `--check` reports that instead for either index, so use `uv lock --check` to see it without syncing). Installing from a mirror requires regenerating the lock or dropping `--locked`, and both change what the image installs, so keep the default index for a checkout you intend to contribute back. `NPM_REGISTRY=https://registry.npmmirror.com` is independent of this: it applies to the frontend image's `pnpm` install and to the gateway image's Lark CLI install (`@larksuite/cli`, which `backend/Dockerfile` installs with npm).
 

README_fr.md

@@ -231,7 +231,7 @@ make docker-init    # Pull sandbox image (only once or when image updates)
 make docker-start   # Start services (auto-detects sandbox mode from config.yaml)
 ```
 
-`make docker-start` ne lance `provisioner` que si `config.yaml` utilise le mode provisioner (`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider` avec `provisioner_url`).
+`make docker-start` ne lance `provisioner` que si `config.yaml` utilise le mode provisioner (`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider` avec `provisioner_url`). Les utilisateurs qui lancent Compose directement peuvent ajouter `--profile provisioner` à la commande dev pour le même mode ; la stack par défaut ne nécessite pas Kubernetes.
 Les processus backend récupèrent automatiquement les changements dans `config.yaml` au prochain accès à la configuration, donc les mises à jour de métadonnées des modèles ne nécessitent pas de redémarrage manuel en développement.
 
 > [!TIP]

README_ja.md

@@ -222,7 +222,7 @@ make docker-init    # サンドボックスイメージをプル（初回また
 make docker-start   # サービスを開始（config.yamlからサンドボックスモードを自動検出）
 ```
 
-`make docker-start`は、`config.yaml`がプロビジョナーモード（`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider`と`provisioner_url`）を使用している場合にのみ`provisioner`を起動します。
+`make docker-start`は、`config.yaml`がプロビジョナーモード（`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider`と`provisioner_url`）を使用している場合にのみ`provisioner`を起動します。Compose を直接使う場合は、同じモードにするため dev コマンドに `--profile provisioner` を追加できます。デフォルト構成に Kubernetes は不要です。
 
 **本番環境**（ローカルでイメージをビルドし、ランタイム設定とデータをマウント）：
 

README_zh.md

@@ -305,7 +305,7 @@ make docker-init    # 拉取 sandbox 镜像（首次运行或镜像更新时执
 make docker-start   # 启动服务（会根据 config.yaml 自动判断 sandbox 模式）
 ```
 
-如果 `config.yaml` 使用的是 provisioner 模式（`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider` 且配置了 `provisioner_url`），`make docker-start` 才会启动 `provisioner`。
+如果 `config.yaml` 使用的是 provisioner 模式（`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider` 且配置了 `provisioner_url`），`make docker-start` 才会启动 `provisioner`。直接使用 Docker Compose 的用户可以在 dev 命令中加入 `--profile provisioner` 来启用相同模式；默认栈不需要 Kubernetes。
 
 **生产模式**（本地构建镜像，并挂载运行期配置与数据）：
 

backend/tests/test_compose_default_bind_host.py

@@ -167,7 +167,11 @@ def _render(tmp_path: Path, variant: str) -> subprocess.CompletedProcess[str]:
         DEER_FLOW_INTERNAL_AUTH_TOKEN="test-token",
     )
     return subprocess.run(
-        [DOCKER, "compose", "-p", "deer-flow-env-file-test", "-f", str(docker_dir / COMPOSE_PATHS[variant].name), "config", "--format", "json"],
+        # Enable every profile so profile-gated services (the dev provisioner)
+        # still render: `config` omits them otherwise, and these assertions are
+        # about a present .env reaching each service, not about what `up`
+        # starts by default.
+        [DOCKER, "compose", "--profile", "*", "-p", "deer-flow-env-file-test", "-f", str(docker_dir / COMPOSE_PATHS[variant].name), "config", "--format", "json"],
         env=env,
         capture_output=True,
         text=True,

backend/tests/test_docker_sandbox_mode_detection.py

@@ -11,6 +11,7 @@ import tempfile
 from pathlib import Path
 
 import pytest
+import yaml
 from support.shell import find_script_bash
 
 REPO_ROOT = Path(__file__).resolve().parents[2]
@@ -292,6 +293,14 @@ sandbox:
     assert _detect_mode_with_config(config) == "local"
 
 
+def test_dev_compose_keeps_provisioner_behind_opt_in_profile():
+    """A direct `docker compose up` must not start the Kubernetes provisioner."""
+    compose_path = REPO_ROOT / "docker" / "docker-compose-dev.yaml"
+    compose = yaml.safe_load(compose_path.read_text(encoding="utf-8"))
+
+    assert compose["services"]["provisioner"]["profiles"] == ["provisioner"]
+
+
 def _seed_compose_file(tmp_root: Path) -> None:
     """Give require_compose_file the file it validates."""
     (tmp_root / "docker-compose-dev.yaml").write_text("services: {}\n", encoding="utf-8")
@@ -649,6 +658,60 @@ start
         assert "Docker socket not found" in result.stdout
 
 
+@pytest.mark.parametrize(
+    ("sandbox_provider", "expected_profile"),
+    [
+        ("deerflow.sandbox.local:LocalSandboxProvider", False),
+        (
+            "deerflow.community.aio_sandbox:AioSandboxProvider\n  provisioner_url: http://provisioner:8002",
+            True,
+        ),
+    ],
+)
+def test_start_toggles_provisioner_profile(sandbox_provider, expected_profile):
+    """make docker-start enables the profile only for provisioner mode."""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        tmp_root = Path(tmpdir)
+        _seed_compose_file(tmp_root)
+        _seed_env_examples(tmp_root)
+        (tmp_root / "config.yaml").write_text(
+            f"sandbox:\n  use: {sandbox_provider}\n",
+            encoding="utf-8",
+        )
+        marker = tmp_root / "start_invoke.txt"
+
+        _run_docker_sh(
+            tmp_root,
+            f"""
+COMPOSE_CMD=capture_compose
+capture_compose() {{ printf '%s\\n' "$*" > '{marker}'; }}
+start
+""",
+        )
+
+        recorded = marker.read_text(encoding="utf-8")
+        assert ("--profile provisioner" in recorded) is expected_profile
+
+
+def test_stop_includes_profile_gated_provisioner():
+    """make docker-stop must remove a provisioner started with the profile."""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        tmp_root = Path(tmpdir)
+        _seed_compose_file(tmp_root)
+        marker = tmp_root / "stop_invoke.txt"
+
+        _run_docker_sh(
+            tmp_root,
+            f"""
+COMPOSE_CMD=capture_compose
+capture_compose() {{ printf '%s\\n' "$*" > '{marker}'; }}
+stop
+""",
+        )
+
+        assert "--profile provisioner down" in marker.read_text(encoding="utf-8")
+
+
 def test_aio_dood_socket_preflight_rejects_missing_custom_socket_on_windows():
     """Windows Git Bash must fail if a custom non-existent socket path is specified."""
     with tempfile.TemporaryDirectory() as tmpdir:

docker/docker-compose-dev.yaml

@@ -17,8 +17,10 @@
 # Running Compose yourself works too, but export DEER_FLOW_ROOT as the absolute
 # checkout path first (see CONTRIBUTING.md for the Windows form). Unset, the
 # DEER_FLOW_HOST_BASE_DIR and THREADS_HOST_PATH values below render as
-# /backend/.deer-flow, so the provisioner bind-mounts the wrong host paths:
+# /backend/.deer-flow, so the provisioner bind-mounts the wrong host paths.
+# The default stack does not need Kubernetes:
 #   DEER_FLOW_ROOT="$PWD" docker compose -f docker/docker-compose-dev.yaml up --build
+# Add `--profile provisioner` when using provisioner/Kubernetes sandbox mode.
 # The -f path resolves against your shell's directory, not against this file:
 # from the repository root use docker/docker-compose-dev.yaml; from inside
 # docker/ use the bare docker-compose-dev.yaml, as make docker-start does.
@@ -57,6 +59,8 @@ services:
   # cluster via the K8s API.
   # Backend accesses sandboxes directly via host.docker.internal:{NodePort}.
   provisioner:
+    profiles:
+      - provisioner
     build:
       context: ./provisioner
       dockerfile: Dockerfile

scripts/docker.sh

@@ -295,6 +295,7 @@ start() {
     services="redis frontend gateway nginx"
     if [ "$sandbox_mode" = "provisioner" ]; then
         services="redis frontend gateway provisioner nginx"
+        COMPOSE_CMD="$COMPOSE_CMD --profile provisioner"
     fi
 
     # Only aio mode (AioSandboxProvider without provisioner_url) needs the host
@@ -438,6 +439,14 @@ logs() {
 
     compose_preflight
 
+    # The provisioner is profile-gated in docker-compose-dev.yaml. Enable the
+    # profile for dev logs that need to include it; --prod uses a different file.
+    if [ "$is_prod" = 0 ]; then
+        if [ "$service" = "provisioner" ] || { [ -z "$service" ] && [ "$(detect_sandbox_mode)" = "provisioner" ]; }; then
+            COMPOSE_CMD="$COMPOSE_CMD --profile provisioner"
+        fi
+    fi
+
     # Append --env-file only after compose_preflight(): its Compose detection
     # may rebuild COMPOSE_CMD, which would drop anything appended before it.
     if [ "$is_prod" = 1 ] && [ -f "$PROJECT_ROOT/.env" ]; then
@@ -459,7 +468,8 @@ logs() {
 stop() {
     compose_preflight
     echo "Stopping Docker development services..."
-    cd "$DOCKER_DIR" && $COMPOSE_CMD down
+    # Include profile-gated services so a running provisioner is removed too.
+    cd "$DOCKER_DIR" && $COMPOSE_CMD --profile provisioner down
     echo "Cleaning up sandbox containers..."
     bash "$SCRIPT_DIR/cleanup-containers.sh" deer-flow-sandbox 2>/dev/null || true
     echo -e "${GREEN}✓ Docker services stopped${NC}"
@@ -467,7 +477,13 @@ stop() {
 
 # Restart Docker development environment
 restart() {
+    local sandbox_mode
+
     compose_preflight
+    sandbox_mode="$(detect_sandbox_mode)"
+    if [ "$sandbox_mode" = "provisioner" ]; then
+        COMPOSE_CMD="$COMPOSE_CMD --profile provisioner"
+    fi
     echo "========================================"
     echo "  Restarting DeerFlow Docker Services"
     echo "========================================"