From c5c0dfa54ef75a4b359b9700808d5523559800fc Mon Sep 17 00:00:00 2001 From: Nan Gao Date: Wed, 7 Oct 2026 07:10:05 +0800 Subject: [PATCH] fix(helm): wire PROVISIONER_API_KEY through the chart's app Secret so a default install can create sandboxes (#6365) * fix(helm): wire PROVISIONER_API_KEY through the chart's app Secret The chart enables the sandbox provisioner by default and points config.sandbox.provisioner_url at it, but nothing rendered PROVISIONER_API_KEY. Since the provisioner started requiring the key (#4116) its verify_api_key middleware answers 401 to every /api/* request while the key is empty or mismatched, so a default install could not create a single sandbox. docker-compose was unaffected: it reads the key from .env. - secret-app.yaml generates the key on first install and preserves it across upgrades via lookup, like the other app secrets. - The provisioner and gateway Pods read it from that one Secret. The gateway env is required while provisioner.enabled (the default config references $PROVISIONER_API_KEY and the harness fails on an unset variable) and optional otherwise; existingAppSecret must carry it while the provisioner is enabled. - values.yaml sets sandbox.provisioner_api_key: $PROVISIONER_API_KEY; the README config example, the existingAppSecret key list, the _helpers.tpl comment and NOTES.txt document the key. - backend/tests/test_helm_provisioner_api_key.py pins the values, template source and docs directly and, with helm, the rendered env on both Deployments and the Secret key. * docs(changelog): link the Helm provisioner API key entry to #6365 * fix(helm): render the gateway's PROVISIONER_API_KEY only for the bundled provisioner An explicit env entry wins over envFrom, so emitting the chart-managed gateway env with provisioner.enabled=false overrode the PROVISIONER_API_KEY an operator supplies through `secrets` for an external provisioner with the generated, unrelated value, turning every sandbox request into a 401 on upgrade. The env is now rendered only while the bundled provisioner is enabled (no `optional` flag), leaving the external credential source untouched. Addresses the review on #6365. * fix(helm): keep the default config loadable with the bundled provisioner disabled The default config references $PROVISIONER_API_KEY unconditionally and the harness refuses to start on an unset variable, so with provisioner.enabled=false and no external key the gateway failed at startup instead of booting and only failing sandbox calls at use time, as the chart did before the key existed. The harness resolves an empty variable to "", so the gateway Deployment now injects an empty PROVISIONER_API_KEY in that mode. The entry stays omitted when an external key arrives through `secrets` or `existingSecret`, since an explicit env entry would override the envFrom value. Tests cover the disabled/no-key, disabled/`secrets` and disabled/`existingSecret` renders plus the resolver contract. Addresses the review on #6365. * fix(helm): default PROVISIONER_API_KEY in the gateway start command when the bundled provisioner is off Treating the presence of `existingSecret` as proof that it carries PROVISIONER_API_KEY left the common case of a user-managed provider Secret holding only model keys with an unset variable, so the default config's $PROVISIONER_API_KEY reference still stopped the gateway at startup. The chart cannot see inside that Secret and an explicit env entry would override an envFrom value, so the explicit empty env is replaced by a shell default in the gateway start command, rendered only with provisioner.enabled=false: `export PROVISIONER_API_KEY="${PROVISIONER_API_KEY:-}"` leaves a supplied key untouched and turns an unset variable into "", which the harness resolves to an empty string. Tests run the extracted prefix through sh for both cases and cover the no-key, `secrets` and `existingSecret` renders. Addresses the review on #6365. --------- Co-authored-by: Willem Jiang --- CHANGELOG.md | 20 ++ .../tests/test_helm_provisioner_api_key.py | 181 ++++++++++++++++++ deploy/helm/deer-flow/README.md | 30 ++- deploy/helm/deer-flow/templates/NOTES.txt | 8 +- deploy/helm/deer-flow/templates/_helpers.tpl | 9 +- .../templates/gateway-deployment.yaml | 31 ++- .../templates/provisioner-deployment.yaml | 11 ++ .../helm/deer-flow/templates/secret-app.yaml | 9 + deploy/helm/deer-flow/values.yaml | 20 +- 9 files changed, 303 insertions(+), 16 deletions(-) create mode 100644 backend/tests/test_helm_provisioner_api_key.py diff --git a/CHANGELOG.md b/CHANGELOG.md index badf98fe53..da78341425 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -660,6 +660,25 @@ This release closes that milestone with **439 merged pull requests**. the new turn could silently disappear from it. Until the interrupted worker returns, a prompt in that conversation now shows a notice instead of starting a second run; other conversations stay available through `/new` and `/resume`. ([#6350]) +- **helm:** A default Helm install can create sandboxes again. The chart + enables the sandbox provisioner and points `config.sandbox.provisioner_url` + at it, but nothing rendered `PROVISIONER_API_KEY`: the provisioner + Deployment had no such env, the `-app` Secret no such key, and the + embedded config no `provisioner_api_key`. Since the provisioner started + requiring the key (#4116), its `verify_api_key` middleware answers 401 to + every `/api/*` request while the key is empty or mismatched, so every + sandbox creation failed. The app Secret now generates the key once and + preserves it across upgrades like the other app secrets, the gateway and + provisioner Pods read it from that one Secret (a user-managed + `existingAppSecret` must carry it while `provisioner.enabled` is true), and + the default `config` sets `sandbox.provisioner_api_key: + $PROVISIONER_API_KEY`, which the chart README's config example now keeps + too. With `provisioner.enabled: false` the gateway takes the key an + operator supplies through `secrets` or `existingSecret` (external + provisioner); otherwise its start command defaults the variable to an + empty string, so the default `config` still loads and the gateway boots as + before even when a user-managed provider Secret holds only model keys. + docker-compose was unaffected: it reads the key from `.env`. ([#6365]) - **sandbox:** With host bash enabled, the local sandbox no longer keeps a thread on the skill view of the last restricted Agent that ran there. That view is only maintained while host bash is off, but `LocalSandboxProvider` @@ -8849,4 +8868,5 @@ with **180 merged pull requests** since the first 2.0 milestone tag. [#6350]: https://github.com/bytedance/deer-flow/pull/6350 [#6351]: https://github.com/bytedance/deer-flow/pull/6351 [#6354]: https://github.com/bytedance/deer-flow/pull/6354 +[#6365]: https://github.com/bytedance/deer-flow/pull/6365 [#6388]: https://github.com/bytedance/deer-flow/pull/6388 diff --git a/backend/tests/test_helm_provisioner_api_key.py b/backend/tests/test_helm_provisioner_api_key.py new file mode 100644 index 0000000000..0fec66441b --- /dev/null +++ b/backend/tests/test_helm_provisioner_api_key.py @@ -0,0 +1,181 @@ +"""Regression tests for the Helm chart's provisioner API key. + +The chart enables the sandbox provisioner by default and points +``config.sandbox.provisioner_url`` at it, but nothing rendered +``PROVISIONER_API_KEY``: the provisioner Deployment had no such env, the app +Secret no such key, and the embedded config no ``provisioner_api_key``. Since +the provisioner started requiring the key (#4116), its ``verify_api_key`` +middleware answers 401 to every ``/api/*`` request while the key is empty or +mismatched, so a default install could not create a single sandbox. +docker-compose was unaffected because it reads the key from ``.env``. + +These tests pin the generated, upgrade-preserved key in the app Secret, its +injection into both the gateway and provisioner Pods from that one Secret, the +``existingAppSecret`` wiring, and the ``$PROVISIONER_API_KEY`` reference in the +default config. + +The ``helm template`` tests skip when helm is not installed; CI's runner has it. +""" + +from __future__ import annotations + +import re +import shutil +import subprocess +from pathlib import Path + +import pytest +import yaml + +from deerflow.config.app_config import AppConfig + +REPO_ROOT = Path(__file__).resolve().parents[2] +CHART = REPO_ROOT / "deploy" / "helm" / "deer-flow" +VALUES = CHART / "values.yaml" +APP_SECRET_TEMPLATE = CHART / "templates" / "secret-app.yaml" +README = CHART / "README.md" +NOTES = CHART / "templates" / "NOTES.txt" +GATEWAY_TEMPLATE = CHART / "templates" / "gateway-deployment.yaml" + +ENV_NAME = "PROVISIONER_API_KEY" +SHELL_DEFAULT = 'export PROVISIONER_API_KEY="${PROVISIONER_API_KEY:-}" && ' + + +def _values() -> dict: + return yaml.safe_load(VALUES.read_text(encoding="utf-8")) + + +def _rendered_config() -> dict: + return yaml.safe_load(_values()["config"]) + + +def _render_chart(*settings: str) -> list[dict]: + helm = shutil.which("helm") + if helm is None: + pytest.skip("helm is unavailable") + command = [helm, "template", "deer-flow", str(CHART)] + for setting in settings: + command.extend(["--set", setting]) + rendered = subprocess.run(command, check=True, capture_output=True, text=True).stdout + return [document for document in yaml.safe_load_all(rendered) if isinstance(document, dict)] + + +def _by_kind(documents: list[dict], kind: str, name_suffix: str) -> dict: + return next(document for document in documents if document.get("kind") == kind and document["metadata"]["name"].endswith(name_suffix)) + + +def _container(documents: list[dict], deployment_suffix: str, container_name: str) -> dict: + containers = _by_kind(documents, "Deployment", deployment_suffix)["spec"]["template"]["spec"]["containers"] + return next(container for container in containers if container["name"] == container_name) + + +def _env(container: dict) -> dict[str, dict]: + return {item["name"]: item for item in container["env"]} + + +def _secret_ref(container: dict, env_name: str) -> dict: + return _env(container)[env_name]["valueFrom"]["secretKeyRef"] + + +def test_default_config_references_the_provisioner_api_key() -> None: + """The provisioner is on by default, so the gateway must present the key it was given.""" + values = _values() + assert values["provisioner"]["enabled"] is True + sandbox = _rendered_config()["sandbox"] + assert sandbox["provisioner_url"] == "http://provisioner:8002" + assert sandbox["provisioner_api_key"] == f"${ENV_NAME}", "the harness resolves $VAR from the gateway env" + + +def test_app_secret_template_preserves_the_provisioner_api_key_across_upgrades() -> None: + """``helm template`` never sees ``lookup`` results, so pin the preservation in the source.""" + template = APP_SECRET_TEMPLATE.read_text(encoding="utf-8") + assert f'index $prev.data "{ENV_NAME}"' in template, "a rotated key on upgrade would desynchronize the two Pods mid-rollout" + assert f"{ENV_NAME}: {{{{ $provisionerKey | quote }}}}" in template + + +def test_docs_mention_the_provisioner_api_key() -> None: + assert ENV_NAME in README.read_text(encoding="utf-8") + assert ENV_NAME in NOTES.read_text(encoding="utf-8") + assert "provisioner_api_key: $PROVISIONER_API_KEY" in README.read_text(encoding="utf-8"), "the README config example replaces the chart default wholesale" + + +def test_rendered_app_secret_carries_the_provisioner_api_key() -> None: + secret = _by_kind(_render_chart(), "Secret", "-app") + assert secret["stringData"][ENV_NAME] + + +def test_both_pods_read_the_key_from_the_same_app_secret() -> None: + documents = _render_chart() + provisioner = _secret_ref(_container(documents, "-provisioner", "provisioner"), ENV_NAME) + gateway = _secret_ref(_container(documents, "-gateway", "gateway"), ENV_NAME) + assert provisioner["name"].endswith("-app") + assert provisioner["key"] == ENV_NAME + assert provisioner.get("optional", False) is False, "a provisioner without the key rejects every request" + assert gateway["name"] == provisioner["name"], "the middleware compares the two values byte for byte" + assert gateway["key"] == ENV_NAME + assert gateway.get("optional", False) is False, "the default config references $PROVISIONER_API_KEY and the harness fails on an unset variable" + + +def test_existing_app_secret_is_honored_by_the_provisioner() -> None: + documents = _render_chart("existingAppSecret=my-app-secret") + assert not any(document.get("kind") == "Secret" and document["metadata"]["name"].endswith("-app") for document in documents) + assert _secret_ref(_container(documents, "-provisioner", "provisioner"), ENV_NAME)["name"] == "my-app-secret" + assert _secret_ref(_container(documents, "-gateway", "gateway"), ENV_NAME)["name"] == "my-app-secret" + + +def test_harness_needs_the_variable_set_but_accepts_an_empty_value(monkeypatch: pytest.MonkeyPatch) -> None: + """Pins the contract the gateway Deployment relies on when the bundled provisioner is disabled.""" + reference = {"sandbox": {"provisioner_api_key": f"${ENV_NAME}"}} + monkeypatch.delenv(ENV_NAME, raising=False) + with pytest.raises(ValueError, match=ENV_NAME): + AppConfig.resolve_env_variables(reference) + monkeypatch.setenv(ENV_NAME, "") + assert AppConfig.resolve_env_variables(reference) == {"sandbox": {"provisioner_api_key": ""}} + + +@pytest.mark.parametrize("external", [None, "external-provisioner-key"]) +def test_shell_default_sets_the_variable_only_when_nothing_else_did(external: str | None) -> None: + """Pins the start-command prefix from the template source: it must leave an envFrom-supplied key + alone and turn an unset variable into "" (the only value the harness accepts short of a real key).""" + prefix = re.search(r"\}\}(export PROVISIONER_API_KEY=.*?&& )\{\{ end \}\}cd backend", GATEWAY_TEMPLATE.read_text(encoding="utf-8")) + assert prefix is not None and prefix.group(1) == SHELL_DEFAULT + env = {"PATH": "/usr/bin:/bin"} | ({ENV_NAME: external} if external else {}) + result = subprocess.run(["sh", "-c", prefix.group(1) + f"printenv {ENV_NAME}"], env=env, capture_output=True, text=True) + assert result.returncode == 0, "the variable must be set in every case" + assert result.stdout == f"{external or ''}\n" + + +def test_default_start_command_carries_no_shell_default() -> None: + """With the bundled provisioner the key is injected and required; the start command stays as it was.""" + gateway = _container(_render_chart(), "-gateway", "gateway") + assert not " ".join(gateway["args"]).startswith("export") + + +def test_gateway_key_is_omitted_when_an_external_key_is_supplied() -> None: + """An explicit env entry wins over envFrom, so rendering it would override the key an operator + supplies through ``secrets`` for an external provisioner with the generated, unrelated value.""" + documents = _render_chart("provisioner.enabled=false", "secrets.PROVISIONER_API_KEY=external-provisioner-key") + assert not any(document.get("kind") == "Deployment" and document["metadata"]["name"].endswith("-provisioner") for document in documents) + gateway = _container(documents, "-gateway", "gateway") + assert ENV_NAME not in _env(gateway) + assert any(source["secretRef"]["name"].endswith("-provider") for source in gateway["envFrom"]), "the external key still arrives through the provider Secret" + assert _by_kind(documents, "Secret", "-provider")["stringData"][ENV_NAME] == "external-provisioner-key" + assert " ".join(gateway["args"]).startswith(SHELL_DEFAULT), "the shell default keeps the supplied key" + + +def test_user_managed_provider_secret_without_the_key_still_boots_the_gateway() -> None: + """``existingSecret`` is a generic provider Secret that often holds only model keys; the chart + cannot see inside it, so it must neither override a key it carries nor leave the variable unset.""" + gateway = _container(_render_chart("provisioner.enabled=false", "existingSecret=my-provider-secret"), "-gateway", "gateway") + assert ENV_NAME not in _env(gateway) + assert [source["secretRef"]["name"] for source in gateway["envFrom"]] == ["my-provider-secret"] + assert " ".join(gateway["args"]).startswith(SHELL_DEFAULT) + + +def test_disabled_provisioner_without_any_key_still_boots_the_gateway() -> None: + """The default config still references $PROVISIONER_API_KEY; an unset variable would stop the gateway + from starting at all, where the chart used to boot it and only fail sandbox calls at use time.""" + gateway = _container(_render_chart("provisioner.enabled=false"), "-gateway", "gateway") + assert ENV_NAME not in _env(gateway) + assert "envFrom" not in gateway + assert " ".join(gateway["args"]).startswith(SHELL_DEFAULT) diff --git a/deploy/helm/deer-flow/README.md b/deploy/helm/deer-flow/README.md index 71b58f9775..8275998025 100644 --- a/deploy/helm/deer-flow/README.md +++ b/deploy/helm/deer-flow/README.md @@ -145,6 +145,7 @@ config: | sandbox: use: deerflow.community.aio_sandbox:AioSandboxProvider provisioner_url: http://provisioner:8002 + provisioner_api_key: $PROVISIONER_API_KEY # injected from the app Secret database: backend: postgres postgres_url: $DATABASE_URL @@ -238,7 +239,9 @@ curl http://localhost:2026/health # gateway health via nginx Hit the Ingress host (map it in `/etc/hosts` for local clusters) to load the UI. -Provisioner sanity check: +Provisioner sanity check (`/health` is unauthenticated; `/api/*` requires the +`PROVISIONER_API_KEY` the chart generates into the app Secret, so a 401 from +the gateway's sandbox calls means the two Pods disagree on that key): ```bash kubectl -n deer-flow exec deploy/deer-flow-provisioner -- curl -s localhost:8002/health @@ -301,11 +304,26 @@ kubectl -n deer-flow exec deploy/deer-flow-provisioner -- curl -s localhost:8002 multi-instance gateway (same rule), and the rollout strategy is surge-then-drain (`maxSurge: 1`, `maxUnavailable: 0`). - **App secret.** `-app` holds `BETTER_AUTH_SECRET`, - `DEER_FLOW_INTERNAL_AUTH_TOKEN` and `AUTH_JWT_SECRET` (the session-cookie - signing key), each generated once and preserved across upgrades via - `lookup`. `existingAppSecret` points the gateway and frontend at a Secret - you manage instead (no `-app` is generated); it must carry all - three keys. `AUTH_JWT_SECRET` is required whenever the gateway is + `DEER_FLOW_INTERNAL_AUTH_TOKEN`, `AUTH_JWT_SECRET` (the session-cookie + signing key) and `PROVISIONER_API_KEY` (the key the gateway presents to the + sandbox provisioner), each generated once and preserved across upgrades via + `lookup`. `existingAppSecret` points the gateway, frontend and provisioner + at a Secret you manage instead (no `-app` is generated); it must + carry the first three keys and, while `provisioner.enabled` is true, also + `PROVISIONER_API_KEY`: the provisioner's `verify_api_key` middleware + answers 401 to every `/api/*` request while its key is empty or differs + from the one the gateway sends, so a deployment without it cannot create a + single sandbox. The chart injects the same Secret value into both Pods and + the default `config` references it as `sandbox.provisioner_api_key: + $PROVISIONER_API_KEY` (keep that line when you override `config:`; the + harness refuses to start when a referenced variable is unset). With + `provisioner.enabled: false` the gateway takes `PROVISIONER_API_KEY` from + `secrets` or `existingSecret` when you supply one, so an external + provisioner keeps its key (the chart emits no `env` entry, which would win + over `envFrom`); otherwise the gateway's start command defaults the + variable to an empty string, so the default `config` still loads even when + a user-managed provider Secret holds only model keys. `AUTH_JWT_SECRET` is + required whenever the gateway is multi-instance — without it, concurrently booting Pods race to write their own `.jwt_secret` on the home volume and sign sessions with different keys — and only a single Pod may omit it and fall back to that file. **Upgrading a diff --git a/deploy/helm/deer-flow/templates/NOTES.txt b/deploy/helm/deer-flow/templates/NOTES.txt index b2e09b7123..f5f1004cc0 100644 --- a/deploy/helm/deer-flow/templates/NOTES.txt +++ b/deploy/helm/deer-flow/templates/NOTES.txt @@ -69,9 +69,11 @@ Provisioner / sandbox notes: Generated secrets (persisted across upgrades): - • BETTER_AUTH_SECRET, DEER_FLOW_INTERNAL_AUTH_TOKEN and AUTH_JWT_SECRET (the - session-cookie signing key shared by every gateway Pod) are stored in the - Secret `{{ include "deer-flow.appSecret" . }}`. + • BETTER_AUTH_SECRET, DEER_FLOW_INTERNAL_AUTH_TOKEN, AUTH_JWT_SECRET (the + session-cookie signing key shared by every gateway Pod) and + PROVISIONER_API_KEY (the key the gateway presents to the sandbox + provisioner, which answers 401 to every /api/* request without it) are + stored in the Secret `{{ include "deer-flow.appSecret" . }}`. Database ({{ if .Values.postgresql.enabled }}bundled postgres{{ else }}external postgres{{ end }}): diff --git a/deploy/helm/deer-flow/templates/_helpers.tpl b/deploy/helm/deer-flow/templates/_helpers.tpl index a6a41d0f17..2ec38d4343 100644 --- a/deploy/helm/deer-flow/templates/_helpers.tpl +++ b/deploy/helm/deer-flow/templates/_helpers.tpl @@ -71,10 +71,11 @@ imagePullSecrets: {{- end -}} {{/* Name of the Secret holding the app secrets (BETTER_AUTH_SECRET, - DEER_FLOW_INTERNAL_AUTH_TOKEN, AUTH_JWT_SECRET). `existingAppSecret` points - every consumer (gateway, frontend) at a user-managed Secret instead and - skips generating one; it must carry all three keys for more than one - gateway instance. */}} + DEER_FLOW_INTERNAL_AUTH_TOKEN, AUTH_JWT_SECRET, PROVISIONER_API_KEY). + `existingAppSecret` points every consumer (gateway, frontend, + provisioner) at a user-managed Secret instead and skips generating one; + it must carry PROVISIONER_API_KEY while the provisioner is enabled and + AUTH_JWT_SECRET for more than one gateway instance. */}} {{- define "deer-flow.appSecret" -}} {{- if .Values.existingAppSecret -}}{{- .Values.existingAppSecret -}} {{- else -}}{{- printf "%s-app" (include "deer-flow.fullname" .) -}}{{- end -}} diff --git a/deploy/helm/deer-flow/templates/gateway-deployment.yaml b/deploy/helm/deer-flow/templates/gateway-deployment.yaml index 1cd2a51ba0..ce2d6a9eae 100644 --- a/deploy/helm/deer-flow/templates/gateway-deployment.yaml +++ b/deploy/helm/deer-flow/templates/gateway-deployment.yaml @@ -91,7 +91,18 @@ spec: workingDir: /app command: ["sh", "-c"] args: - - cd backend && PYTHONPATH=. uv run --no-sync uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 --workers 1 --timeout-graceful-shutdown {{ .Values.gateway.uvicornGracefulShutdownSeconds | int }} + {{- /* With the bundled provisioner disabled, the default config still + references $PROVISIONER_API_KEY; the harness refuses to start on + an unset variable but resolves an empty one to "". The chart + cannot see whether `secrets`/`existingSecret` carry the key (a + user-managed provider Secret often holds only model keys), and an + explicit env entry would override an envFrom value, so the shell + defaults the variable to "" only when nothing else set it. An + external provisioner's key is never touched and the gateway boots + as it did before the key existed, failing sandbox calls at use + time. With the bundled provisioner the key is injected below and + required, so no default is needed. */}} + - {{ if not .Values.provisioner.enabled }}export PROVISIONER_API_KEY="${PROVISIONER_API_KEY:-}" && {{ end }}cd backend && PYTHONPATH=. uv run --no-sync uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 --workers 1 --timeout-graceful-shutdown {{ .Values.gateway.uvicornGracefulShutdownSeconds | int }} env: - name: CI value: "true" @@ -139,6 +150,24 @@ spec: name: {{ include "deer-flow.appSecret" . }} key: AUTH_JWT_SECRET optional: {{ ne (include "deer-flow.gatewayMultiInstance" .) "true" }} + {{- if .Values.provisioner.enabled }} + # Shared gateway -> bundled-provisioner key. config.yaml references + # it as sandbox.provisioner_api_key: $PROVISIONER_API_KEY, and the + # harness refuses to start when a referenced variable is unset, so + # the key is required here (the provisioner rejects every /api/* + # request without it anyway) and a user-managed existingAppSecret + # must carry it. Rendered only for the bundled provisioner: an + # explicit env entry wins over envFrom, so emitting it with + # provisioner.enabled=false would override the PROVISIONER_API_KEY + # an operator supplies through `secrets`/`existingSecret` for an + # external provisioner. In that mode the start command above + # defaults the variable to "" when nothing else set it. + - name: PROVISIONER_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "deer-flow.appSecret" . }} + key: PROVISIONER_API_KEY + {{- end }} # Tells the Gateway's startup gate that peers share the database; # its GATEWAY_WORKERS count only sees this Pod. Derived from # gateway.replicas unless gateway.multiInstance is set explicitly: diff --git a/deploy/helm/deer-flow/templates/provisioner-deployment.yaml b/deploy/helm/deer-flow/templates/provisioner-deployment.yaml index d117c66605..a787f00362 100644 --- a/deploy/helm/deer-flow/templates/provisioner-deployment.yaml +++ b/deploy/helm/deer-flow/templates/provisioner-deployment.yaml @@ -49,6 +49,17 @@ spec: value: {{ include "deer-flow.namespace" . | quote }} - name: SANDBOX_IMAGE value: {{ .Values.provisioner.sandboxImage | quote }} + # Shared secret the gateway sends as X-API-Key. verify_api_key in + # docker/provisioner/app.py answers 401 to every /api/* request + # while this is empty or mismatched, so a provisioner without it + # cannot create a single sandbox: the key is required (a + # user-managed existingAppSecret must carry it) and comes from the + # same Secret the gateway reads. + - name: PROVISIONER_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "deer-flow.appSecret" . }} + key: PROVISIONER_API_KEY # In-cluster API access via the mounted ServiceAccount token. # K8S_API_SERVER is intentionally unset (do NOT use host.docker.internal). # Service type for per-sandbox Services. ClusterIP (default) routes the diff --git a/deploy/helm/deer-flow/templates/secret-app.yaml b/deploy/helm/deer-flow/templates/secret-app.yaml index 5fb67023ca..23103535e8 100644 --- a/deploy/helm/deer-flow/templates/secret-app.yaml +++ b/deploy/helm/deer-flow/templates/secret-app.yaml @@ -3,10 +3,12 @@ {{- $betterAuth := "" -}} {{- $internalToken := "" -}} {{- $jwtSecret := "" -}} +{{- $provisionerKey := "" -}} {{- if $prev -}} {{- $betterAuth = index $prev.data "BETTER_AUTH_SECRET" | default "" | b64dec -}} {{- $internalToken = index $prev.data "DEER_FLOW_INTERNAL_AUTH_TOKEN" | default "" | b64dec -}} {{- $jwtSecret = index $prev.data "AUTH_JWT_SECRET" | default "" | b64dec -}} +{{- $provisionerKey = index $prev.data "PROVISIONER_API_KEY" | default "" | b64dec -}} {{- end -}} {{- if not $betterAuth -}}{{- $betterAuth = randAlphaNum 48 | lower -}}{{- end -}} {{- if not $internalToken -}}{{- $internalToken = randAlphaNum 40 -}}{{- end -}} @@ -14,6 +16,12 @@ so sessions survive, and shared by every gateway Pod so a session signed on one replica validates on another. */ -}} {{- if not $jwtSecret -}}{{- $jwtSecret = randAlphaNum 48 -}}{{- end -}} +{{- /* Shared gateway -> sandbox-provisioner key. The provisioner's + verify_api_key middleware answers 401 to every /api/* request while + PROVISIONER_API_KEY is empty or mismatched, so both Pods read this one + value. Generated once and preserved across upgrades so a rollout never + leaves the two sides on different keys. */ -}} +{{- if not $provisionerKey -}}{{- $provisionerKey = randAlphaNum 48 -}}{{- end -}} apiVersion: v1 kind: Secret metadata: @@ -26,4 +34,5 @@ stringData: BETTER_AUTH_SECRET: {{ $betterAuth | quote }} DEER_FLOW_INTERNAL_AUTH_TOKEN: {{ $internalToken | quote }} AUTH_JWT_SECRET: {{ $jwtSecret | quote }} + PROVISIONER_API_KEY: {{ $provisionerKey | quote }} {{- end -}} diff --git a/deploy/helm/deer-flow/values.yaml b/deploy/helm/deer-flow/values.yaml index c7f867a18d..2683aba4d4 100644 --- a/deploy/helm/deer-flow/values.yaml +++ b/deploy/helm/deer-flow/values.yaml @@ -132,6 +132,15 @@ nginx: # this namespace via a ServiceAccount + RBAC. provisioner: enabled: true + # The gateway authenticates to the provisioner with PROVISIONER_API_KEY, + # which the chart generates into the `-app` Secret (see + # existingAppSecret) and injects into both Pods; `config.sandbox` below + # references it as `provisioner_api_key: $PROVISIONER_API_KEY`. The + # provisioner rejects every /api/* request without a matching key. With + # `enabled: false` the gateway takes PROVISIONER_API_KEY from `secrets` / + # `existingSecret` when you supply one (external provisioner); otherwise + # its start command defaults the variable to "", so the default `config` + # below still loads even when a user-managed Secret lacks the key. resources: requests: cpu: 50m @@ -289,8 +298,10 @@ secrets: {} existingSecret: "" # -- Use an existing Secret for the app secrets instead of generating # `-app`. It must hold BETTER_AUTH_SECRET (frontend), -# DEER_FLOW_INTERNAL_AUTH_TOKEN and, for more than one gateway instance, -# AUTH_JWT_SECRET (the session-cookie signing key every Pod must share). +# DEER_FLOW_INTERNAL_AUTH_TOKEN, PROVISIONER_API_KEY (the shared gateway -> +# sandbox-provisioner key, required while `provisioner.enabled`) and, for +# more than one gateway instance, AUTH_JWT_SECRET (the session-cookie +# signing key every Pod must share). existingAppSecret: "" # -- Ingress in front of nginx (port 2026). nginx preserves all internal routing. @@ -340,6 +351,11 @@ config: | sandbox: use: deerflow.community.aio_sandbox:AioSandboxProvider provisioner_url: http://provisioner:8002 + # Sent as X-API-Key on every provisioner request. The chart injects the + # generated `-app` key into the gateway and provisioner Pods as + # PROVISIONER_API_KEY; without a matching value the provisioner answers + # 401 and no sandbox can be created. + provisioner_api_key: $PROVISIONER_API_KEY image: enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest port: 8080 replicas: 3