From bc9ee9645cc1aa79ca2d230a671b167614085a73 Mon Sep 17 00:00:00 2001 From: Zheng Feng Date: Thu, 9 Jul 2026 15:40:53 +0800 Subject: [PATCH] feat(deploy): first-class Helm chart for Kubernetes deployment (#3987) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(helm): add production-ready Helm chart for Kubernetes deployment Adds deploy/helm/deer-flow, a native-Kubernetes translation of the production docker-compose stack, plus CI to publish its images and chart. * ci(release): gate releases on version-source consistency Add a reusable verify-versions workflow invoked by both chart.yaml and container.yaml on v* tags. It runs scripts/verify_versions.sh against the tag and fails the release — skipping all image and chart publishing — when Chart.yaml (version + appVersion), backend/pyproject.toml, or frontend/package.json don't all match the tag. Add scripts/verify_versions.sh (the check, also runnable locally) and scripts/bump_version.sh (bumps all four sources in lockstep, then self-verifies). Document the release flow in RELEASING.md and link it from AGENTS.md. * fix(deploy): address Helm chart review feedback (#3987) Three review items from willem-bd: 1. nginx IPv6 listen strip never matched. The sed pattern required a `;` immediately after `2026`, but the rendered config emits `listen [::]:2026 default_server;` (space + `default_server` before the `;`), so the line was never deleted and nginx crash-looped on pods without IPv6 (`socket() :::2026 failed (97: Address family not supported)`). Drop the trailing `;` from the pattern so it matches. Same latent bug fixed in docker-compose-dev.yaml. 2. Passwords were spliced into DSNs verbatim, so a password containing URL-special chars (@ : / # ? % [ ] space) produced a malformed DSN and a confusing parse error. Add a `deer-flow.urlEscape` helper (replace-based: Sprig lacks urlqueryescape, and regexReplaceAllLiteral treats the replacement as a regex template so `[`/`]`/`?` break it) and apply it to the password in the postgres and redis DSNs. The raw `postgres-password` / `redis-password` keys stay unencoded - they back POSTGRES_PASSWORD / REDIS_PASSWORD, not a URL segment. 3. NODE_HOST defaulted to "gateway", which can never route: the gateway Service is ClusterIP:8001 and knows nothing of a sandbox NodePort, so a user who skips the caveat gets unreachable sandboxes with no error at install time. Default NODE_HOST to the provisioner pod's node IP via the downward API (status.hostIP) - a NodePort is exposed on every node, so : routes from the gateway on most clusters. `provisioner.nodeHost` remains an override for CNIs/policies that block pod->node-IP traffic. Updated NOTES.txt, values.yaml, and the chart README. (#3929 remains the long-term fix - ClusterIP + cluster-DNS URL removes NODE_HOST and the NodePort exposure entirely.) Validated with helm lint, helm template (incl. a special-char password rendering the encoded DSNs), and a sed pattern-match check. * fix(deploy): address round-2 Helm chart review feedback (#3987) Three "Medium" items from willem-bd: 1. No helm lint / helm template gate before publish. A template regression ships as an immutable OCI artifact (GHCR won't overwrite --version), so gate packaging on `helm lint` + `helm template --include-crds` in chart.yaml before `helm package`. (ct lint / helm-unittest deferred.) 2. Action pinning inconsistent + PR body overstates it. SHA-pin actions/checkout (v6.0.3, df4cb1c0) and actions/attest-build-provenance (v2.4.0, e8998f94) across the publishing workflows (chart.yaml, container.yaml, verify-versions.yml), matching the existing docker/* SHA-pin pattern. Resolves the checkout @v4/@v6 mismatch and makes the "SHA-pinned actions" claim accurate. Other pre-existing workflows left untouched (out of scope for this PR). 3. Provisioner RBAC broader than needed. Dropped the unused update/patch verbs and the pods/exec + events rules from the provisioner Role - audited against docker/provisioner/app.py, which only calls get/create/delete on pods and get/list/create/delete on services. Fixed NOTES.txt to accurately describe the grant instead of understating it as "create Pods and Services". The remaining scope concern - verbs apply to all Pods in the namespace, not just sandbox Pods - is still deferred (RBAC can't scope by label; needs a dedicated namespace or admission control), now noted in NOTES.txt and README. Validated with helm lint + helm template (narrowed Role renders with exactly get/list/watch/create/delete). * feat(helm): enable sandbox+web tools out of the box The chart's default config loaded zero agent tools (config.tools empty -> "Total tools loaded: 0"), so a fresh install gave an agent that could do nothing useful. Add tool_groups + tools to the default config block: - web: web_search (ddg), web_fetch (jina), image_search - no API key - file:read: ls, read_file, glob, grep - file:write: write_file, str_replace - bash The file/bash tools run inside the AIO sandbox the chart already configures; the web tools need outbound internet from the gateway pod (swap backends or drop entries for air-gapped clusters - see config.example.yaml). Also bump config_version 15 -> 19 to match config.example.yaml (the chart had drifted behind). NOTES.txt and the README example updated to match. * ci(helm): add chart validation + config_version drift check on PR Extend the chart workflow with a PR-triggered validate-chart job that runs helm lint, helm template --include-crds, and a config_version drift check: it parses config_version from both config.example.yaml and the chart's values.yaml and fails the build (with a ::error:: naming the files to bump) if the chart is behind the example. This catches the kind of drift this PR is fixing - the chart sat at v15 while the example moved to v19 - before it can merge again. verify-versions and publish-chart stay tag-only; publish-chart now needs: [verify-versions, validate-chart]. validate-chart runs on both PRs and tag pushes: the tag arm is required because a job that `needs` a skipped job is itself skipped under the default success() check, so validate-chart must actually run on tag pushes or publish-chart would never fire. * Bump config version to 20 --- .github/workflows/chart.yaml | 98 +++++ .github/workflows/container.yaml | 63 ++- .github/workflows/verify-versions.yml | 27 ++ AGENTS.md | 1 + RELEASING.md | 126 ++++++ deploy/helm/deer-flow/Chart.yaml | 15 + deploy/helm/deer-flow/README.md | 395 ++++++++++++++++++ deploy/helm/deer-flow/templates/NOTES.txt | 133 ++++++ deploy/helm/deer-flow/templates/_helpers.tpl | 155 +++++++ .../deer-flow/templates/configmap-config.yaml | 10 + .../templates/configmap-extensions.yaml | 10 + .../deer-flow/templates/configmap-nginx.yaml | 225 ++++++++++ .../templates/frontend-deployment.yaml | 75 ++++ .../deer-flow/templates/frontend-service.yaml | 17 + .../templates/gateway-deployment.yaml | 190 +++++++++ .../deer-flow/templates/gateway-service.yaml | 17 + deploy/helm/deer-flow/templates/ingress.yaml | 37 ++ .../deer-flow/templates/nginx-deployment.yaml | 85 ++++ .../deer-flow/templates/nginx-service.yaml | 23 + .../deer-flow/templates/postgres-secret.yaml | 43 ++ .../deer-flow/templates/postgres-service.yaml | 19 + .../templates/postgres-statefulset.yaml | 95 +++++ .../templates/provisioner-deployment.yaml | 94 +++++ .../deer-flow/templates/provisioner-rbac.yaml | 76 ++++ .../templates/provisioner-service.yaml | 19 + deploy/helm/deer-flow/templates/pvc-home.yaml | 18 + .../deer-flow/templates/redis-secret.yaml | 43 ++ .../deer-flow/templates/redis-service.yaml | 19 + .../templates/redis-statefulset.yaml | 106 +++++ .../helm/deer-flow/templates/secret-app.yaml | 22 + .../deer-flow/templates/secret-provider.yaml | 14 + deploy/helm/deer-flow/values.yaml | 320 ++++++++++++++ docker/docker-compose-dev.yaml | 2 +- scripts/bump_version.sh | 97 +++++ scripts/verify_versions.sh | 78 ++++ 35 files changed, 2762 insertions(+), 5 deletions(-) create mode 100644 .github/workflows/chart.yaml create mode 100644 .github/workflows/verify-versions.yml create mode 100644 RELEASING.md create mode 100644 deploy/helm/deer-flow/Chart.yaml create mode 100644 deploy/helm/deer-flow/README.md create mode 100644 deploy/helm/deer-flow/templates/NOTES.txt create mode 100644 deploy/helm/deer-flow/templates/_helpers.tpl create mode 100644 deploy/helm/deer-flow/templates/configmap-config.yaml create mode 100644 deploy/helm/deer-flow/templates/configmap-extensions.yaml create mode 100644 deploy/helm/deer-flow/templates/configmap-nginx.yaml create mode 100644 deploy/helm/deer-flow/templates/frontend-deployment.yaml create mode 100644 deploy/helm/deer-flow/templates/frontend-service.yaml create mode 100644 deploy/helm/deer-flow/templates/gateway-deployment.yaml create mode 100644 deploy/helm/deer-flow/templates/gateway-service.yaml create mode 100644 deploy/helm/deer-flow/templates/ingress.yaml create mode 100644 deploy/helm/deer-flow/templates/nginx-deployment.yaml create mode 100644 deploy/helm/deer-flow/templates/nginx-service.yaml create mode 100644 deploy/helm/deer-flow/templates/postgres-secret.yaml create mode 100644 deploy/helm/deer-flow/templates/postgres-service.yaml create mode 100644 deploy/helm/deer-flow/templates/postgres-statefulset.yaml create mode 100644 deploy/helm/deer-flow/templates/provisioner-deployment.yaml create mode 100644 deploy/helm/deer-flow/templates/provisioner-rbac.yaml create mode 100644 deploy/helm/deer-flow/templates/provisioner-service.yaml create mode 100644 deploy/helm/deer-flow/templates/pvc-home.yaml create mode 100644 deploy/helm/deer-flow/templates/redis-secret.yaml create mode 100644 deploy/helm/deer-flow/templates/redis-service.yaml create mode 100644 deploy/helm/deer-flow/templates/redis-statefulset.yaml create mode 100644 deploy/helm/deer-flow/templates/secret-app.yaml create mode 100644 deploy/helm/deer-flow/templates/secret-provider.yaml create mode 100644 deploy/helm/deer-flow/values.yaml create mode 100755 scripts/bump_version.sh create mode 100755 scripts/verify_versions.sh diff --git a/.github/workflows/chart.yaml b/.github/workflows/chart.yaml new file mode 100644 index 000000000..4240e201b --- /dev/null +++ b/.github/workflows/chart.yaml @@ -0,0 +1,98 @@ +name: Publish Helm Chart + +# Publishes the DeerFlow Helm chart as an OCI artifact to GHCR alongside the +# container images (see container.yaml). Triggers on the same `v*` tags. +# +# On pull requests touching the chart or config.example.yaml, `validate-chart` +# runs lint + template render + a config_version drift check (the chart's +# embedded config_version must not lag config.example.yaml) so a broken or +# stale chart fails the PR, not the release. +# +# Users then install with: +# helm install deer-flow oci://ghcr.io/${{ owner }}/deer-flow --version + +on: + push: + tags: + - "v*" + pull_request: + paths: + - "deploy/helm/deer-flow/**" + - "config.example.yaml" + - ".github/workflows/chart.yaml" + +jobs: + validate-chart: + # Runs on PRs and release tags: catch a broken render or a stale + # config_version before merge / publish. A broken chart published under a + # vX.Y.Z tag is an immutable OCI artifact (GHCR won't let you overwrite + # --version), so a regression must fail here, not on install. + if: github.event_name == 'pull_request' || startsWith(github.ref, 'refs/tags/v') + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Checkout repository + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3 + + # ubuntu-latest ships with helm 3 preinstalled - no setup-helm action needed. + - name: Lint chart + run: helm lint deploy/helm/deer-flow + - name: Validate templates render + run: helm template deer-flow deploy/helm/deer-flow --include-crds >/dev/null + + # The chart's `config:` block embeds a config_version that must not fall + # behind config.example.yaml. A stale version is silent in-cluster (the + # image ships no example to compare against, so _check_config_version + # never warns) but means the chart's config is authored against an older + # schema. Bump it in values.yaml and the README example. config_version + # gates no runtime behavior - it only drives the outdated-warning - so a + # bare version bump needs no field changes. + - name: config_version drift check + run: | + set -eu + example=$(grep -E '^config_version:[[:space:]]+[0-9]+' config.example.yaml | head -1 | awk '{print $2}') + chart=$(awk '/^config:[[:space:]]*\|/{f=1; next} f && /^[[:space:]]+config_version:[[:space:]]+[0-9]+/ {print $2; exit}' deploy/helm/deer-flow/values.yaml) + echo "config.example.yaml config_version=$example" + echo "chart values.yaml config_version=$chart" + if [ -z "$example" ] || [ -z "$chart" ]; then + echo "::error::could not parse config_version from one of the files" + exit 1 + fi + if [ "$chart" -lt "$example" ]; then + echo "::error::chart config_version ($chart) is behind config.example.yaml ($example). Bump 'config_version' in deploy/helm/deer-flow/values.yaml (and the README example) to $example." + exit 1 + fi + + verify-versions: + # Gate the release: every version source must match the v* tag. A forgotten + # bump in Chart.yaml, pyproject.toml, or package.json fails here and skips + # the publish. See scripts/verify_versions.sh. + if: startsWith(github.ref, 'refs/tags/v') + uses: ./.github/workflows/verify-versions.yml + + publish-chart: + if: startsWith(github.ref, 'refs/tags/v') + needs: [verify-versions, validate-chart] + runs-on: ubuntu-latest + permissions: + contents: read + packages: write + steps: + - name: Checkout repository + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3 + + - name: Log in to GHCR + run: | + echo "${{ secrets.GITHUB_TOKEN }}" | \ + helm registry login ghcr.io -u ${{ github.actor }} --password-stdin + + - name: Package chart + run: helm package deploy/helm/deer-flow --destination ./packages + + - name: Push chart to GHCR + run: | + for pkg in ./packages/*.tgz; do + echo "--- pushing $pkg" + helm push "$pkg" oci://ghcr.io/${{ github.repository_owner }} + done diff --git a/.github/workflows/container.yaml b/.github/workflows/container.yaml index 6e5fd7608..336c8f3cb 100644 --- a/.github/workflows/container.yaml +++ b/.github/workflows/container.yaml @@ -6,8 +6,14 @@ on: - "v*" jobs: + verify-versions: + # Gate the release: every version source must match the v* tag. A forgotten + # bump in Chart.yaml, pyproject.toml, or package.json fails here and skips + # all image builds. See scripts/verify_versions.sh. + uses: ./.github/workflows/verify-versions.yml backend-container: + needs: verify-versions runs-on: ubuntu-latest permissions: contents: read @@ -19,7 +25,7 @@ jobs: IMAGE_NAME: ${{ github.repository }}-backend steps: - name: Checkout repository - uses: actions/checkout@v6 + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3 - name: Log in to the Container registry uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 #v3.4.0 with: @@ -47,13 +53,14 @@ jobs: labels: ${{ steps.meta.outputs.labels }} - name: Generate artifact attestation - uses: actions/attest-build-provenance@v2 + uses: actions/attest-build-provenance@e8998f949152b193b063cb0ec769d69d929409be #v2.4.0 with: subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}} subject-digest: ${{ steps.push.outputs.digest }} push-to-registry: true frontend-container: + needs: verify-versions runs-on: ubuntu-latest permissions: contents: read @@ -66,7 +73,7 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v6 + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3 - name: Log in to the Container registry uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 #v3.4.0 with: @@ -94,7 +101,55 @@ jobs: labels: ${{ steps.meta.outputs.labels }} - name: Generate artifact attestation - uses: actions/attest-build-provenance@v2 + uses: actions/attest-build-provenance@e8998f949152b193b063cb0ec769d69d929409be #v2.4.0 + with: + subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}} + subject-digest: ${{ steps.push.outputs.digest }} + push-to-registry: true + + provisioner-container: + needs: verify-versions + runs-on: ubuntu-latest + permissions: + contents: read + packages: write + attestations: write + id-token: write + env: + REGISTRY: ghcr.io + IMAGE_NAME: ${{ github.repository }}-provisioner + + steps: + - name: Checkout repository + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3 + - name: Log in to the Container registry + uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 #v3.4.0 + with: + registry: ${{ env.REGISTRY }} + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + - name: Extract metadata (tags, labels) for Docker + id: meta + uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 #v5.7.0 + with: + images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} + tags: | + type=ref,event=tag + type=ref,event=branch + type=sha + type=raw,value=latest,enable={{is_default_branch}} + - name: Build and push Docker image + id: push + uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 #v6.18.0 + with: + context: docker/provisioner + file: docker/provisioner/Dockerfile + push: true + tags: ${{ steps.meta.outputs.tags }} + labels: ${{ steps.meta.outputs.labels }} + + - name: Generate artifact attestation + uses: actions/attest-build-provenance@e8998f949152b193b063cb0ec769d69d929409be #v2.4.0 with: subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}} subject-digest: ${{ steps.push.outputs.digest }} diff --git a/.github/workflows/verify-versions.yml b/.github/workflows/verify-versions.yml new file mode 100644 index 000000000..eadb285e7 --- /dev/null +++ b/.github/workflows/verify-versions.yml @@ -0,0 +1,27 @@ +name: Verify Versions + +# Reusable workflow: checks that every project version source agrees with the +# git tag that triggered the release. Called by chart.yaml and container.yaml +# on v* tags so a forgotten version bump blocks the entire release (container +# images + chart), not just the chart. +# +# Sources verified: deploy/helm/deer-flow/Chart.yaml (version + appVersion), +# backend/pyproject.toml, frontend/package.json. Logic lives in +# scripts/verify_versions.sh so it can also be run locally. + +on: + workflow_call: + +jobs: + verify-versions: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Checkout repository + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3 + + - name: Verify all version sources match tag + run: | + TAG_VERSION="${GITHUB_REF_NAME#v}" + bash scripts/verify_versions.sh "$TAG_VERSION" diff --git a/AGENTS.md b/AGENTS.md index 1b40cc84c..5aaedbdb0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -113,6 +113,7 @@ Rule of thumb: **root `make` = the full application**; **`backend/Makefile` and `README_ja.md`, `README_fr.md`, `README_ru.md`) - Security policy → **[SECURITY.md](SECURITY.md)** - Changes → **[CHANGELOG.md](CHANGELOG.md)** +- Cutting a release → **[RELEASING.md](RELEASING.md)** ## Cross-Cutting Conventions diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 000000000..5b866e630 --- /dev/null +++ b/RELEASING.md @@ -0,0 +1,126 @@ +# Releasing DeerFlow + +DeerFlow releases are **tag-driven**: pushing a `v*` git tag triggers the +publishing workflows. There is no separate release script that bumps versions — +the maintainer bumps the version sources, updates the changelog, commits, and +tags. The helper scripts below keep the version sources in lockstep, and CI +gates the release on them agreeing with the tag. + +## Version sources + +A release version must appear, identically, in four places: + +| File | Field | +| -------------------------------------- | -------------------- | +| `backend/pyproject.toml` | `version = "X.Y.Z"` | +| `frontend/package.json` | `"version": "X.Y.Z"` | +| `deploy/helm/deer-flow/Chart.yaml` | `version: X.Y.Z` | +| `deploy/helm/deer-flow/Chart.yaml` | `appVersion: "X.Y.Z"`| + +Plus the git tag `vX.Y.Z` itself, which is the canonical release identifier. + +Container images are tagged from the git tag (not from these files), and the +Helm chart version is validated against the tag — so if any source lags the +tag, the release is blocked (see [Version gate](#version-gate)). + +## Helper scripts + +- `scripts/bump_version.sh ` — set all four fields at once, then + self-verify. Tolerates a leading `v` (e.g. `v2.2.0`). + ```bash + scripts/bump_version.sh 2.2.0 + ``` +- `scripts/verify_versions.sh [version]` — check that all sources agree. With + no argument it requires mutual equality; with an argument it requires every + source to equal it. Exits non-zero on mismatch. Run it locally before tagging + to catch drift early: + ```bash + scripts/verify_versions.sh 2.2.0 + ``` + +## Release procedure + +1. **Bump the version** across all sources: + ```bash + scripts/bump_version.sh 2.2.0 + ``` +2. **Update `CHANGELOG.md`**: rename the `## [Unreleased]` section to + `## [2.2.0] — YYYY-MM-DD` (note the em dash `—`), and add a link reference + at the bottom of the file: + ``` + [2.2.0]: https://github.com/bytedance/deer-flow/releases/tag/v2.2.0 + ``` + Start a fresh `## [Unreleased]` section above it for the next cycle. +3. **Commit** the version + changelog changes: + ```bash + git add -A + git commit -m "release: v2.2.0" + ``` +4. **Tag and push**: + ```bash + git tag v2.2.0 + git push origin v2.2.0 + ``` + Pushing the tag triggers the publishing workflows (below). + +## What CI publishes on a `v*` tag + +- `.github/workflows/container.yaml` — builds and pushes `backend`, + `frontend`, and `provisioner` images to `ghcr.io`, tagged with the release + version (and `latest` on the default branch). +- `.github/workflows/chart.yaml` — packages the Helm chart and pushes it as an + OCI artifact to `ghcr.io`. Users install with: + ```bash + helm install deer-flow oci://ghcr.io//deer-flow --version 2.2.0 + ``` + +## Version gate + +Both publishing workflows call `.github/workflows/verify-versions.yml` as their +first job. It runs `scripts/verify_versions.sh` against the tag (minus the +`v`). If any of the four version sources doesn't match the tag, the verify job +fails and **all** publish jobs are skipped — no images, no chart. + +When it fails, the job annotation names the offending file and suggests the +fix: + +``` +::error::frontend/package.json is '2.1.0' but expected '2.2.0'. +Tip: run scripts/bump_version.sh 2.2.0 to align all sources. +``` + +## Pre-releases (RCs) + +Pre-release tags like `v2.2.0-rc1` are valid `v*` tags and trigger the same +workflows. The version sources must equal the full pre-release string +(`2.2.0-rc1`) — the gate compares exact strings. Use the same procedure with +the rc version: + +```bash +scripts/bump_version.sh 2.2.0-rc1 +# update CHANGELOG, commit, tag v2.2.0-rc1, push +``` + +## Recovering from a failed gate + +If the gate failed because a source was forgotten: + +1. Run `scripts/bump_version.sh ` to align the sources. +2. Amend or add a follow-up commit. +3. Delete and re-create the tag, then push it: + ```bash + git tag -d v2.2.0 + git tag v2.2.0 + git push origin :refs/tags/v2.2.0 + git push origin v2.2.0 + ``` + +Re-pushing the tag re-triggers the workflows. Because the gate blocks **all** +artifacts when it fails, nothing was published under the bad tag, so re-tagging +is safe — no images or chart were pushed to overwrite. + +## Post-release + +Optionally draft a **GitHub Release** from the tag, pasting the corresponding +`CHANGELOG.md` section as the release notes. The changelog link references +point at these release URLs. diff --git a/deploy/helm/deer-flow/Chart.yaml b/deploy/helm/deer-flow/Chart.yaml new file mode 100644 index 000000000..24d445e58 --- /dev/null +++ b/deploy/helm/deer-flow/Chart.yaml @@ -0,0 +1,15 @@ +apiVersion: v2 +name: deer-flow +description: DeerFlow — LangGraph-based AI super-agent (gateway + frontend + nginx + sandbox provisioner) deployed to Kubernetes. +type: application +version: 2.1.0 +appVersion: "2.1.0" +keywords: + - deerflow + - langgraph + - ai-agent + - llm +home: https://github.com/bytedance/deer-flow +icon: https://raw.githubusercontent.com/bytedance/deer-flow/main/frontend/public/images/deer.svg +maintainers: + - name: DeerFlow diff --git a/deploy/helm/deer-flow/README.md b/deploy/helm/deer-flow/README.md new file mode 100644 index 000000000..7a1ac815f --- /dev/null +++ b/deploy/helm/deer-flow/README.md @@ -0,0 +1,395 @@ +# DeerFlow Helm Chart + +Deploys the full DeerFlow stack to Kubernetes: **gateway** (backend + embedded +LangGraph runtime), **frontend** (Next.js), **nginx** (internal reverse proxy +preserving the compose routing), and the **provisioner** (K8s-native sandbox +that spawns code-execution Pods on demand). + +This chart translates the production `docker/docker-compose.yaml` into native +Kubernetes resources. No existing repo files are modified. + +## Prerequisites + +- A Kubernetes cluster (Docker Desktop K8s, OrbStack, kind, k3d, or a real cluster). +- `kubectl` + `helm` 3 installed. +- The three DeerFlow images — either the published ones (see "Install the + published chart" below) or built locally (see step 1). +- An Ingress controller (e.g. ingress-nginx) if you enable `ingress`. + +## Install the published chart (GHCR) + +The chart and all three images are published to GHCR on every `v*` release tag +(see `.github/workflows/container.yaml` and `chart.yaml`). Skip the build step +and install directly: + +```bash +helm install deer-flow oci://ghcr.io//deer-flow \ + --version \ + -n deer-flow --create-namespace \ + -f my-values.yaml +``` + +where `` is the GitHub owner the chart is published from and `` +matches the release tag without the leading `v` (tag `v0.1.0` → `--version +0.1.0`). Point the chart at the published images: + +```yaml +image: + registry: ghcr.io/ # owner prefix; images are /deer-flow- + tag: "" # match the release tag (sans leading `v`) + pullSecrets: + - { name: regcred } # only if the GHCR package is private +``` + +The chart's `gatewayImage` / `frontendImage` / `provisionerImage` defaults +already match the published image names (`deer-flow-backend`, +`deer-flow-frontend`, `deer-flow-provisioner`), so only `registry` and `tag` +are required. New GHCR packages default to **private** — flip the package to +public in its GHCR settings page for unauthenticated pulls, otherwise create a +pull secret (step 1) and reference it via `image.pullSecrets`. + +> The OCI chart and the images are versioned independently of the chart's +> `appVersion`; always set `image.tag` to the release that matches your chart +> `--version` unless you have a reason to pin differently. + +## 1. Build & push images (custom builds only) + +Skip this section if you're using the published chart above. To build the +images yourself from the existing Dockerfiles: + +```bash +REGISTRY=ghcr.io/yourorg TAG=latest ./deploy/helm/deer-flow/scripts/build-and-push.sh +``` + +This produces `$REGISTRY/deer-flow-backend`, `$REGISTRY/deer-flow-frontend`, +`$REGISTRY/deer-flow-provisioner`. The chart's image-name defaults match these. + +If your registry needs auth, create a pull secret: + +```bash +kubectl create secret docker-registry regcred \ + --docker-server=ghcr.io \ + --docker-username=youruser \ + --docker-password=yourtoken \ + -n deer-flow +``` + +## 2. Configure values + +Copy and edit `values.yaml` → `my-values.yaml`. At minimum set: + +```yaml +image: + registry: ghcr.io/yourorg + tag: latest + pullSecrets: + - { name: regcred } + +ingress: + enabled: true + className: nginx + host: deer-flow.example.com + tls: + enabled: true + secretName: deer-flow-tls + +secrets: + OPENAI_API_KEY: sk-... + # add channel tokens, search keys, etc. as needed +``` + +Provide your model config under `config` (keep secrets as `$VAR` references — +they resolve from the `secrets` map): + +```yaml +config: | + config_version: 20 + models: + - name: gpt-4 + use: langchain_openai:ChatOpenAI + model: gpt-4 + api_key: $OPENAI_API_KEY + request_timeout: 600.0 + sandbox: + use: deerflow.community.aio_sandbox:AioSandboxProvider + provisioner_url: http://provisioner:8002 + database: + backend: postgres + postgres_url: $DATABASE_URL + checkpointer: + type: postgres + connection_string: $DATABASE_URL + stream_bridge: + type: redis # cross-pod SSE; URL from DEER_FLOW_STREAM_BRIDGE_REDIS_URL + # Tools MUST be listed explicitly - the agent gets none otherwise + # (BUILTIN_TOOLS only adds present_file + ask_clarification). The chart + # default in values.yaml enables the sandbox tools + web tools (web_search, + # web_fetch, image_search - no API key); when you override `config:`, copy + # them in. Full list in values.yaml / config.example.yaml. The web tools need + # outbound egress from the gateway pod. + tool_groups: + - name: web + - name: file:read + - name: file:write + - name: bash + tools: + - name: web_search + group: web + use: deerflow.community.ddg_search.tools:web_search_tool + max_results: 5 + - name: web_fetch + group: web + use: deerflow.community.jina_ai.tools:web_fetch_tool + timeout: 10 + - name: image_search + group: web + use: deerflow.community.image_search.tools:image_search_tool + max_results: 5 + - name: bash + group: bash + use: deerflow.sandbox.tools:bash_tool + # also: ls, read_file, glob, grep, write_file, str_replace (see values.yaml) +``` + +`$DATABASE_URL` is injected from the postgres Secret (see below). The +`checkpointer:` section is required for multi-replica operation — the LangGraph +Store (cross-thread memory + thread list) reads it and does not fall back to +`database:`. `stream_bridge.type: redis` is the default and routes live SSE +events through the bundled redis StatefulSet (or `redis.external`). +Because `config:` is a single override blob, a partial `config:` replaces the +chart default entirely - keep the `tools:`/`tool_groups:` block (or the agent +will have no tools) and the `sandbox:`/`database:`/`checkpointer:`/`stream_bridge:` +sections shown above. + +## 3. Install (from a local chart checkout) + +For a custom build or local development, install from the chart directory: + +```bash +helm install deer-flow deploy/helm/deer-flow \ + -n deer-flow --create-namespace \ + -f my-values.yaml +``` + +## 4. Verify + +```bash +kubectl -n deer-flow get pods +kubectl -n deer-flow port-forward svc/nginx 2026:2026 +curl http://localhost:2026/health # gateway health via nginx +``` + +Hit the Ingress host (map it in `/etc/hosts` for local clusters) to load the UI. + +Provisioner sanity check: + +```bash +kubectl -n deer-flow exec deploy/deer-flow-provisioner -- curl -s localhost:8002/health +``` + +## Architecture notes + +- **PostgreSQL is the default database.** A bundled single-instance postgres + StatefulSet (`postgresql.enabled: true`) runs in the namespace and the gateway + connects via the in-cluster Service. The DSN is auto-generated into a Secret + (key `database-url`) and injected as `DATABASE_URL`; `config.yaml` references + it as `$DATABASE_URL` in `database.postgres_url`. Schema is bootstrapped + automatically on gateway startup (alembic `create_all` + `stamp head`). + For real HA, disable the bundled instance and point at a managed DB: + ```yaml + postgresql: + enabled: false + external: + host: mydb.example.com # or set databaseUrl / existingSecret + port: 5432 + database: deerflow + username: deerflow + password: changeme + ``` +- **Gateway replicas.** Postgres + the Redis stream bridge together make the + gateway's *persisted* state (checkpointer + run/thread metadata) and *live + stream* path cross-pod-safe. The default is still 1 replica: **do not raise + `gateway.replicas` past 1 yet.** Run control — `create_or_reject` dedup, + `cancel`, and orphan reconciliation — is still worker-local (in-process + `asyncio.Lock` + in-memory `record.task`), tracked by [issue + #3948](https://github.com/bytedance/deer-flow/issues/3948). With >1 replica a + double-submit can create two runs on one thread (checkpoint corruption), a + cancel can land on a non-owner pod (409), and a crashed pod's runs stay + `pending`/`running` forever. Stay on 1 replica until that work lands. +- **Redis stream bridge.** A bundled single-instance redis StatefulSet + (`redis.enabled: true`, `redis:7-alpine`) runs in the namespace and the + gateway connects via the in-cluster Service. Per-run SSE events are stored in + Redis Streams (PR #3191) so a client connected to any gateway pod receives + live events and reconnect resumes from `Last-Event-ID`. The URL is + auto-generated into a Secret (key `redis-url`) and injected as + `DEER_FLOW_STREAM_BRIDGE_REDIS_URL`; `config.yaml` sets `stream_bridge.type: + redis` by default. No-auth by default (ClusterIP isolation, matching compose); + set `redis.auth.password` to enable AUTH. For a managed Redis, disable the + bundled instance and point at it via `redis.external`. +- **Persistence.** A PVC (`-home`) backs `/app/backend/.deer-flow` + (sqlite DB, memory, custom agents, per-thread user-data). The gateway mounts + it with `subPath: deer-flow` so the layout matches the provisioner's PVC + user-data mode. Default `ReadWriteOnce`; use `ReadWriteMany` (NFS) on + multi-node clusters so sandbox Pods on other nodes can mount it. +- **Provisioner RBAC.** The provisioner gets a ServiceAccount with a namespaced + Role (get/list/watch/create/delete on pods + services) and a narrow ClusterRole + (namespace get/create). It uses in-cluster service-account creds — no + kubeconfig mount. The unused update/patch/pods-exec/events verbs were dropped + (audited against `docker/provisioner/app.py`). +- **Skills.** Disabled by default (emptyDir at `/app/skills`). Populate via + `skills.existingClaim` or `skills.configMap`, or bake skills into a custom + gateway image. + +## Security + +### Enforced posture + +All workloads run as **non-root** with **all Linux capabilities dropped**. No +container escalates privileges or runs as uid 0. + +| workload | runAsUser | fsGroup | writable-path handling | +|---|---|---|---| +| gateway | 1000 | 1000 | `.deer-flow` PVC group-writable via fsGroup; `PYTHONDONTWRITEBYTECODE=1` suppresses `.pyc` writes; `UV_CACHE_DIR=/tmp` | +| frontend | 1000 (`node`) | 1000 | `emptyDir` at `/app/frontend/.next/cache` (root-owned in the image) | +| nginx | 101 (`nginx`) | 101 | command writes the rendered config to `/tmp/nginx.conf` and loads `nginx -c /tmp/nginx.conf` (since `/etc/nginx` is root-owned); `emptyDir` at `/var/cache/nginx` | +| provisioner | 1000 | — | no PVC; `PYTHONDONTWRITEBYTECODE=1` | +| postgres | 999 (`postgres`) | 999 | official `postgres:16` entrypoint detects non-root and skips the chown/gosu dance; data PVC group-writable via fsGroup | +| redis | 999 (`redis`) | 999 | official `redis:7-alpine` entrypoint detects non-root and skips the gosu dance; data PVC group-writable via fsGroup | + +Every container sets: + +- `runAsNonRoot: true` +- `allowPrivilegeEscalation: false` +- `capabilities.drop: ["ALL"]` +- `seccompProfile: { type: RuntimeDefault }` + +All listening ports are >1024 (8001 / 3000 / 2026 / 8002 / 5432), so no +`NET_BIND_SERVICE` capability is required. + +**ConfigMap rollout.** ConfigMaps mount via `subPath`, which does **not** receive +in-place updates — a `helm upgrade` that changes only a ConfigMap would leave +pods on stale config. Each pod template carries a `checksum/*` annotation (SHA256 +of the rendered ConfigMap): `checksum/config` + `checksum/extensions` on the +gateway, `checksum/nginx` on nginx. Any content change alters the pod spec and +triggers a rolling restart. + +**Resource defaults.** Every workload ships with modest requests+limits in +`values.yaml`; override per workload (`gateway.resources`, `frontend.resources`, +`nginx.resources`, `provisioner.resources`, `postgresql.primary.resources`, +`redis.primary.resources`). + +### Not yet enforced (deferred hardening) + +These are intentionally **not** set in this chart revision. Each can be added +per-workload with testing: + +- **`readOnlyRootFilesystem: true`** — makes the container's root filesystem + immutable so a compromised process can't persist changes to the image. Not + enabled because it requires auditing every runtime write path and mounting an + `emptyDir` over each. Known paths: + - gateway / frontend / nginx / provisioner: `/tmp` (uv cache, python tempfiles, + the nginx config + pid, node temp) — one `emptyDir` at `/tmp` each. + - postgres: `/tmp` **and** `/var/run/postgresql` (the Unix-socket dir). + The first four are mechanical. **postgres is the hard case** — the official + image writes its socket to `/var/run/postgresql` and isn't designed for a + read-only root, so it may need socket-path redirection (`PGHOST`/`unix_socket_directories`). + Optionally, add `USER` directives to the `backend/Dockerfile`, + `frontend/Dockerfile`, and `docker/provisioner/Dockerfile` so the images are + non-root by default (defense in depth — the chart already forces the uid via + `securityContext`, so this is not required). A cluster enforcing the + `restricted` Pod Security Admission standard would require this setting. +- **Provisioner RBAC narrowing.** The Role grants get/list/watch/create/delete + on pods and services in the namespace (update/patch/pods-exec/events were + dropped as unused). These verbs still apply to *all* Pods in the namespace, + not just sandbox Pods — RBAC can't scope by label, so the remaining + options are a dedicated sandbox namespace or admission control (OPA/Kyverno). +- **`startupProbe`.** Workloads have readiness + liveness probes but no startup + probe. The gateway's `livenessProbe.initialDelaySeconds: 30` covers slow starts + today; a `startupProbe` would let it take arbitrarily long to initialize + without risking a liveness kill during a cold start (e.g. slow model config + load). + +None of these affect correctness of the current deployment. + +### Migrating an existing volume to non-root + +`fsGroup` does **not** apply to `subPath` mounts, and it changes group ownership +but not file mode — so a PVC written by an earlier **root** run (e.g. a cluster +that ran the gateway as root before enabling this hardening, or a backup restore +of root-owned files) will keep files like `.jwt_secret` at `0600 root:root`. The +non-root gateway (uid 1000) then can't read them and crashes on the first auth +request with `RuntimeError: Failed to read JWT secret from .../​.jwt_secret`. + +**Fresh installs are unaffected** — uid 1000 creates every file as `1000:1000`. + +To fix an existing root-written PVC, run a one-shot root pod that chowns the +volume to the gateway uid (1000), then restart the gateway: + +```bash +cat <<'EOF' | kubectl apply -n deer-flow -f - +apiVersion: v1 +kind: Pod +metadata: { name: fix-home-perms, namespace: deer-flow } +spec: + restartPolicy: Never + containers: + - name: chown + image: busybox:1.36 + command: ["sh", "-c"] + args: ["chown -R 1000:1000 /home-pvc/deer-flow && chmod -R g+rwX /home-pvc/deer-flow"] + volumeMounts: + - { name: home, mountPath: /home-pvc } + volumes: + - name: home + persistentVolumeClaim: { claimName: deer-flow-deer-flow-home } +EOF +kubectl -n deer-flow wait --for=condition=Ready pod/fix-home-perms --timeout=30s +kubectl -n deer-flow delete pod fix-home-perms +kubectl -n deer-flow rollout restart deploy/deer-flow-deer-flow-gateway +``` + +(On a single-node cluster the fix pod can mount the RWO PVC concurrently with the +gateway; on multi-node, scale the gateway to 0 first.) A durable alternative — +an opt-in root `volumePermissions` initContainer that chowns on every start (the +Bitnami pattern) — is not yet wired into this chart; it would introduce a root +container, so it's left as an operator decision for now. + +## Sandbox NodePort reachability + +The provisioner returns `http://{NODE_HOST}:{NodePort}` to the gateway so the +agent can reach its sandbox. In Docker Compose `NODE_HOST=host.docker.internal`; +in Kubernetes `NODE_HOST` **defaults to the provisioner pod's node IP** via the +[downward API](https://kubernetes.io/docs/concepts/workloads/pods/downward-api/) +(`status.hostIP`). Because a NodePort is exposed on every node, the gateway can +reach `:` on most clusters without any configuration. + +Override `provisioner.nodeHost` only if your CNI or network policy blocks +pod->node-IP traffic: + +```bash +kubectl get nodes -o wide # use INTERNAL-IP or EXTERNAL-IP +``` + +```yaml +provisioner: + nodeHost: 192.168.x.x +``` + +On multi-node clusters, also switch `persistence.home.accessMode` to +`ReadWriteMany`. + +## Lint / dry-run + +```bash +helm lint deploy/helm/deer-flow +helm template deer-flow deploy/helm/deer-flow -n deer-flow -f my-values.yaml | \ + kubectl apply --dry-run=client -f - +``` + +## Uninstall + +```bash +helm uninstall deer-flow -n deer-flow +# the PVC is NOT deleted by default — remove it manually if desired: +kubectl -n deer-flow delete pvc -l app.kubernetes.io/instance=deer-flow +``` diff --git a/deploy/helm/deer-flow/templates/NOTES.txt b/deploy/helm/deer-flow/templates/NOTES.txt new file mode 100644 index 000000000..2453c67ab --- /dev/null +++ b/deploy/helm/deer-flow/templates/NOTES.txt @@ -0,0 +1,133 @@ +DeerFlow has been deployed. + + namespace: {{ include "deer-flow.namespace" . }} + +{{- if not .Values.image.registry }} + + ⚠️ WARNING: `image.registry` is empty. The gateway/frontend/provisioner + images won't resolve. Set `image.registry` in your values and build+push the + three images (see deploy/helm/deer-flow/scripts/build-and-push.sh). +{{- end }} + +Get the status: + + kubectl -n {{ include "deer-flow.namespace" . }} get pods + +Quick port-forward to nginx (bypasses Ingress): + + kubectl -n {{ include "deer-flow.namespace" . }} port-forward svc/nginx 2026:2026 + # then open http://localhost:2026 + +{{- if .Values.ingress.enabled }} + +Ingress is enabled for host {{ .Values.ingress.host }}. Ensure your Ingress +controller is installed and DNS/hosts maps the host to the controller. +{{- if not .Values.ingress.tls.enabled }} +TLS is disabled — enable `ingress.tls` and provide a secret (or cert-manager) +for HTTPS. +{{- end }} +{{- end }} + +Provisioner / sandbox notes: + + • The provisioner ServiceAccount has a namespaced Role (get, list, watch, + create, delete on Pods + Services incl. pods/log, in this namespace) and a + narrow ClusterRole (namespace get + create, cluster-wide). These verbs + apply to *all* Pods in the namespace, not just sandbox Pods - scoping to + sandbox Pods is deferred hardening (see README "Not yet enforced"). + • Sandbox Pods are created in namespace {{ include "deer-flow.namespace" . }}. + • PVC mode is enabled: sandbox Pods mount the `{{ include "deer-flow.homePVC" . }}` + PVC for per-thread user-data. + + ⚠️ Sandbox NodePort reachability: + The provisioner returns `http://{NODE_HOST}:{NodePort}` to the gateway so + the agent can talk to its sandbox. NODE_HOST defaults to the provisioner + pod's node IP via the Kubernetes downward API, which routes on most clusters + because a NodePort is exposed on every node. + + If your CNI or network policy blocks pod->node-IP traffic, set + `provisioner.nodeHost` to an address the gateway can reach that routes to + a node IP: + + kubectl get nodes -o wide + + provisioner: + nodeHost: + + On multi-node clusters you additionally need `persistence.home.accessMode: + ReadWriteMany` (e.g. NFS) so sandbox Pods on other nodes can mount the + shared user-data PVC. + +Generated secrets (persisted across upgrades): + + • BETTER_AUTH_SECRET and DEER_FLOW_INTERNAL_AUTH_TOKEN are stored in the + Secret `{{ include "deer-flow.appSecret" . }}`. + +Database ({{ if .Values.postgresql.enabled }}bundled postgres{{ else }}external postgres{{ end }}): + +{{- if .Values.postgresql.enabled }} + • A postgres StatefulSet (`{{ include "deer-flow.postgresFullname" . }}`) + is deployed in this namespace. The gateway connects via the in-cluster + Service `{{ include "deer-flow.postgresFullname" . }}:5432`. + • DATABASE_URL is in Secret `{{ include "deer-flow.databaseUrlSecret" . }}` + (key `database-url`); the auto-generated password is in key + `postgres-password`. Both persist across upgrades. + • Schema is bootstrapped automatically on gateway startup (alembic + create_all + stamp head). No manual migration step needed. + • To run real HA, disable this (`postgresql.enabled: false`) and point at a + managed DB via `postgresql.external`. +{{- else }} + • The gateway reads DATABASE_URL from Secret + `{{ include "deer-flow.databaseUrlSecret" . }}` (key `database-url`). + Ensure that Secret exists and contains your managed-postgres DSN. +{{- end }} + +Redis stream bridge ({{ if .Values.redis.enabled }}bundled redis{{ else if (include "deer-flow.redisConfigured" .) }}external redis{{ else }}none{{ end }}): + +{{- if .Values.redis.enabled }} + • A redis StatefulSet (`{{ include "deer-flow.redisFullname" . }}`) is deployed + in this namespace. The gateway connects via the in-cluster Service + `{{ include "deer-flow.redisFullname" . }}:6379`. + • DEER_FLOW_STREAM_BRIDGE_REDIS_URL is in Secret + `{{ include "deer-flow.redisUrlSecret" . }}` (key `redis-url`). + • Per-run SSE events are stored in Redis Streams so a client connected to any + gateway pod receives live events; reconnect resumes from Last-Event-ID. + • To use a managed Redis, disable this (`redis.enabled: false`) and point at it + via `redis.external`. +{{- else if (include "deer-flow.redisConfigured" .) }} + • The gateway reads DEER_FLOW_STREAM_BRIDGE_REDIS_URL from Secret + `{{ include "deer-flow.redisUrlSecret" . }}` (key `redis-url`). Ensure that + Secret exists and contains your managed-Redis URL. +{{- else }} + • No redis configured — the gateway falls back to the in-process memory stream + bridge. This is single-pod only: cross-pod SSE delivery and reconnect will + not work with `gateway.replicas > 1`. +{{- end }} + +{{- if or .Values.secrets .Values.existingSecret }} + +Provider/channel keys are in Secret `{{ include "deer-flow.providerSecret" . }}` +and injected into the gateway via envFrom. Reference them from config.yaml as +$VAR. +{{- else }} + + ⚠️ No provider secrets configured. Add at least one model under `config` + (e.g. an OpenAI model with `api_key: $OPENAI_API_KEY`) and supply the key + under `secrets` in your values, or the agent will have no LLM to call. +{{- end }} + +Agent tools: the chart enables the sandbox tools (ls, read_file, glob, grep, +write_file, str_replace, bash - run inside the AIO sandbox) and web tools +(web_search, web_fetch, image_search - no API key) by default. Add or swap +tools under `config` -> `tools:` (see config.example.yaml); the web tools need +outbound internet from the gateway pod. + +Pod security: + • All pods run non-root: gateway/frontend/provisioner uid 1000, nginx uid 101, + postgres uid 999. All Linux capabilities are dropped and privilege escalation + is disabled on every container. + • ConfigMap changes roll pods automatically (checksum annotations on the + gateway and nginx pod templates). + • See README's "Security" section for the full posture and the deferred + hardening items (readOnlyRootFilesystem, provisioner RBAC narrowing, + startupProbe). diff --git a/deploy/helm/deer-flow/templates/_helpers.tpl b/deploy/helm/deer-flow/templates/_helpers.tpl new file mode 100644 index 000000000..45b9d3ef3 --- /dev/null +++ b/deploy/helm/deer-flow/templates/_helpers.tpl @@ -0,0 +1,155 @@ +{{/* +Common helpers for the DeerFlow chart. +*/}} + +{{- define "deer-flow.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{- define "deer-flow.fullname" -}} +{{- $name := default .Chart.Name .Values.nameOverride -}} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{- define "deer-flow.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{- define "deer-flow.labels" -}} +helm.sh/chart: {{ include "deer-flow.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "deer-flow.selectorLabels" -}} +app.kubernetes.io/name: {{ include "deer-flow.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "deer-flow.namespace" -}} +{{- default .Release.Namespace .Values.namespace -}} +{{- end -}} + +{{- define "deer-flow.imagePullSecrets" -}} +{{- with .Values.image.pullSecrets }} +imagePullSecrets: +{{- toYaml . | nindent 0 }} +{{- end }} +{{- end -}} + +{{/* Fully-qualified image refs for the three DeerFlow images. + When `image.registry` is empty, omit the prefix so the ref is + `deer-flow-gateway:latest` (local-image mode, imagePullPolicy: Never). */}} +{{- define "deer-flow.gatewayImage" -}} +{{- if .Values.image.registry -}}{{- printf "%s/%s:%s" .Values.image.registry .Values.image.gatewayImage .Values.image.tag -}} +{{- else -}}{{- printf "%s:%s" .Values.image.gatewayImage .Values.image.tag -}}{{- end -}} +{{- end -}} + +{{- define "deer-flow.frontendImage" -}} +{{- if .Values.image.registry -}}{{- printf "%s/%s:%s" .Values.image.registry .Values.image.frontendImage .Values.image.tag -}} +{{- else -}}{{- printf "%s:%s" .Values.image.frontendImage .Values.image.tag -}}{{- end -}} +{{- end -}} + +{{- define "deer-flow.provisionerImage" -}} +{{- if .Values.image.registry -}}{{- printf "%s/%s:%s" .Values.image.registry .Values.image.provisionerImage .Values.image.tag -}} +{{- else -}}{{- printf "%s:%s" .Values.image.provisionerImage .Values.image.tag -}}{{- end -}} +{{- end -}} + +{{- define "deer-flow.nginxImage" -}} +{{- printf "%s:%s" .Values.nginx.image.repository .Values.nginx.image.tag -}} +{{- end -}} + +{{/* PVC name for the .deer-flow home directory. */}} +{{- define "deer-flow.homePVC" -}} +{{- printf "%s-home" (include "deer-flow.fullname" .) -}} +{{- end -}} + +{{/* Name of the Secret holding provider/channel keys. */}} +{{- define "deer-flow.providerSecret" -}} +{{- if .Values.existingSecret -}}{{- .Values.existingSecret -}} +{{- else -}}{{- printf "%s-provider" (include "deer-flow.fullname" .) -}}{{- end -}} +{{- end -}} + +{{/* Name of the Secret holding generated app secrets (auth token, better-auth). */}} +{{- define "deer-flow.appSecret" -}} +{{- printf "%s-app" (include "deer-flow.fullname" .) -}} +{{- end -}} + +{{/* Name of the postgres StatefulSet/Service. */}} +{{- define "deer-flow.postgresFullname" -}} +{{- printf "%s-postgres" (include "deer-flow.fullname" .) -}} +{{- end -}} + +{{/* Name of the Secret holding DATABASE_URL (and, in bundled mode, the + postgres superuser password). Resolution order: + 1. postgresql.external.existingSecret (user-managed, key=database-url) + 2. postgresql.existingSecret (user-managed, bundled image) + 3. chart-managed secret `-postgres` + Only #3 is created by this chart; #1/#2 must exist already. */}} +{{- define "deer-flow.databaseUrlSecret" -}} +{{- if .Values.postgresql.external.existingSecret -}}{{- .Values.postgresql.external.existingSecret -}} +{{- else if .Values.postgresql.existingSecret -}}{{- .Values.postgresql.existingSecret -}} +{{- else -}}{{- include "deer-flow.postgresFullname" . -}}{{- end -}} +{{- end -}} + +{{/* Name of the redis StatefulSet/Service. */}} +{{- define "deer-flow.redisFullname" -}} +{{- printf "%s-redis" (include "deer-flow.fullname" .) -}} +{{- end -}} + +{{/* Name of the Secret holding the redis stream-bridge URL (key `redis-url`, + plus `redis-password` in bundled mode when a password is set). Resolution: + 1. redis.external.existingSecret (user-managed, key=redis-url) + 2. redis.existingSecret (user-managed, bundled image) + 3. chart-managed secret `-redis` + Only #3 is created by this chart; #1/#2 must exist already. */}} +{{- define "deer-flow.redisUrlSecret" -}} +{{- if .Values.redis.external.existingSecret -}}{{- .Values.redis.external.existingSecret -}} +{{- else if .Values.redis.existingSecret -}}{{- .Values.redis.existingSecret -}} +{{- else -}}{{- include "deer-flow.redisFullname" . -}}{{- end -}} +{{- end -}} + +{{/* Whether any redis stream-bridge backend is configured (bundled StatefulSet, + external URL, or a user-managed Secret). Drives the env injection in the + gateway deployment. */}} +{{- define "deer-flow.redisConfigured" -}} +{{- or .Values.redis.enabled .Values.redis.external.redisUrl .Values.redis.external.existingSecret .Values.redis.existingSecret -}} +{{- end -}} + +{{/* SHA256 checksums of the ConfigMaps. Mount these as pod-template + annotations: ConfigMaps mounted via subPath do NOT receive live updates, + so a `helm upgrade` that only changes a ConfigMap would leave pods on stale + config. A checksum annotation makes any content change alter the pod spec, + which triggers a rolling restart. */}} +{{- define "deer-flow.configChecksum" -}} +{{- include (print $.Template.BasePath "/configmap-config.yaml") . | sha256sum -}} +{{- end -}} + +{{- define "deer-flow.extensionsChecksum" -}} +{{- include (print $.Template.BasePath "/configmap-extensions.yaml") . | sha256sum -}} +{{- end -}} + +{{- define "deer-flow.nginxChecksum" -}} +{{- include (print $.Template.BasePath "/configmap-nginx.yaml") . | sha256sum -}} +{{- end -}} + +{{/* Percent-encode a string for safe interpolation into a URL userinfo + (password) segment of a DSN. Sprig lacks urlqueryescape, and + regexReplaceAllLiteral treats `replacement` as a regex template so chars + like `[`, `]`, `?` break it - so we chain plain `replace` calls instead. + `%` is encoded first to avoid double-encoding the percent signs emitted + for the other characters. Covers the URL-special chars a managed-DB + password might contain (`@ : / # ? % [ ]` and space). */}} +{{- define "deer-flow.urlEscape" -}} +{{- $s := . -}} +{{- $s = replace "%" "%25" $s -}} +{{- $s = replace "@" "%40" $s -}} +{{- $s = replace ":" "%3A" $s -}} +{{- $s = replace "/" "%2F" $s -}} +{{- $s = replace "#" "%23" $s -}} +{{- $s = replace "?" "%3F" $s -}} +{{- $s = replace "[" "%5B" $s -}} +{{- $s = replace "]" "%5D" $s -}} +{{- $s = replace " " "%20" $s -}} +{{- $s -}} +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/configmap-config.yaml b/deploy/helm/deer-flow/templates/configmap-config.yaml new file mode 100644 index 000000000..cb1da259a --- /dev/null +++ b/deploy/helm/deer-flow/templates/configmap-config.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "deer-flow.fullname" . }}-config + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} +data: + config.yaml: | +{{ .Values.config | default "" | indent 4 }} diff --git a/deploy/helm/deer-flow/templates/configmap-extensions.yaml b/deploy/helm/deer-flow/templates/configmap-extensions.yaml new file mode 100644 index 000000000..1af01e57b --- /dev/null +++ b/deploy/helm/deer-flow/templates/configmap-extensions.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "deer-flow.fullname" . }}-extensions + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} +data: + extensions_config.json: | +{{ .Values.extensionsConfig | default "" | indent 4 }} diff --git a/deploy/helm/deer-flow/templates/configmap-nginx.yaml b/deploy/helm/deer-flow/templates/configmap-nginx.yaml new file mode 100644 index 000000000..4fc4257d3 --- /dev/null +++ b/deploy/helm/deer-flow/templates/configmap-nginx.yaml @@ -0,0 +1,225 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "deer-flow.fullname" . }}-nginx + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} +data: + nginx.conf: | + events { + worker_connections 1024; + } + pid /tmp/nginx.pid; + http { + sendfile on; + tcp_nopush on; + tcp_nodelay on; + keepalive_timeout 65; + types_hash_max_size 2048; + + access_log /dev/stdout; + error_log /dev/stderr; + + # Preserve an upstream proxy's X-Forwarded-Proto so the Gateway sees the + # real client scheme when nginx runs behind an Ingress/TLS terminator. + map $http_x_forwarded_proto $forwarded_proto { + default $scheme; + "~*^https" https; + } + + # K8s Services resolve at config-parse time (no resolver directive needed). + upstream gateway_upstream { + server gateway:8001; + keepalive 32; + } + upstream frontend_upstream { + server frontend:3000; + keepalive 32; + } + {{- if .Values.provisioner.enabled }} + upstream provisioner_upstream { + server provisioner:8002; + keepalive 16; + } + {{- end }} + + server { + listen 2026 default_server; + listen [::]:2026 default_server; + server_name _; + + proxy_buffering off; + proxy_cache off; + + # Static liveness endpoint — always 200 if nginx itself is alive. + # Decouples the liveness probe from gateway availability so nginx + # is not restart-looped while the gateway pulls its image or is + # otherwise down. Readiness still proxies /health to the gateway so + # traffic is not routed to an nginx that cannot reach it. + location = /nginx-health { + access_log off; + default_type text/plain; + return 200 "ok"; + } + + # LangGraph-compatible API routes (rewrite /api/langgraph/* -> /api/*). + location /api/langgraph/ { + rewrite ^/api/langgraph/(.*) /api/$1 break; + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + proxy_set_header Connection ''; + proxy_set_header X-Accel-Buffering no; + proxy_connect_timeout 600s; + proxy_send_timeout 600s; + proxy_read_timeout 600s; + chunked_transfer_encoding on; + } + + location /api/models { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + location /api/memory { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + location /api/mcp { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + location /api/skills { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + location /api/agents { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + # Uploads — large bodies, no request buffering. + location ~ ^/api/threads/[^/]+/uploads { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + client_max_body_size 100M; + proxy_request_buffering off; + } + + location ~ ^/api/threads { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + location /docs { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + location /redoc { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + location /openapi.json { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + location /health { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + # Provisioner (sandbox management). Omitted when the provisioner is + # disabled; /api/sandboxes then falls through to the catch-all /api/ + # below and the gateway answers (404 / "sandbox not configured"). + {{- if .Values.provisioner.enabled }} + location /api/sandboxes { + proxy_pass http://provisioner_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + {{- end }} + + # Catch-all for other /api/ routes (e.g. /api/v1/auth/*). + location /api/ { + proxy_pass http://gateway_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + } + + # Everything else -> frontend (with WebSocket upgrade for HMR/sockets). + location / { + proxy_pass http://frontend_upstream; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $forwarded_proto; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection 'upgrade'; + proxy_cache_bypass $http_upgrade; + proxy_connect_timeout 600s; + proxy_send_timeout 600s; + proxy_read_timeout 600s; + } + } + } diff --git a/deploy/helm/deer-flow/templates/frontend-deployment.yaml b/deploy/helm/deer-flow/templates/frontend-deployment.yaml new file mode 100644 index 000000000..ff4c06f0a --- /dev/null +++ b/deploy/helm/deer-flow/templates/frontend-deployment.yaml @@ -0,0 +1,75 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "deer-flow.fullname" . }}-frontend + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: frontend +spec: + replicas: {{ .Values.frontend.replicas }} + selector: + matchLabels: + {{- include "deer-flow.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: frontend + template: + metadata: + labels: + {{- include "deer-flow.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: frontend + spec: + {{- include "deer-flow.imagePullSecrets" . | nindent 6 }} + securityContext: + # Non-root: node:22-alpine ships a `node` user (uid 1000). fsGroup 1000 + # is harmless here (no PVC) but keeps the pod spec uniform. + runAsNonRoot: true + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 + seccompProfile: + type: RuntimeDefault + containers: + - name: frontend + image: {{ include "deer-flow.frontendImage" . }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + workingDir: /app/frontend + args: ["pnpm", "start"] + ports: + - name: http + containerPort: 3000 + env: + - name: NODE_ENV + value: production + - name: DEER_FLOW_INTERNAL_GATEWAY_BASE_URL + value: http://gateway:8001 + - name: BETTER_AUTH_SECRET + valueFrom: + secretKeyRef: + name: {{ include "deer-flow.appSecret" . }} + key: BETTER_AUTH_SECRET + readinessProbe: + tcpSocket: + port: http + initialDelaySeconds: 5 + periodSeconds: 10 + livenessProbe: + tcpSocket: + port: http + initialDelaySeconds: 30 + periodSeconds: 20 + {{- with .Values.frontend.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + # Next.js writes runtime cache to .next/cache; the dir is root-owned + # in the prod image, so mount an emptyDir to make it writable by 1000. + - name: next-cache + mountPath: /app/frontend/.next/cache + volumes: + - name: next-cache + emptyDir: {} diff --git a/deploy/helm/deer-flow/templates/frontend-service.yaml b/deploy/helm/deer-flow/templates/frontend-service.yaml new file mode 100644 index 000000000..b1dec842d --- /dev/null +++ b/deploy/helm/deer-flow/templates/frontend-service.yaml @@ -0,0 +1,17 @@ +apiVersion: v1 +kind: Service +metadata: + name: frontend + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: frontend +spec: + type: ClusterIP + ports: + - name: http + port: 3000 + targetPort: http + selector: + {{- include "deer-flow.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: frontend diff --git a/deploy/helm/deer-flow/templates/gateway-deployment.yaml b/deploy/helm/deer-flow/templates/gateway-deployment.yaml new file mode 100644 index 000000000..6227ebc93 --- /dev/null +++ b/deploy/helm/deer-flow/templates/gateway-deployment.yaml @@ -0,0 +1,190 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "deer-flow.fullname" . }}-gateway + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: gateway +spec: + replicas: {{ .Values.gateway.replicas }} + selector: + matchLabels: + {{- include "deer-flow.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: gateway + template: + metadata: + labels: + {{- include "deer-flow.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: gateway + annotations: + # Roll pods when the mounted ConfigMaps change (subPath mounts don't + # get live updates, so without this a config-only upgrade is ignored). + checksum/config: {{ include "deer-flow.configChecksum" . }} + checksum/extensions: {{ include "deer-flow.extensionsChecksum" . }} + spec: + {{- include "deer-flow.imagePullSecrets" . | nindent 6 }} + securityContext: + # Non-root: the gateway image has no USER directive and would otherwise + # run as root. uid/gid 1000; fsGroup 1000 so the .deer-flow PVC is + # group-writable for memory/sqlite/custom-agent state. + runAsNonRoot: true + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 + seccompProfile: + type: RuntimeDefault + # initContainer ensures the subPath `deer-flow/` exists on the PV so the + # volumeMount succeeds on first boot, and the PVC layout matches what the + # provisioner's PVC user-data mode expects (deer-flow/users/.../user-data). + {{- if .Values.persistence.home.enabled }} + initContainers: + - name: init-home + image: busybox:1.36 + command: ["sh", "-c", "mkdir -p /home-pvc/deer-flow/data"] + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - name: home + mountPath: /home-pvc + {{- end }} + containers: + - name: gateway + image: {{ include "deer-flow.gatewayImage" . }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + workingDir: /app + command: ["sh", "-c"] + args: + - cd backend && PYTHONPATH=. uv run --no-sync uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 --workers 1 + env: + - name: CI + value: "true" + # Running as non-root uid 1000: suppress .pyc writes under the + # root-owned /app (avoids PermissionError noise; harmless fallback). + - name: PYTHONDONTWRITEBYTECODE + value: "1" + # uv would cache to ~/.cache (no home dir as 1000); point it at /tmp + # (1777). --no-sync means no install/cache writes in practice. + - name: UV_CACHE_DIR + value: /tmp + - name: DEER_FLOW_PROJECT_ROOT + value: /app + - name: DEER_FLOW_HOME + value: /app/backend/.deer-flow + - name: DEER_FLOW_CONFIG_PATH + value: /app/backend/config.yaml + - name: DEER_FLOW_EXTENSIONS_CONFIG_PATH + value: /app/backend/extensions_config.json + - name: DEER_FLOW_CHANNELS_LANGGRAPH_URL + value: http://gateway:8001/api + - name: DEER_FLOW_CHANNELS_GATEWAY_URL + value: http://gateway:8001 + - name: DEER_FLOW_HOST_BASE_DIR + value: /app/backend/.deer-flow + - name: DEER_FLOW_HOST_SKILLS_PATH + value: /app/skills + - name: GATEWAY_HOST + value: 0.0.0.0 + - name: GATEWAY_PORT + value: "8001" + - name: DEER_FLOW_INTERNAL_AUTH_TOKEN + valueFrom: + secretKeyRef: + name: {{ include "deer-flow.appSecret" . }} + key: DEER_FLOW_INTERNAL_AUTH_TOKEN + {{- /* PostgreSQL DSN — only mounted when postgres is configured + (bundled StatefulSet, external databaseUrl, or an existing + Secret). In sqlite/memory mode the env is omitted and + config.yaml's database.postgres_url is never read. */ -}} + {{- $pgConfigured := or .Values.postgresql.enabled .Values.postgresql.external.databaseUrl .Values.postgresql.external.existingSecret .Values.postgresql.existingSecret -}} + {{- if $pgConfigured }} + - name: DATABASE_URL + valueFrom: + secretKeyRef: + name: {{ include "deer-flow.databaseUrlSecret" . }} + key: database-url + {{- end }} + {{- /* Redis stream-bridge URL — only mounted when redis is configured + (bundled StatefulSet, external URL, or an existing Secret). + Drives cross-pod SSE delivery (PR #3191). redis-py connects via + raw TCP so HTTP_PROXY/NO_PROXY do not apply. */ -}} + {{- if (include "deer-flow.redisConfigured" .) }} + - name: DEER_FLOW_STREAM_BRIDGE_REDIS_URL + valueFrom: + secretKeyRef: + name: {{ include "deer-flow.redisUrlSecret" . }} + key: redis-url + {{- end }} + - name: NO_PROXY + value: localhost,127.0.0.1,::1,gateway,frontend,nginx,provisioner + - name: no_proxy + value: localhost,127.0.0.1,::1,gateway,frontend,nginx,provisioner + {{- if or .Values.secrets .Values.existingSecret }} + envFrom: + - secretRef: + name: {{ include "deer-flow.providerSecret" . }} + {{- end }} + ports: + - containerPort: 8001 + name: http + readinessProbe: + httpGet: + path: /health + port: http + initialDelaySeconds: 5 + periodSeconds: 10 + livenessProbe: + httpGet: + path: /health + port: http + initialDelaySeconds: 30 + periodSeconds: 20 + {{- with .Values.gateway.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: config + mountPath: /app/backend/config.yaml + subPath: config.yaml + readOnly: true + - name: extensions + mountPath: /app/backend/extensions_config.json + subPath: extensions_config.json + readOnly: true + - name: skills + mountPath: /app/skills + readOnly: true + {{- if .Values.persistence.home.enabled }} + - name: home + mountPath: /app/backend/.deer-flow + subPath: deer-flow + {{- end }} + volumes: + - name: config + configMap: + name: {{ include "deer-flow.fullname" . }}-config + - name: extensions + configMap: + name: {{ include "deer-flow.fullname" . }}-extensions + - name: skills + {{- if .Values.skills.existingClaim }} + persistentVolumeClaim: + claimName: {{ .Values.skills.existingClaim }} + {{- else if .Values.skills.configMap }} + configMap: + name: {{ .Values.skills.configMap }} + {{- else }} + emptyDir: {} + {{- end }} + {{- if .Values.persistence.home.enabled }} + - name: home + persistentVolumeClaim: + claimName: {{ include "deer-flow.homePVC" . }} + {{- end }} diff --git a/deploy/helm/deer-flow/templates/gateway-service.yaml b/deploy/helm/deer-flow/templates/gateway-service.yaml new file mode 100644 index 000000000..0db74bde6 --- /dev/null +++ b/deploy/helm/deer-flow/templates/gateway-service.yaml @@ -0,0 +1,17 @@ +apiVersion: v1 +kind: Service +metadata: + name: gateway + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: gateway +spec: + type: ClusterIP + ports: + - name: http + port: 8001 + targetPort: http + selector: + {{- include "deer-flow.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: gateway diff --git a/deploy/helm/deer-flow/templates/ingress.yaml b/deploy/helm/deer-flow/templates/ingress.yaml new file mode 100644 index 000000000..cc5e641c4 --- /dev/null +++ b/deploy/helm/deer-flow/templates/ingress.yaml @@ -0,0 +1,37 @@ +{{- if .Values.ingress.enabled -}} +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: {{ include "deer-flow.fullname" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + {{- with .Values.ingress.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + {{- if .Values.ingress.className }} + ingressClassName: {{ .Values.ingress.className | quote }} + {{- end }} + {{- if .Values.ingress.tls.enabled }} + tls: + - hosts: + {{- $hosts := .Values.ingress.tls.hosts | default (list .Values.ingress.host) }} + {{- toYaml $hosts | nindent 8 }} + {{- if .Values.ingress.tls.secretName }} + secretName: {{ .Values.ingress.tls.secretName | quote }} + {{- end }} + {{- end }} + rules: + - host: {{ .Values.ingress.host | quote }} + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: nginx + port: + number: 2026 +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/nginx-deployment.yaml b/deploy/helm/deer-flow/templates/nginx-deployment.yaml new file mode 100644 index 000000000..5eb88df7c --- /dev/null +++ b/deploy/helm/deer-flow/templates/nginx-deployment.yaml @@ -0,0 +1,85 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "deer-flow.fullname" . }}-nginx + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: nginx +spec: + replicas: {{ .Values.nginx.replicas }} + selector: + matchLabels: + {{- include "deer-flow.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: nginx + template: + metadata: + labels: + {{- include "deer-flow.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: nginx + annotations: + # Roll pods when the nginx ConfigMap changes (subPath mount). + checksum/nginx: {{ include "deer-flow.nginxChecksum" . }} + spec: + securityContext: + # Non-root: nginx:alpine ships a `nginx` user (uid 101). /etc/nginx and + # /var/cache/nginx are root-owned, so the command writes the rendered + # config to /tmp and we mount an emptyDir on the cache dir (below). + runAsNonRoot: true + runAsUser: 101 + runAsGroup: 101 + fsGroup: 101 + seccompProfile: + type: RuntimeDefault + containers: + - name: nginx + image: {{ include "deer-flow.nginxImage" . }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + ports: + - name: http + containerPort: 2026 + # Strip the IPv6 listen line when IPv6 is unavailable (mirrors compose). + # Writes the config to /tmp (1777, writable by uid 101) instead of + # /etc/nginx (root-owned), then loads it with `nginx -c`. + command: ["sh", "-c"] + args: + - >- + cp /etc/nginx/nginx.conf.template /tmp/nginx.conf && + test -e /proc/net/if_inet6 || sed -i '/^[[:space:]]*listen[[:space:]]\+\[::\]:2026[[:space:]]/d' /tmp/nginx.conf && + nginx -c /tmp/nginx.conf -g 'daemon off;' + readinessProbe: + httpGet: + path: /health + port: http + initialDelaySeconds: 5 + periodSeconds: 10 + livenessProbe: + httpGet: + path: /nginx-health + port: http + initialDelaySeconds: 15 + periodSeconds: 20 + {{- with .Values.nginx.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: nginx-config + mountPath: /etc/nginx/nginx.conf.template + subPath: nginx.conf + readOnly: true + # nginx creates client_body/proxy temp dirs under /var/cache/nginx + # at startup; the dir is root-owned, so mount an emptyDir (writable + # by fsGroup 101). The pid already lives at /tmp/nginx.pid (config). + - name: nginx-cache + mountPath: /var/cache/nginx + volumes: + - name: nginx-config + configMap: + name: {{ include "deer-flow.fullname" . }}-nginx + - name: nginx-cache + emptyDir: {} diff --git a/deploy/helm/deer-flow/templates/nginx-service.yaml b/deploy/helm/deer-flow/templates/nginx-service.yaml new file mode 100644 index 000000000..0a1188288 --- /dev/null +++ b/deploy/helm/deer-flow/templates/nginx-service.yaml @@ -0,0 +1,23 @@ +apiVersion: v1 +kind: Service +metadata: + name: nginx + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: nginx +spec: + type: {{ .Values.nginx.service.type | default "ClusterIP" }} + {{- if and (eq (default "ClusterIP" .Values.nginx.service.type) "LoadBalancer") .Values.nginx.service.loadBalancerIP }} + loadBalancerIP: {{ .Values.nginx.service.loadBalancerIP | quote }} + {{- end }} + ports: + - name: http + port: {{ .Values.nginx.service.port | default 2026 }} + targetPort: http + {{- if and (eq (default "ClusterIP" .Values.nginx.service.type) "NodePort") .Values.nginx.service.nodePort }} + nodePort: {{ .Values.nginx.service.nodePort }} + {{- end }} + selector: + {{- include "deer-flow.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: nginx diff --git a/deploy/helm/deer-flow/templates/postgres-secret.yaml b/deploy/helm/deer-flow/templates/postgres-secret.yaml new file mode 100644 index 000000000..e81c4e591 --- /dev/null +++ b/deploy/helm/deer-flow/templates/postgres-secret.yaml @@ -0,0 +1,43 @@ +{{- /* + Postgres Secret — holds DATABASE_URL (key `database-url`) and, in bundled + mode, the postgres superuser password (key `postgres-password`). + + Created when the chart owns the credentials: + • bundled mode (postgresql.enabled=true) → generates password, builds DSN + • external mode with postgresql.external.databaseUrl set → wraps the DSN verbatim + NOT created when the user manages the Secret themselves: + • postgresql.external.existingSecret OR postgresql.existingSecret → referenced only +*/ -}} +{{- $userSecret := or .Values.postgresql.external.existingSecret .Values.postgresql.existingSecret -}} +{{- $wrapExternal := and (not .Values.postgresql.enabled) .Values.postgresql.external.databaseUrl -}} +{{- if and (not $userSecret) (or .Values.postgresql.enabled $wrapExternal) -}} +{{- $password := "" -}} +{{- $dsn := "" -}} +{{- if .Values.postgresql.enabled -}} +{{- /* bundled: password persists across upgrades via lookup */ -}} +{{- $prev := lookup "v1" "Secret" (include "deer-flow.namespace" .) (include "deer-flow.databaseUrlSecret" .) -}} +{{- if $prev -}} +{{- $password = index $prev.data "postgres-password" | default "" | b64dec -}} +{{- end -}} +{{- if not $password -}}{{- $password = randAlphaNum 32 -}}{{- end -}} +{{- $dsn = printf "postgresql://%s:%s@%s:5432/%s" .Values.postgresql.auth.username (include "deer-flow.urlEscape" $password) (include "deer-flow.postgresFullname" .) .Values.postgresql.auth.database -}} +{{- else -}} +{{- /* external: user-supplied DSN, verbatim */ -}} +{{- $dsn = .Values.postgresql.external.databaseUrl -}} +{{- end -}} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "deer-flow.databaseUrlSecret" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: postgres +type: Opaque +stringData: + database-url: {{ $dsn | quote }} + {{- if .Values.postgresql.enabled }} + # Superuser password for the bundled postgres StatefulSet (POSTGRES_PASSWORD). + postgres-password: {{ $password | quote }} + {{- end }} +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/postgres-service.yaml b/deploy/helm/deer-flow/templates/postgres-service.yaml new file mode 100644 index 000000000..10c192b44 --- /dev/null +++ b/deploy/helm/deer-flow/templates/postgres-service.yaml @@ -0,0 +1,19 @@ +{{- if .Values.postgresql.enabled -}} +apiVersion: v1 +kind: Service +metadata: + name: {{ include "deer-flow.postgresFullname" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: postgres +spec: + type: ClusterIP + ports: + - name: tcp-postgres + port: 5432 + targetPort: tcp-postgres + selector: + {{- include "deer-flow.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: postgres +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/postgres-statefulset.yaml b/deploy/helm/deer-flow/templates/postgres-statefulset.yaml new file mode 100644 index 000000000..2bd1521ae --- /dev/null +++ b/deploy/helm/deer-flow/templates/postgres-statefulset.yaml @@ -0,0 +1,95 @@ +{{- if .Values.postgresql.enabled -}} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: {{ include "deer-flow.postgresFullname" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: postgres +spec: + serviceName: {{ include "deer-flow.postgresFullname" . }} + replicas: 1 + selector: + matchLabels: + {{- include "deer-flow.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: postgres + template: + metadata: + labels: + {{- include "deer-flow.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: postgres + spec: + {{- include "deer-flow.imagePullSecrets" . | nindent 6 }} + securityContext: + # Non-root: postgres:16 ships a `postgres` user (uid 999). The official + # entrypoint detects non-root, skips the chown/gosu dance, and runs + # initdb as 999. fsGroup 999 makes the data PVC group-writable. + runAsNonRoot: true + runAsUser: 999 + runAsGroup: 999 + fsGroup: 999 + seccompProfile: + type: RuntimeDefault + containers: + - name: postgres + image: "{{ .Values.postgresql.image.repository }}:{{ .Values.postgresql.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + ports: + - name: tcp-postgres + containerPort: 5432 + env: + - name: POSTGRES_USER + value: {{ .Values.postgresql.auth.username | quote }} + - name: POSTGRES_DB + value: {{ .Values.postgresql.auth.database | quote }} + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "deer-flow.databaseUrlSecret" . }} + key: postgres-password + readinessProbe: + exec: + command: ["sh", "-c", "pg_isready -U $POSTGRES_USER -d $POSTGRES_DB"] + initialDelaySeconds: 5 + periodSeconds: 10 + livenessProbe: + exec: + command: ["sh", "-c", "pg_isready -U $POSTGRES_USER -d $POSTGRES_DB"] + initialDelaySeconds: 30 + periodSeconds: 20 + {{- with .Values.postgresql.primary.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: data + mountPath: /var/lib/postgresql/data + {{- if not .Values.postgresql.primary.persistence.enabled }} + # emptyDir fallback (testing only — data is lost on pod restart). + volumes: + - name: data + emptyDir: {} + {{- end }} + {{- if .Values.postgresql.primary.persistence.enabled }} + volumeClaimTemplates: + - metadata: + name: data + labels: + {{- include "deer-flow.labels" . | nindent 10 }} + app.kubernetes.io/component: postgres + spec: + accessModes: + - {{ .Values.postgresql.primary.persistence.accessMode | quote }} + resources: + requests: + storage: {{ .Values.postgresql.primary.persistence.size | quote }} + {{- with .Values.postgresql.primary.persistence.storageClass }} + storageClassName: {{ . | quote }} + {{- end }} + {{- end }} +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/provisioner-deployment.yaml b/deploy/helm/deer-flow/templates/provisioner-deployment.yaml new file mode 100644 index 000000000..013e1eef2 --- /dev/null +++ b/deploy/helm/deer-flow/templates/provisioner-deployment.yaml @@ -0,0 +1,94 @@ +{{- if .Values.provisioner.enabled -}} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "deer-flow.fullname" . }}-provisioner + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: provisioner +spec: + replicas: 1 + selector: + matchLabels: + {{- include "deer-flow.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: provisioner + template: + metadata: + labels: + {{- include "deer-flow.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: provisioner + spec: + {{- include "deer-flow.imagePullSecrets" . | nindent 6 }} + serviceAccountName: {{ include "deer-flow.fullname" . }}-provisioner + securityContext: + # Non-root: provisioner image has no USER directive. uid/gid 1000. + # No fsGroup — it mounts no PVCs (it references PVC names for the + # sandbox Pods it spawns, never mounting them itself). + runAsNonRoot: true + runAsUser: 1000 + runAsGroup: 1000 + seccompProfile: + type: RuntimeDefault + containers: + - name: provisioner + image: {{ include "deer-flow.provisionerImage" . }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + ports: + - name: http + containerPort: 8002 + env: + # Running as non-root uid 1000: suppress .pyc writes under root-owned /app. + - name: PYTHONDONTWRITEBYTECODE + value: "1" + - name: K8S_NAMESPACE + value: {{ include "deer-flow.namespace" . | quote }} + - name: SANDBOX_IMAGE + value: {{ .Values.provisioner.sandboxImage | quote }} + # In-cluster API access via the mounted ServiceAccount token. + # K8S_API_SERVER is intentionally unset (do NOT use host.docker.internal). + # NODE_HOST: the address the gateway uses to reach sandbox NodePorts. + # When `provisioner.nodeHost` is set, use it; otherwise default to this + # pod's node IP via the downward API. A NodePort is exposed on every + # node, so : routes from the gateway on most clusters. + # Override only when pod->node-IP traffic is blocked by the CNI/policy. + - name: NODE_HOST + {{- if .Values.provisioner.nodeHost }} + value: {{ .Values.provisioner.nodeHost | quote }} + {{- else }} + valueFrom: + fieldRef: + fieldPath: status.hostIP + {{- end }} + # PVC mode — sandbox Pods mount the same PVCs the gateway uses. + {{- if .Values.persistence.home.enabled }} + - name: USERDATA_PVC_NAME + value: {{ include "deer-flow.homePVC" . | quote }} + {{- end }} + {{- if .Values.skills.existingClaim }} + - name: SKILLS_PVC_NAME + value: {{ .Values.skills.existingClaim | quote }} + {{- end }} + - name: SANDBOX_CONTAINER_PORT + value: {{ .Values.provisioner.sandboxPort | quote }} + readinessProbe: + httpGet: + path: /health + port: http + initialDelaySeconds: 5 + periodSeconds: 10 + livenessProbe: + httpGet: + path: /health + port: http + initialDelaySeconds: 15 + periodSeconds: 20 + {{- with .Values.provisioner.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/provisioner-rbac.yaml b/deploy/helm/deer-flow/templates/provisioner-rbac.yaml new file mode 100644 index 000000000..cf8503f99 --- /dev/null +++ b/deploy/helm/deer-flow/templates/provisioner-rbac.yaml @@ -0,0 +1,76 @@ +{{- if .Values.provisioner.enabled -}} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "deer-flow.fullname" . }}-provisioner + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: provisioner +--- +# Namespaced permissions: manage sandbox Pods + Services in this namespace. +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "deer-flow.fullname" . }}-provisioner + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: provisioner +rules: + # Verbs mirror docker/provisioner/app.py, which only calls get/create/delete + # on pods and get/list/create/delete on services. update/patch, pods/exec, + # and events were unused and dropped (least privilege). These verbs still + # apply to *all* Pods in the namespace - scoping to sandbox Pods is deferred + # (see README "Not yet enforced"). + - apiGroups: [""] + resources: ["pods", "pods/log", "services"] + verbs: ["get", "list", "watch", "create", "delete"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "deer-flow.fullname" . }}-provisioner + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: provisioner +subjects: + - kind: ServiceAccount + name: {{ include "deer-flow.fullname" . }}-provisioner + namespace: {{ include "deer-flow.namespace" . }} +roleRef: + kind: Role + name: {{ include "deer-flow.fullname" . }}-provisioner + apiGroup: rbac.authorization.k8s.io +--- +# Cluster-scoped: read/create Namespaces so the provisioner can ensure the +# sandbox namespace exists (mirrors docker/provisioner/README.md:202-206). +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ include "deer-flow.fullname" . }}-provisioner-ns + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: provisioner +rules: + - apiGroups: [""] + resources: ["namespaces"] + verbs: ["get", "list", "create"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ include "deer-flow.fullname" . }}-provisioner-ns + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: provisioner +subjects: + - kind: ServiceAccount + name: {{ include "deer-flow.fullname" . }}-provisioner + namespace: {{ include "deer-flow.namespace" . }} +roleRef: + kind: ClusterRole + name: {{ include "deer-flow.fullname" . }}-provisioner-ns + apiGroup: rbac.authorization.k8s.io +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/provisioner-service.yaml b/deploy/helm/deer-flow/templates/provisioner-service.yaml new file mode 100644 index 000000000..5cef41f6c --- /dev/null +++ b/deploy/helm/deer-flow/templates/provisioner-service.yaml @@ -0,0 +1,19 @@ +{{- if .Values.provisioner.enabled -}} +apiVersion: v1 +kind: Service +metadata: + name: provisioner + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: provisioner +spec: + type: ClusterIP + ports: + - name: http + port: 8002 + targetPort: http + selector: + {{- include "deer-flow.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: provisioner +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/pvc-home.yaml b/deploy/helm/deer-flow/templates/pvc-home.yaml new file mode 100644 index 000000000..e1628cbf0 --- /dev/null +++ b/deploy/helm/deer-flow/templates/pvc-home.yaml @@ -0,0 +1,18 @@ +{{- if .Values.persistence.home.enabled -}} +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: {{ include "deer-flow.homePVC" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} +spec: + accessModes: + - {{ .Values.persistence.home.accessMode | quote }} + resources: + requests: + storage: {{ .Values.persistence.home.size | quote }} + {{- with .Values.persistence.home.storageClass }} + storageClassName: {{ . | quote }} + {{- end }} +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/redis-secret.yaml b/deploy/helm/deer-flow/templates/redis-secret.yaml new file mode 100644 index 000000000..50035545c --- /dev/null +++ b/deploy/helm/deer-flow/templates/redis-secret.yaml @@ -0,0 +1,43 @@ +{{- /* + Redis Secret — holds the stream-bridge URL (key `redis-url`) and, in bundled + mode with a password, the redis AUTH password (key `redis-password`). + + Created when the chart owns the credentials: + • bundled mode (redis.enabled=true) → builds DSN from in-cluster Service + • external mode with redis.external.redisUrl set → wraps the URL verbatim + NOT created when the user manages the Secret themselves: + • redis.external.existingSecret OR redis.existingSecret → referenced only + + Auth: empty `redis.auth.password` (default) = no auth, matching the compose + deployment (ClusterIP isolation only). Set a password to enable ACL/AUTH; + the DSN then becomes redis://:@host:6379/0. +*/ -}} +{{- $userSecret := or .Values.redis.external.existingSecret .Values.redis.existingSecret -}} +{{- $wrapExternal := and (not .Values.redis.enabled) .Values.redis.external.redisUrl -}} +{{- if and (not $userSecret) (or .Values.redis.enabled $wrapExternal) -}} +{{- $password := .Values.redis.auth.password -}} +{{- $dsn := "" -}} +{{- if .Values.redis.enabled -}} +{{- /* bundled: DSN points at the in-cluster Service */ -}} +{{- $auth := ternary (printf ":%s@" (include "deer-flow.urlEscape" $password)) "" (ne $password "") -}} +{{- $dsn = printf "redis://%s%s:6379/0" $auth (include "deer-flow.redisFullname" .) -}} +{{- else -}} +{{- /* external: user-supplied URL, verbatim */ -}} +{{- $dsn = .Values.redis.external.redisUrl -}} +{{- end -}} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "deer-flow.redisUrlSecret" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: redis +type: Opaque +stringData: + redis-url: {{ $dsn | quote }} + {{- if and .Values.redis.enabled $password }} + # AUTH password for the bundled redis StatefulSet (REDIS_PASSWORD). + redis-password: {{ $password | quote }} + {{- end }} +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/redis-service.yaml b/deploy/helm/deer-flow/templates/redis-service.yaml new file mode 100644 index 000000000..39ac173c3 --- /dev/null +++ b/deploy/helm/deer-flow/templates/redis-service.yaml @@ -0,0 +1,19 @@ +{{- if .Values.redis.enabled -}} +apiVersion: v1 +kind: Service +metadata: + name: {{ include "deer-flow.redisFullname" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: redis +spec: + type: ClusterIP + ports: + - name: tcp-redis + port: 6379 + targetPort: tcp-redis + selector: + {{- include "deer-flow.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: redis +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/redis-statefulset.yaml b/deploy/helm/deer-flow/templates/redis-statefulset.yaml new file mode 100644 index 000000000..a09483091 --- /dev/null +++ b/deploy/helm/deer-flow/templates/redis-statefulset.yaml @@ -0,0 +1,106 @@ +{{- if .Values.redis.enabled -}} +{{- $password := .Values.redis.auth.password -}} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: {{ include "deer-flow.redisFullname" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} + app.kubernetes.io/component: redis +spec: + serviceName: {{ include "deer-flow.redisFullname" . }} + replicas: 1 + selector: + matchLabels: + {{- include "deer-flow.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: redis + template: + metadata: + labels: + {{- include "deer-flow.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: redis + spec: + {{- include "deer-flow.imagePullSecrets" . | nindent 6 }} + securityContext: + # Non-root: redis:7-alpine ships a `redis` user (uid 999). The official + # entrypoint detects non-root and skips the gosu dance. fsGroup 999 + # makes the data PVC group-writable. + runAsNonRoot: true + runAsUser: 999 + runAsGroup: 999 + fsGroup: 999 + seccompProfile: + type: RuntimeDefault + containers: + - name: redis + image: "{{ .Values.redis.image.repository }}:{{ .Values.redis.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + ports: + - name: tcp-redis + containerPort: 6379 + {{- if $password }} + # AUTH password is read at runtime so it never appears in the pod spec + # args. `redis-server --requirepass` + `redis-cli -a` both read it + # from this env. + env: + - name: REDIS_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "deer-flow.redisUrlSecret" . }} + key: redis-password + {{- end }} + command: ["sh", "-c"] + args: + - exec redis-server --appendonly yes{{ if $password }} --requirepass "$REDIS_PASSWORD"{{ end }} + readinessProbe: + exec: + command: + - sh + - -c + - {{ if $password }}redis-cli --no-auth-warning -a "$REDIS_PASSWORD" ping{{ else }}redis-cli ping{{ end }} + initialDelaySeconds: 5 + periodSeconds: 10 + livenessProbe: + exec: + command: + - sh + - -c + - {{ if $password }}redis-cli --no-auth-warning -a "$REDIS_PASSWORD" ping{{ else }}redis-cli ping{{ end }} + initialDelaySeconds: 30 + periodSeconds: 20 + {{- with .Values.redis.primary.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: data + mountPath: /data + {{- if not .Values.redis.primary.persistence.enabled }} + # emptyDir fallback (testing only — data is lost on pod restart). + volumes: + - name: data + emptyDir: {} + {{- end }} + {{- if .Values.redis.primary.persistence.enabled }} + volumeClaimTemplates: + - metadata: + name: data + labels: + {{- include "deer-flow.labels" . | nindent 10 }} + app.kubernetes.io/component: redis + spec: + accessModes: + - {{ .Values.redis.primary.persistence.accessMode | quote }} + resources: + requests: + storage: {{ .Values.redis.primary.persistence.size | quote }} + {{- with .Values.redis.primary.persistence.storageClass }} + storageClassName: {{ . | quote }} + {{- end }} + {{- end }} +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/secret-app.yaml b/deploy/helm/deer-flow/templates/secret-app.yaml new file mode 100644 index 000000000..03c48fa66 --- /dev/null +++ b/deploy/helm/deer-flow/templates/secret-app.yaml @@ -0,0 +1,22 @@ +{{- if not .Values.existingAppSecret -}} +{{- $prev := lookup "v1" "Secret" (include "deer-flow.namespace" .) (include "deer-flow.appSecret" .) -}} +{{- $betterAuth := "" -}} +{{- $internalToken := "" -}} +{{- if $prev -}} +{{- $betterAuth = index $prev.data "BETTER_AUTH_SECRET" | default "" | b64dec -}} +{{- $internalToken = index $prev.data "DEER_FLOW_INTERNAL_AUTH_TOKEN" | default "" | b64dec -}} +{{- end -}} +{{- if not $betterAuth -}}{{- $betterAuth = randAlphaNum 48 | lower -}}{{- end -}} +{{- if not $internalToken -}}{{- $internalToken = randAlphaNum 40 -}}{{- end -}} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "deer-flow.appSecret" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} +type: Opaque +stringData: + BETTER_AUTH_SECRET: {{ $betterAuth | quote }} + DEER_FLOW_INTERNAL_AUTH_TOKEN: {{ $internalToken | quote }} +{{- end -}} diff --git a/deploy/helm/deer-flow/templates/secret-provider.yaml b/deploy/helm/deer-flow/templates/secret-provider.yaml new file mode 100644 index 000000000..0c38b5e14 --- /dev/null +++ b/deploy/helm/deer-flow/templates/secret-provider.yaml @@ -0,0 +1,14 @@ +{{- if and (not .Values.existingSecret) .Values.secrets -}} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "deer-flow.providerSecret" . }} + namespace: {{ include "deer-flow.namespace" . }} + labels: + {{- include "deer-flow.labels" . | nindent 4 }} +type: Opaque +stringData: +{{- range $k, $v := .Values.secrets }} + {{ $k }}: {{ $v | quote }} +{{- end }} +{{- end -}} diff --git a/deploy/helm/deer-flow/values.yaml b/deploy/helm/deer-flow/values.yaml new file mode 100644 index 000000000..44153c25e --- /dev/null +++ b/deploy/helm/deer-flow/values.yaml @@ -0,0 +1,320 @@ +# DeerFlow Helm chart values +# +# Copy to my-values.yaml, edit, and install with: +# helm install deer-flow deploy/helm/deer-flow -n deer-flow --create-namespace -f my-values.yaml + +# -- Target namespace (also used as the K8s sandbox namespace for provisioner-spawned Pods). +namespace: deer-flow + +# -- Image registry & tag for the three DeerFlow images you build+push. +# Required: set `registry` to your registry (e.g. ghcr.io/yourorg). +image: + registry: "" # REQUIRED, e.g. ghcr.io/yourorg + tag: "latest" + pullPolicy: IfNotPresent + # -- Existing image pull secrets, e.g. [{ name: regcred }] + pullSecrets: [] + + # Image names match what .github/workflows/container.yaml publishes on GHCR + # as ${repository}- (e.g. ghcr.io//deer-flow-backend). Set + # `registry` to the owner prefix (e.g. ghcr.io/) and `tag` to consume + # the published images. + gatewayImage: deer-flow-backend + frontendImage: deer-flow-frontend + provisionerImage: deer-flow-provisioner + +# -- Gateway (backend) deployment. +gateway: + replicas: 1 # Safe default. Postgres + the Redis stream bridge are + # wired, but multi-replica needs issue #3948's run-control + # work (cancel/dedup/reconcile) — see README "Gateway replicas". + resources: + requests: + cpu: 200m + memory: 512Mi + limits: + cpu: "2" + memory: 2Gi + +# -- Frontend (Next.js) deployment. +frontend: + replicas: 1 + resources: + requests: + cpu: 100m + memory: 256Mi + limits: + cpu: "1" + memory: 512Mi + +# -- nginx reverse-proxy deployment (preserves compose routing). +nginx: + replicas: 1 + image: + repository: nginx + tag: alpine + resources: + requests: + cpu: 50m + memory: 64Mi + limits: + cpu: 500m + memory: 256Mi + service: + # -- Service type fronting nginx. `LoadBalancer` on Docker Desktop / kind / + # OrbStack routes to localhost (no Ingress controller needed). `ClusterIP` + # pairs with the Ingress resource. `NodePort` exposes on a node port. + type: ClusterIP + port: 2026 + loadBalancerIP: "" + nodePort: "" + +# -- Sandbox provisioner (K8s-native code execution). Creates sandbox Pods in +# this namespace via a ServiceAccount + RBAC. +provisioner: + enabled: true + resources: + requests: + cpu: 50m + memory: 128Mi + limits: + cpu: 500m + memory: 512Mi + # -- AIO-compatible sandbox container image used for sandbox Pods. + sandboxImage: "enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest" + # -- Host the gateway uses to reach sandbox NodePorts. Empty (default) falls + # back to the provisioner pod's node IP via the Kubernetes downward API, + # which routes on most clusters because a NodePort is exposed on every + # node. Set explicitly only when pod->node-IP traffic is blocked by your + # CNI/network policy. In Docker Compose this is `host.docker.internal`. + nodeHost: "" + # -- Sandbox container port (must match the sandboxImage's listening port). + sandboxPort: 8080 + +# -- PostgreSQL database. Bundled mode (default) deploys a single-instance +# postgres StatefulSet; set `enabled: false` to use an external managed DB. +# The gateway reads DATABASE_URL from the resolved Secret and config.yaml +# references it as $DATABASE_URL in database.postgres_url. +postgresql: + # -- Deploy a bundled postgres StatefulSet. Disable to use an external DB. + enabled: true + image: + repository: postgres + tag: "16" + auth: + database: deerflow + username: deerflow + # -- Password auto-generated (32 chars, persisted across upgrades) when + # empty. Ignored if existingSecret is set. + password: "" + # -- Use an existing Secret (key `database-url`, plus `postgres-password` + # in bundled mode) instead of generating one. Skips Secret creation. + existingSecret: "" + primary: + resources: + requests: + cpu: 100m + memory: 256Mi + limits: + cpu: "1" + memory: 1Gi + persistence: + enabled: true + storageClass: "" # "" = cluster default + accessMode: ReadWriteOnce # RWX (NFS) only needed if postgres itself is HA + size: 20Gi + # -- External postgres (used when `enabled: false`). Provide either a full + # `databaseUrl` (chart wraps it into a Secret for you) or an + # `existingSecret` you manage (key `database-url`). + external: + # -- Full DSN, e.g. postgresql://user:pass@host:5432/deerflow. Chart + # writes it into the Secret so it never touches the gateway env directly. + databaseUrl: "" + # -- Secret you manage (key `database-url`). Use this with External Secrets + # / Vault / Sealed Secrets so the DSN never appears in values files. + existingSecret: "" + +# -- Redis stream bridge. The gateway stores per-run SSE events in Redis Streams +# so live run events reach a client connected to any gateway pod (cross-pod +# SSE delivery + reconnect replay, PR #3191). Bundled mode (default) deploys +# a single-instance redis StatefulSet; set `enabled: false` to use an external +# managed Redis. The gateway reads DEER_FLOW_STREAM_BRIDGE_REDIS_URL from the +# resolved Secret; config.yaml's stream_bridge.type is `redis` by default. +redis: + # -- Deploy a bundled redis StatefulSet. Disable to use an external Redis. + enabled: true + image: + repository: redis + tag: "7-alpine" + auth: + # -- Empty (default) = no auth, matching the compose deployment (ClusterIP + # isolation only). Set a password to enable AUTH; the DSN becomes + # redis://:@host:6379/0. Ignored if existingSecret is set. + password: "" + # -- Use an existing Secret (key `redis-url`, plus `redis-password` if auth) + # instead of generating one. Skips Secret creation. + existingSecret: "" + primary: + resources: + requests: + cpu: 50m + memory: 64Mi + limits: + cpu: "1" + memory: 512Mi + persistence: + enabled: true + storageClass: "" # "" = cluster default + accessMode: ReadWriteOnce # RWX (NFS) only needed if redis itself is HA + size: 5Gi + # -- External redis (used when `enabled: false`). Provide either a full + # `redisUrl` (chart wraps it into a Secret for you) or an `existingSecret` + # you manage (key `redis-url`). + external: + # -- Full URL, e.g. redis://:pass@myredis.example:6379/0. Chart writes it + # into the Secret so it never touches the gateway env directly. + redisUrl: "" + # -- Secret you manage (key `redis-url`). Use with External Secrets / Vault + # / Sealed Secrets so the URL never appears in values files. + existingSecret: "" + +# -- Persistent volume for runtime state (.deer-flow): sqlite DB, memory, +# custom agents, and per-thread user-data. Also mounted (PVC mode) into +# provisioner-spawned sandbox Pods. +persistence: + home: + enabled: true + storageClass: "" # "" = cluster default + accessMode: ReadWriteOnce # Use ReadWriteMany (NFS/etc.) for multi-node + size: 10Gi + +# -- Skills library mounted at /app/skills. Default: emptyDir (skills disabled). +# Populate via an existing PVC or ConfigMap, or bake skills into a custom +# gateway image. Provisioner PVC mode references this same claim when set. +skills: + enabled: false + existingClaim: "" + configMap: "" + +# -- Provider/channel/search secrets injected as env vars into the gateway. +# Reference them from config.yaml as $VAR. Example: +# secrets: +# OPENAI_API_KEY: "sk-..." +# FEISHU_APP_ID: "cli_xxx" +# FEISHU_APP_SECRET: "xxx" +# GITHUB_TOKEN: "ghp_xxx" +secrets: {} +# -- Use an existing Secret instead of creating one from `secrets` above. +existingSecret: "" + +# -- Ingress in front of nginx (port 2026). nginx preserves all internal routing. +ingress: + enabled: true + className: "nginx" + host: "deer-flow.example.com" + annotations: {} + tls: + enabled: false + secretName: "" + # hosts: [] # defaults to [ingress.host] + +# -- DeerFlow config.yaml content. Secrets MUST stay as $VAR references — never +# inline literal secret values here. The default enables provisioner sandbox. +config: | + config_version: 20 + log_level: info + + models: [] + # Example (uncomment & set the matching secret in `secrets`): + # - name: gpt-4 + # display_name: GPT-4 + # use: langchain_openai:ChatOpenAI + # model: gpt-4 + # api_key: $OPENAI_API_KEY + # request_timeout: 600.0 + + sandbox: + use: deerflow.community.aio_sandbox:AioSandboxProvider + provisioner_url: http://provisioner:8002 + image: enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest + port: 8080 + replicas: 3 + + database: + # PostgreSQL is the default backend (bundled StatefulSet, or external). + # DATABASE_URL is injected from the postgres Secret; $DATABASE_URL is + # resolved by the harness before the config is instantiated. + backend: postgres + postgres_url: $DATABASE_URL + + # The LangGraph Store (cross-thread memory + thread list) reads this legacy + # `checkpointer:` section — it does NOT fall back to `database:` the way the + # checkpointer does. Point it at the same postgres so the Store is shared + # across gateway pods (required for multi-replica operation). + checkpointer: + type: postgres + connection_string: $DATABASE_URL + + # Stream bridge: `redis` (default) stores per-run SSE events in Redis Streams + # so live run events reach a client on any gateway pod (cross-pod SSE delivery + # + reconnect replay, PR #3191). The URL is read from the + # DEER_FLOW_STREAM_BRIDGE_REDIS_URL env var injected from the redis Secret. + # Set `type: memory` (and disable the redis chart) for single-pod-only mode. + stream_bridge: + type: redis + + memory: + storage_path: memory.json + + # -- Tools configuration. The agent gets NO tools unless they're listed here + # (BUILTIN_TOOLS only adds present_file + ask_clarification). The file/bash + # tools run inside the AIO sandbox configured above. The web tools + # (web_search, web_fetch, image_search) need no API key but require outbound + # internet from the gateway pod - swap backends or remove entries for + # air-gapped clusters (see config.example.yaml). + tool_groups: + - name: web + - name: file:read + - name: file:write + - name: bash + + tools: + - name: web_search + group: web + use: deerflow.community.ddg_search.tools:web_search_tool + max_results: 5 + - name: web_fetch + group: web + use: deerflow.community.jina_ai.tools:web_fetch_tool + timeout: 10 + - name: image_search + group: web + use: deerflow.community.image_search.tools:image_search_tool + max_results: 5 + - name: ls + group: file:read + use: deerflow.sandbox.tools:ls_tool + - name: read_file + group: file:read + use: deerflow.sandbox.tools:read_file_tool + - name: glob + group: file:read + use: deerflow.sandbox.tools:glob_tool + max_results: 200 + - name: grep + group: file:read + use: deerflow.sandbox.tools:grep_tool + max_results: 100 + - name: write_file + group: file:write + use: deerflow.sandbox.tools:write_file_tool + - name: str_replace + group: file:write + use: deerflow.sandbox.tools:str_replace_tool + - name: bash + group: bash + use: deerflow.sandbox.tools:bash_tool + +# -- DeerFlow extensions_config.json content (MCP servers + skill state). +extensionsConfig: | + {"mcpServers":{},"skills":{}} diff --git a/docker/docker-compose-dev.yaml b/docker/docker-compose-dev.yaml index 69c185baa..f7f11169f 100644 --- a/docker/docker-compose-dev.yaml +++ b/docker/docker-compose-dev.yaml @@ -93,7 +93,7 @@ services: - | set -e cp /etc/nginx/nginx.conf.template /etc/nginx/nginx.conf - test -e /proc/net/if_inet6 || sed -i '/^[[:space:]]*listen[[:space:]]\+\[::\]:2026;/d' /etc/nginx/nginx.conf + test -e /proc/net/if_inet6 || sed -i '/^[[:space:]]*listen[[:space:]]\+\[::\]:2026[[:space:]]/d' /etc/nginx/nginx.conf exec nginx -g 'daemon off;' depends_on: - frontend diff --git a/scripts/bump_version.sh b/scripts/bump_version.sh new file mode 100755 index 000000000..d82fd2931 --- /dev/null +++ b/scripts/bump_version.sh @@ -0,0 +1,97 @@ +#!/usr/bin/env bash +# Bump the project version across every version source in lockstep. +# +# Usage: +# scripts/bump_version.sh # e.g. scripts/bump_version.sh 2.2.0 +# +# Updates: +# backend/pyproject.toml (version = "...") +# frontend/package.json ("version": "...") +# deploy/helm/deer-flow/Chart.yaml (version: + appVersion:) +# +# This does NOT edit CHANGELOG.md or create/push a git tag — keep those manual. +# After running, commit and tag v to trigger the release workflows +# (container.yaml + chart.yaml), which gate on scripts/verify_versions.sh. + +set -euo pipefail + +if [ "$#" -ne 1 ]; then + echo "Usage: $0 (e.g. $0 2.2.0)" >&2 + exit 1 +fi + +VERSION="${1#v}" # tolerate a leading "v" (tag form) if passed by mistake + +if ! printf '%s' "$VERSION" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+([0-9A-Za-z.+-]+)?$'; then + echo "error: '$VERSION' is not a valid SemVer (expected X.Y.Z[, -prerelease])" >&2 + exit 1 +fi + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +PYPROJECT="$ROOT/backend/pyproject.toml" +PACKAGE="$ROOT/frontend/package.json" +CHART="$ROOT/deploy/helm/deer-flow/Chart.yaml" + +for f in "$PYPROJECT" "$PACKAGE" "$CHART"; do + if [ ! -f "$f" ]; then + echo "error: expected version file not found: $f" >&2 + exit 1 + fi +done + +python3 - "$PYPROJECT" "$PACKAGE" "$CHART" "$VERSION" <<'PY' +import re +import sys + +pyproject, package, chart, version = sys.argv[1:5] + +# backend/pyproject.toml — version = "..." +with open(pyproject) as f: + src = f.read() +new = re.sub(r'(?m)^version\s*=\s*".*?"', f'version = "{version}"', src, count=1) +if new == src: + sys.exit(f"error: no top-level 'version' field in {pyproject}") +with open(pyproject, "w") as f: + f.write(new) + +# frontend/package.json — "version": "..." (preserve indentation; minimal diff) +with open(package) as f: + src = f.read() +new = re.sub( + r'(?m)^(\s*)"version"\s*:\s*".*?"', + lambda m: f'{m.group(1)}"version": "{version}"', + src, + count=1, +) +if new == src: + sys.exit(f'error: no top-level "version" field in {package}') +with open(package, "w") as f: + f.write(new) + +# deploy/helm/deer-flow/Chart.yaml — version: X.Y.Z and appVersion: "X.Y.Z" +with open(chart) as f: + src = f.read() +new = re.sub(r'(?m)^version:\s*\S+', f'version: {version}', src, count=1) +new = re.sub(r'(?m)^appVersion:\s*".*?"', f'appVersion: "{version}"', new, count=1) +if new == src: + sys.exit(f"error: could not find version/appVersion in {chart}") +with open(chart, "w") as f: + f.write(new) +PY + +echo "Bumped version to $VERSION in:" +echo " backend/pyproject.toml" +echo " frontend/package.json" +echo " deploy/helm/deer-flow/Chart.yaml (version + appVersion)" +echo + +if ! bash "$ROOT/scripts/verify_versions.sh" "$VERSION"; then + echo "error: post-bump verification failed." >&2 + exit 1 +fi + +echo +echo "Next steps:" +echo " 1. Update CHANGELOG.md" +echo " 2. git add -A && git commit -m \"release: v$VERSION\"" +echo " 3. git tag v$VERSION && git push origin v$VERSION" diff --git a/scripts/verify_versions.sh b/scripts/verify_versions.sh new file mode 100755 index 000000000..e85dbf0b9 --- /dev/null +++ b/scripts/verify_versions.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# Verify that every project version source agrees. +# +# Sources checked: +# deploy/helm/deer-flow/Chart.yaml — version + appVersion +# backend/pyproject.toml — version +# frontend/package.json — version +# +# Usage: +# scripts/verify_versions.sh # all sources must be mutually equal +# scripts/verify_versions.sh 2.1.0 # all sources must equal 2.1.0 +# +# Exit status is 0 when consistent, 1 otherwise. The release workflows +# (.github/workflows/chart.yaml and container.yaml) call this on v* tags — via +# the reusable .github/workflows/verify-versions.yml — to gate publishing when +# a version source was forgotten. + +set -uo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" + +CHART="$ROOT/deploy/helm/deer-flow/Chart.yaml" +PYPROJECT="$ROOT/backend/pyproject.toml" +PACKAGE="$ROOT/frontend/package.json" + +for f in "$CHART" "$PYPROJECT" "$PACKAGE"; do + if [ ! -f "$f" ]; then + echo "::error::missing version file: $f" >&2 + exit 1 + fi +done + +CHART_VERSION=$(awk '/^version:/ {print $2; exit}' "$CHART") +APP_VERSION=$(awk '/^appVersion:/ {gsub(/"/, ""); print $2; exit}' "$CHART") +PY_VERSION=$(awk -F'"' '/^version[[:space:]]*=/ {print $2; exit}' "$PYPROJECT") +JS_VERSION=$(grep -m1 '"version"' "$PACKAGE" | awk -F'"' '{print $4}') + +printf 'Chart.yaml version: %s\n' "$CHART_VERSION" +printf 'Chart.yaml appVersion: %s\n' "$APP_VERSION" +printf 'backend/pyproject.toml: %s\n' "$PY_VERSION" +printf 'frontend/package.json: %s\n' "$JS_VERSION" + +# mismatch : prints a GitHub Actions annotation and +# returns 1 when they differ, 0 when equal. +mismatch() { + if [ "$2" != "$3" ]; then + echo "::error::$1 is '$2' but expected '$3'." >&2 + return 1 + fi + return 0 +} + +EXPECTED="${1:-}" +status=0 + +if [ -n "$EXPECTED" ]; then + printf 'Expected: %s (from tag v%s)\n\n' "$EXPECTED" "$EXPECTED" + mismatch "Chart.yaml version" "$CHART_VERSION" "$EXPECTED" || status=1 + mismatch "Chart.yaml appVersion" "$APP_VERSION" "$EXPECTED" || status=1 + mismatch "backend/pyproject.toml" "$PY_VERSION" "$EXPECTED" || status=1 + mismatch "frontend/package.json" "$JS_VERSION" "$EXPECTED" || status=1 +else + echo + mismatch "Chart.yaml appVersion" "$APP_VERSION" "$CHART_VERSION" || status=1 + mismatch "backend/pyproject.toml" "$PY_VERSION" "$CHART_VERSION" || status=1 + mismatch "frontend/package.json" "$JS_VERSION" "$CHART_VERSION" || status=1 +fi + +if [ "$status" -ne 0 ]; then + if [ -n "$EXPECTED" ]; then + echo "Tip: run scripts/bump_version.sh $EXPECTED to align all sources." >&2 + else + echo "Tip: run scripts/bump_version.sh to align all sources." >&2 + fi + exit 1 +fi + +echo "OK — all version sources agree on ${CHART_VERSION}."