mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-08 22:08:46 +00:00
bc9ee9645c
* feat(helm): add production-ready Helm chart for Kubernetes deployment Adds deploy/helm/deer-flow, a native-Kubernetes translation of the production docker-compose stack, plus CI to publish its images and chart. * ci(release): gate releases on version-source consistency Add a reusable verify-versions workflow invoked by both chart.yaml and container.yaml on v* tags. It runs scripts/verify_versions.sh against the tag and fails the release — skipping all image and chart publishing — when Chart.yaml (version + appVersion), backend/pyproject.toml, or frontend/package.json don't all match the tag. Add scripts/verify_versions.sh (the check, also runnable locally) and scripts/bump_version.sh (bumps all four sources in lockstep, then self-verifies). Document the release flow in RELEASING.md and link it from AGENTS.md. * fix(deploy): address Helm chart review feedback (#3987) Three review items from willem-bd: 1. nginx IPv6 listen strip never matched. The sed pattern required a `;` immediately after `2026`, but the rendered config emits `listen [::]:2026 default_server;` (space + `default_server` before the `;`), so the line was never deleted and nginx crash-looped on pods without IPv6 (`socket() :::2026 failed (97: Address family not supported)`). Drop the trailing `;` from the pattern so it matches. Same latent bug fixed in docker-compose-dev.yaml. 2. Passwords were spliced into DSNs verbatim, so a password containing URL-special chars (@ : / # ? % [ ] space) produced a malformed DSN and a confusing parse error. Add a `deer-flow.urlEscape` helper (replace-based: Sprig lacks urlqueryescape, and regexReplaceAllLiteral treats the replacement as a regex template so `[`/`]`/`?` break it) and apply it to the password in the postgres and redis DSNs. The raw `postgres-password` / `redis-password` keys stay unencoded - they back POSTGRES_PASSWORD / REDIS_PASSWORD, not a URL segment. 3. NODE_HOST defaulted to "gateway", which can never route: the gateway Service is ClusterIP:8001 and knows nothing of a sandbox NodePort, so a user who skips the caveat gets unreachable sandboxes with no error at install time. Default NODE_HOST to the provisioner pod's node IP via the downward API (status.hostIP) - a NodePort is exposed on every node, so <node-IP>:<NodePort> routes from the gateway on most clusters. `provisioner.nodeHost` remains an override for CNIs/policies that block pod->node-IP traffic. Updated NOTES.txt, values.yaml, and the chart README. (#3929 remains the long-term fix - ClusterIP + cluster-DNS URL removes NODE_HOST and the NodePort exposure entirely.) Validated with helm lint, helm template (incl. a special-char password rendering the encoded DSNs), and a sed pattern-match check. * fix(deploy): address round-2 Helm chart review feedback (#3987) Three "Medium" items from willem-bd: 1. No helm lint / helm template gate before publish. A template regression ships as an immutable OCI artifact (GHCR won't overwrite --version), so gate packaging on `helm lint` + `helm template --include-crds` in chart.yaml before `helm package`. (ct lint / helm-unittest deferred.) 2. Action pinning inconsistent + PR body overstates it. SHA-pin actions/checkout (v6.0.3, df4cb1c0) and actions/attest-build-provenance (v2.4.0, e8998f94) across the publishing workflows (chart.yaml, container.yaml, verify-versions.yml), matching the existing docker/* SHA-pin pattern. Resolves the checkout @v4/@v6 mismatch and makes the "SHA-pinned actions" claim accurate. Other pre-existing workflows left untouched (out of scope for this PR). 3. Provisioner RBAC broader than needed. Dropped the unused update/patch verbs and the pods/exec + events rules from the provisioner Role - audited against docker/provisioner/app.py, which only calls get/create/delete on pods and get/list/create/delete on services. Fixed NOTES.txt to accurately describe the grant instead of understating it as "create Pods and Services". The remaining scope concern - verbs apply to all Pods in the namespace, not just sandbox Pods - is still deferred (RBAC can't scope by label; needs a dedicated namespace or admission control), now noted in NOTES.txt and README. Validated with helm lint + helm template (narrowed Role renders with exactly get/list/watch/create/delete). * feat(helm): enable sandbox+web tools out of the box The chart's default config loaded zero agent tools (config.tools empty -> "Total tools loaded: 0"), so a fresh install gave an agent that could do nothing useful. Add tool_groups + tools to the default config block: - web: web_search (ddg), web_fetch (jina), image_search - no API key - file:read: ls, read_file, glob, grep - file:write: write_file, str_replace - bash The file/bash tools run inside the AIO sandbox the chart already configures; the web tools need outbound internet from the gateway pod (swap backends or drop entries for air-gapped clusters - see config.example.yaml). Also bump config_version 15 -> 19 to match config.example.yaml (the chart had drifted behind). NOTES.txt and the README example updated to match. * ci(helm): add chart validation + config_version drift check on PR Extend the chart workflow with a PR-triggered validate-chart job that runs helm lint, helm template --include-crds, and a config_version drift check: it parses config_version from both config.example.yaml and the chart's values.yaml and fails the build (with a ::error:: naming the files to bump) if the chart is behind the example. This catches the kind of drift this PR is fixing - the chart sat at v15 while the example moved to v19 - before it can merge again. verify-versions and publish-chart stay tag-only; publish-chart now needs: [verify-versions, validate-chart]. validate-chart runs on both PRs and tag pushes: the tag arm is required because a job that `needs` a skipped job is itself skipped under the default success() check, so validate-chart must actually run on tag pushes or publish-chart would never fire. * Bump config version to 20
99 lines
4.1 KiB
YAML
99 lines
4.1 KiB
YAML
name: Publish Helm Chart
|
|
|
|
# Publishes the DeerFlow Helm chart as an OCI artifact to GHCR alongside the
|
|
# container images (see container.yaml). Triggers on the same `v*` tags.
|
|
#
|
|
# On pull requests touching the chart or config.example.yaml, `validate-chart`
|
|
# runs lint + template render + a config_version drift check (the chart's
|
|
# embedded config_version must not lag config.example.yaml) so a broken or
|
|
# stale chart fails the PR, not the release.
|
|
#
|
|
# Users then install with:
|
|
# helm install deer-flow oci://ghcr.io/${{ owner }}/deer-flow --version <ver>
|
|
|
|
on:
|
|
push:
|
|
tags:
|
|
- "v*"
|
|
pull_request:
|
|
paths:
|
|
- "deploy/helm/deer-flow/**"
|
|
- "config.example.yaml"
|
|
- ".github/workflows/chart.yaml"
|
|
|
|
jobs:
|
|
validate-chart:
|
|
# Runs on PRs and release tags: catch a broken render or a stale
|
|
# config_version before merge / publish. A broken chart published under a
|
|
# vX.Y.Z tag is an immutable OCI artifact (GHCR won't let you overwrite
|
|
# --version), so a regression must fail here, not on install.
|
|
if: github.event_name == 'pull_request' || startsWith(github.ref, 'refs/tags/v')
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3
|
|
|
|
# ubuntu-latest ships with helm 3 preinstalled - no setup-helm action needed.
|
|
- name: Lint chart
|
|
run: helm lint deploy/helm/deer-flow
|
|
- name: Validate templates render
|
|
run: helm template deer-flow deploy/helm/deer-flow --include-crds >/dev/null
|
|
|
|
# The chart's `config:` block embeds a config_version that must not fall
|
|
# behind config.example.yaml. A stale version is silent in-cluster (the
|
|
# image ships no example to compare against, so _check_config_version
|
|
# never warns) but means the chart's config is authored against an older
|
|
# schema. Bump it in values.yaml and the README example. config_version
|
|
# gates no runtime behavior - it only drives the outdated-warning - so a
|
|
# bare version bump needs no field changes.
|
|
- name: config_version drift check
|
|
run: |
|
|
set -eu
|
|
example=$(grep -E '^config_version:[[:space:]]+[0-9]+' config.example.yaml | head -1 | awk '{print $2}')
|
|
chart=$(awk '/^config:[[:space:]]*\|/{f=1; next} f && /^[[:space:]]+config_version:[[:space:]]+[0-9]+/ {print $2; exit}' deploy/helm/deer-flow/values.yaml)
|
|
echo "config.example.yaml config_version=$example"
|
|
echo "chart values.yaml config_version=$chart"
|
|
if [ -z "$example" ] || [ -z "$chart" ]; then
|
|
echo "::error::could not parse config_version from one of the files"
|
|
exit 1
|
|
fi
|
|
if [ "$chart" -lt "$example" ]; then
|
|
echo "::error::chart config_version ($chart) is behind config.example.yaml ($example). Bump 'config_version' in deploy/helm/deer-flow/values.yaml (and the README example) to $example."
|
|
exit 1
|
|
fi
|
|
|
|
verify-versions:
|
|
# Gate the release: every version source must match the v* tag. A forgotten
|
|
# bump in Chart.yaml, pyproject.toml, or package.json fails here and skips
|
|
# the publish. See scripts/verify_versions.sh.
|
|
if: startsWith(github.ref, 'refs/tags/v')
|
|
uses: ./.github/workflows/verify-versions.yml
|
|
|
|
publish-chart:
|
|
if: startsWith(github.ref, 'refs/tags/v')
|
|
needs: [verify-versions, validate-chart]
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3
|
|
|
|
- name: Log in to GHCR
|
|
run: |
|
|
echo "${{ secrets.GITHUB_TOKEN }}" | \
|
|
helm registry login ghcr.io -u ${{ github.actor }} --password-stdin
|
|
|
|
- name: Package chart
|
|
run: helm package deploy/helm/deer-flow --destination ./packages
|
|
|
|
- name: Push chart to GHCR
|
|
run: |
|
|
for pkg in ./packages/*.tgz; do
|
|
echo "--- pushing $pkg"
|
|
helm push "$pkg" oci://ghcr.io/${{ github.repository_owner }}
|
|
done
|