mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-10 14:58:46 +00:00
a9145e426c
* feat(frontend): show real project version on About page The About page heading was hardcoded "About DeerFlow 2.0" while the real version is 2.1.0, kept in lockstep across backend/pyproject.toml, frontend/package.json, and Chart.yaml by scripts/verify_versions.sh. With the nightly workflow shipping unreleased main daily, a hardcoded version drifts further from reality every day. Wire the heading to the real version, nightly-aware: - src/version.ts reads NEXT_PUBLIC_APP_VERSION (nightly override) with a package.json fallback for release/local builds. - about-content.ts interpolates APP_VERSION into the heading; the "DeerFlow 1.0 and 2.0" milestone copy in acknowledgments is left as-is. - Dockerfile adds ARG APP_VERSION / ENV NEXT_PUBLIC_APP_VERSION in the builder stage so nightly CI can stamp the version. - nightly.yaml computes the nightly string once in prepare (<base>-nightly.<date>-<sha>, mirroring the chart scheme) and feeds it to both the frontend image build-arg and the chart publish (refactored to consume the shared output, so the displayed version and chart version can't drift). Release builds (container.yaml) are untouched and fall back to package.json. Tests cover the version branches and the heading interpolation; RELEASING.md notes the About page as a derived version consumer. * test(frontend): assert positive About-page version value Address review feedback on #4126: the unset-env assertion was negative-only (checked the stale "2.0" literal was gone), so it would still pass if APP_VERSION interpolated to "" or undefined. Mirror version.test.ts by asserting the heading carries the real resolved APP_VERSION. * ci(nightly): fail loudly on empty base version Address review feedback on #4126: the prepare step's BASE (parsed from Chart.yaml) had no empty-guard. A malformed/missing version would yield `-nightly.<date>-<sha>` (invalid semver) that now flows into both the chart publish and the frontend About-page version. Add `set -euo pipefail` plus a `test -n "$BASE"` guard mirroring publish-chart. `pipefail` matters - without it the pipeline's exit code is awk's, so `set -e` alone wouldn't catch a grep miss.
235 lines
11 KiB
YAML
235 lines
11 KiB
YAML
name: Nightly Build
|
|
|
|
# Nightly build of the three DeerFlow container images (backend, frontend,
|
|
# provisioner) and the Helm chart, published to GHCR from the default branch.
|
|
#
|
|
# This mirrors the tag-driven release workflows (container.yaml + chart.yaml)
|
|
# but trades the `v*` tag for date-based tags, since nightly ships unreleased
|
|
# `main`. The `verify-versions` gate is intentionally skipped - there is no tag
|
|
# to match - and `latest` is left untouched so it keeps tracking the last `v*`
|
|
# release. Images are amd64-only to match the release builds.
|
|
#
|
|
# Artifacts (under the running repo's owner):
|
|
# ghcr.io/<owner>/deer-flow-{backend,frontend,provisioner}:nightly
|
|
# ghcr.io/<owner>/deer-flow-{backend,frontend,provisioner}:nightly-YYYYMMDD
|
|
# oci://ghcr.io/<owner>/deer-flow chart version <base>-nightly.YYYYMMDD-<sha>
|
|
#
|
|
# The nightly chart defaults image.tag=nightly and image.registry=ghcr.io/<owner>
|
|
# (patched in-workflow, never committed), so installing it pulls the matching
|
|
# nightly images with no values overrides.
|
|
#
|
|
# Restricted to the upstream repo: every job is gated on
|
|
# `github.repository == 'bytedance/deer-flow'`, so a scheduled run or manual
|
|
# dispatch on a fork skips all jobs rather than pushing to the fork's own GHCR
|
|
# namespace. Scheduled workflows also only fire on the default branch.
|
|
|
|
on:
|
|
schedule:
|
|
- cron: "0 16 * * *" # 16:00 UTC daily (adjust as needed)
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: nightly
|
|
# Don't cancel a running nightly - a mid-cancel leaves a half-pushed set.
|
|
cancel-in-progress: false
|
|
|
|
env:
|
|
REGISTRY: ghcr.io
|
|
|
|
jobs:
|
|
prepare:
|
|
if: github.repository == 'bytedance/deer-flow'
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
date: ${{ steps.date.outputs.date }}
|
|
nightly_version: ${{ steps.nightly_version.outputs.nightly_version }}
|
|
steps:
|
|
- name: Set nightly date (UTC)
|
|
id: date
|
|
# Shared by build-images (image tag) and publish-chart (chart version)
|
|
# so the two stay in sync for a given run.
|
|
run: echo "date=$(date -u +%Y%m%d)" >> "$GITHUB_OUTPUT"
|
|
- name: Checkout repository
|
|
# Needed to read Chart.yaml's base version for the nightly version
|
|
# string computed below.
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3
|
|
- name: Compute nightly version
|
|
id: nightly_version
|
|
# Single source of truth for the nightly version string:
|
|
# <base>-nightly.<YYYYMMDD>-<short_sha> (mirrors the chart's nightly
|
|
# scheme). build-images injects it into the frontend image (About-page
|
|
# version) and publish-chart stamps it on Chart.yaml, so the version
|
|
# users see and the chart version can't drift apart.
|
|
run: |
|
|
set -euo pipefail
|
|
BASE=$(grep -m1 '^version:' deploy/helm/deer-flow/Chart.yaml | awk '{print $2}')
|
|
# A malformed/missing Chart.yaml version would yield an empty BASE ->
|
|
# `-nightly.<date>-<sha>` (invalid semver) that now flows into both
|
|
# the chart publish and the frontend About-page version. Fail loudly.
|
|
# `pipefail` matters: without it the pipeline's exit code is awk's,
|
|
# so `set -e` alone wouldn't catch a grep miss.
|
|
test -n "$BASE" || { echo "::error::empty base version from Chart.yaml"; exit 1; }
|
|
SHORT_SHA="${GITHUB_SHA::7}"
|
|
NIGHTLY="${BASE}-nightly.${{ steps.date.outputs.date }}-${SHORT_SHA}"
|
|
echo "nightly_version=${NIGHTLY}" >> "$GITHUB_OUTPUT"
|
|
echo "Nightly version: ${NIGHTLY}"
|
|
|
|
validate-chart:
|
|
if: github.repository == 'bytedance/deer-flow'
|
|
# Catch a broken render or a stale config_version before publish. A broken
|
|
# chart published under an OCI version is immutable (GHCR won't let you
|
|
# overwrite --version), so a regression must fail here, not on install.
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3
|
|
|
|
- name: Lint chart
|
|
run: helm lint deploy/helm/deer-flow
|
|
- name: Validate templates render
|
|
run: helm template deer-flow deploy/helm/deer-flow --include-crds >/dev/null
|
|
|
|
# The chart's `config:` block embeds a config_version that must not fall
|
|
# behind config.example.yaml. Shared with chart.yaml via
|
|
# scripts/check_config_version.sh so the two workflows can't drift.
|
|
- name: config_version drift check
|
|
run: bash scripts/check_config_version.sh
|
|
|
|
build-images:
|
|
needs: prepare
|
|
if: github.repository == 'bytedance/deer-flow'
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
attestations: write
|
|
id-token: write
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- component: backend
|
|
context: .
|
|
file: backend/Dockerfile
|
|
# Bake the `postgres` extra so multi-replica (K8s/Helm) deployments
|
|
# can use shared Postgres. Matches container.yaml's release image.
|
|
build-args: "UV_EXTRAS=postgres"
|
|
- component: frontend
|
|
context: .
|
|
file: frontend/Dockerfile
|
|
build-args: ""
|
|
- component: provisioner
|
|
context: docker/provisioner
|
|
file: docker/provisioner/Dockerfile
|
|
build-args: ""
|
|
env:
|
|
IMAGE_NAME: ${{ github.repository }}-${{ matrix.component }}
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3
|
|
- name: Log in to the Container registry
|
|
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 #v3.4.0
|
|
with:
|
|
registry: ${{ env.REGISTRY }}
|
|
username: ${{ github.actor }}
|
|
password: ${{ secrets.GITHUB_TOKEN }}
|
|
- name: Extract metadata (tags, labels) for Docker
|
|
id: meta
|
|
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 #v5.7.0
|
|
with:
|
|
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
|
# `nightly` rolls forward each run; `nightly-YYYYMMDD` is pinned to a
|
|
# day but mutable within it (a same-day re-dispatch overwrites it);
|
|
# `sha-<short>` is the only truly immutable tag. No `latest` (that
|
|
# stays on v* releases) and no branch/tag refs.
|
|
tags: |
|
|
type=raw,value=nightly
|
|
type=raw,value=nightly-${{ needs.prepare.outputs.date }}
|
|
type=sha
|
|
- name: Build and push Docker image
|
|
id: push
|
|
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 #v6.18.0
|
|
with:
|
|
context: ${{ matrix.context }}
|
|
file: ${{ matrix.file }}
|
|
push: true
|
|
tags: ${{ steps.meta.outputs.tags }}
|
|
labels: ${{ steps.meta.outputs.labels }}
|
|
# APP_VERSION is consumed only by the frontend Dockerfile (it stamps
|
|
# the About-page version); backend/provisioner Dockerfiles don't
|
|
# declare it. Passed via build-arg because build-push-action doesn't
|
|
# forward host env into the BuildKit build.
|
|
build-args: |
|
|
${{ matrix.build-args }}
|
|
${{ matrix.component == 'frontend' && format('APP_VERSION={0}', needs.prepare.outputs.nightly_version) || '' }}
|
|
- name: Generate artifact attestation
|
|
uses: actions/attest-build-provenance@e8998f949152b193b063cb0ec769d69d929409be #v2.4.0
|
|
with:
|
|
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
|
subject-digest: ${{ steps.push.outputs.digest }}
|
|
push-to-registry: true
|
|
|
|
publish-chart:
|
|
# Ship the chart only after images build - it references all three, so a
|
|
# failed image build withholds the chart rather than publishing one that
|
|
# would fail to pull in-cluster.
|
|
needs: [prepare, validate-chart, build-images]
|
|
if: github.repository == 'bytedance/deer-flow'
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
env:
|
|
OWNER: ${{ github.repository_owner }}
|
|
NIGHTLY: ${{ needs.prepare.outputs.nightly_version }}
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 #v6.0.3
|
|
|
|
- name: Patch chart to a nightly version + nightly image defaults
|
|
# Bumps Chart.yaml version/appVersion to the nightly version computed in
|
|
# the prepare job (<base>-nightly.<date>-<sha>, a valid semver
|
|
# prerelease; the short SHA makes each dispatch's version unique, so a
|
|
# same-day re-dispatch re-publishes cleanly - OCI chart versions are
|
|
# immutable and otherwise can't be overwritten). The same string is
|
|
# injected into the frontend image, so the About-page version and the
|
|
# chart version match. Repoints the chart's default image registry/tag
|
|
# at the nightly build. Patches are in-workflow only - nothing is
|
|
# committed back.
|
|
run: |
|
|
set -eu
|
|
CHART=deploy/helm/deer-flow
|
|
echo "Nightly chart version: ${NIGHTLY}"
|
|
sed -i "s|^version:.*|version: ${NIGHTLY}|" "$CHART/Chart.yaml"
|
|
sed -i "s|^appVersion:.*|appVersion: \"${NIGHTLY}\"|" "$CHART/Chart.yaml"
|
|
sed -i "s|^ registry: \"\".*| registry: \"ghcr.io/${OWNER}\"|" "$CHART/values.yaml"
|
|
sed -i 's|^ tag: "latest"| tag: "nightly"|' "$CHART/values.yaml"
|
|
# Gate the patches: sed exits 0 even on zero matches, so a drifted
|
|
# Chart.yaml/values.yaml would otherwise ship a chart that silently
|
|
# pulls the wrong (release `latest`) images. Fail loudly if any patch
|
|
# missed.
|
|
grep -q "^version: ${NIGHTLY}$" "$CHART/Chart.yaml" || { echo "::error::Chart.yaml version sed did not apply"; exit 1; }
|
|
grep -q "^appVersion: \"${NIGHTLY}\"$" "$CHART/Chart.yaml" || { echo "::error::Chart.yaml appVersion sed did not apply"; exit 1; }
|
|
grep -q '^ registry: "ghcr.io/' "$CHART/values.yaml" || { echo "::error::values.yaml registry sed did not apply"; exit 1; }
|
|
grep -q '^ tag: "nightly"' "$CHART/values.yaml" || { echo "::error::values.yaml tag sed did not apply"; exit 1; }
|
|
echo "--- Chart.yaml (head) ---"; sed -n '1,7p' "$CHART/Chart.yaml"
|
|
echo "--- image block ---"; sed -n '11,14p' "$CHART/values.yaml"
|
|
|
|
- name: Lint patched chart
|
|
run: helm lint deploy/helm/deer-flow
|
|
|
|
- name: Log in to GHCR
|
|
run: |
|
|
echo "${{ secrets.GITHUB_TOKEN }}" | \
|
|
helm registry login ghcr.io -u ${{ github.actor }} --password-stdin
|
|
|
|
- name: Package and push chart
|
|
run: |
|
|
helm package deploy/helm/deer-flow --destination ./packages
|
|
for pkg in ./packages/*.tgz; do
|
|
echo "--- pushing $pkg"
|
|
helm push "$pkg" oci://ghcr.io/${{ github.repository_owner }}
|
|
done
|