diff --git a/AGENTS.md b/AGENTS.md index 63bb534071..211a859b8d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -238,13 +238,12 @@ Repo-wide conventions: - **Skill text encoding** — use explicit UTF-8 for textual skill resources. Read `SKILL.md` with `encoding="utf-8-sig"` to strip a leading BOM; write with `encoding="utf-8"`. Never rely on the platform locale. -- **Version sources must stay in lockstep** — a release version must match identically in - `backend/pyproject.toml`, `frontend/package.json`, and `deploy/helm/deer-flow/Chart.yaml` - (`version` + `appVersion`), and `backend/uv.lock` must record the same version for the root - package (uv stores its PEP 440 form, e.g. `2.1.0rc0`). Pushing a `v*` git tag triggers CI - that runs `scripts/verify_versions.sh` and **blocks all publishing** if any source drifts. - Before bumping a version, run `scripts/bump_version.sh ` (aligns the four fields and - refreshes `backend/uv.lock` at once — it needs `uv` on `PATH`) and - `scripts/verify_versions.sh ` to catch drift early. See [RELEASING.md](RELEASING.md). +- **Version lockstep** — keep `backend/pyproject.toml`, + `backend/packages/harness/pyproject.toml`, `frontend/package.json`, and + `deploy/helm/deer-flow/Chart.yaml` (`version`/`appVersion`) identical. + `backend/uv.lock` must match both `deer-flow` and `deerflow-harness` in PEP 440 form. + Run `scripts/bump_version.sh ` (requires `uv` on `PATH`) to update all five + fields and the lock, then `scripts/verify_versions.sh `. `v*` tag CI blocks + all publishing on drift. See [RELEASING.md](RELEASING.md). - **Don't edit `CLAUDE.md`** — it only contains `@AGENTS.md`. All agent guidance changes belong here in `AGENTS.md`; `CLAUDE.md` is a thin import shim. diff --git a/RELEASING.md b/RELEASING.md index 4a8759566b..84ddd05140 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -8,11 +8,12 @@ gates the release on them agreeing with the tag. ## Version sources -A release version must appear, identically, in four places: +A release version must appear, identically, in five places: | File | Field | | -------------------------------------- | -------------------- | | `backend/pyproject.toml` | `version = "X.Y.Z"` | +| `backend/packages/harness/pyproject.toml` | `version = "X.Y.Z"` | | `frontend/package.json` | `"version": "X.Y.Z"` | | `deploy/helm/deer-flow/Chart.yaml` | `version: X.Y.Z` | | `deploy/helm/deer-flow/Chart.yaml` | `appVersion: "X.Y.Z"`| @@ -32,7 +33,7 @@ distinguishes it from a release. ## Helper scripts -- `scripts/bump_version.sh ` — set all four fields at once, refresh +- `scripts/bump_version.sh ` — set all five fields at once, refresh `backend/uv.lock`, then self-verify. Tolerates a leading `v` (e.g. `v2.1.0`). Needs `uv` on `PATH` (`backend/uv.lock` pins the root package version too, and lint CI runs `uv lock --check`); the script fails before editing anything when @@ -77,7 +78,7 @@ distinguishes it from a release. ### Release candidates -Release-candidate tags must include the same prerelease suffix in all four +Release-candidate tags must include the same prerelease suffix in all five version fields. For example, before tagging `v2.1.0-rc0`, run `bash scripts/bump_version.sh 2.1.0-rc0` and run `bash scripts/verify_versions.sh 2.1.0-rc0` from the repository root. Commit @@ -150,7 +151,7 @@ Both features stay opt-in: the provisioner ignores them until Both publishing workflows call `.github/workflows/verify-versions.yml` as their first job. It runs `scripts/verify_versions.sh` against the tag (minus the -`v`). If any of the four version sources doesn't match the tag, or +`v`). If any of the five version sources doesn't match the tag, or `backend/uv.lock` is stale, the verify job fails and **all** publish jobs are skipped — no images, no chart. diff --git a/backend/tests/test_bump_version_lockfile_sync.py b/backend/tests/test_bump_version_lockfile_sync.py index b2b1b9576a..ba35190cca 100644 --- a/backend/tests/test_bump_version_lockfile_sync.py +++ b/backend/tests/test_bump_version_lockfile_sync.py @@ -45,19 +45,20 @@ pytestmark = pytest.mark.skipif( reason="the fake `uv` on PATH is an extensionless shebang script; Windows does not execute those", ) -# Minimal versions of the three files bump_version.sh rewrites, plus the lock -# entry uv writes for the root ("virtual") package. Only the fields the script -# touches are present, so a stray rewrite shows up as a diff. +# Minimal versions of the files bump_version.sh rewrites, plus the lock entries +# uv writes for the root package and the deerflow-harness workspace member. Only +# the fields the script touches are present, so a stray rewrite shows up as a diff. PYPROJECT = '[project]\nname = "deer-flow"\nversion = "2.1.0-rc0"\n' +HARNESS_PYPROJECT = '[project]\nname = "deerflow-harness"\nversion = "2.1.0-rc0"\n' PACKAGE_JSON = '{\n "name": "deer-flow-frontend",\n "version": "2.1.0-rc0",\n "private": true\n}\n' CHART = 'apiVersion: v2\nname: deer-flow\nversion: 2.1.0-rc0\nappVersion: "2.1.0-rc0"\n' -UV_LOCK = 'version = 1\n\n[[package]]\nname = "deer-flow"\nversion = "2.1.0rc0"\nsource = { virtual = "." }\n' +UV_LOCK = 'version = 1\n\n[[package]]\nname = "deer-flow"\nversion = "2.1.0rc0"\nsource = { virtual = "." }\n\n[[package]]\nname = "deerflow-harness"\nversion = "2.1.0rc0"\nsource = { editable = "packages/harness" }\n' -# Emulates the uv behavior the release flow depends on: `uv lock` records the -# pyproject version in the root lock entry, in uv's PEP 440 form (for these -# inputs that only means dropping the `-`), and `uv lock --check` (run by the -# verify_versions.sh self-check) fails while the entry lags. The invocation is -# logged so the test can tell whether the script called uv at all. +# Emulates the uv behavior the release flow depends on: `uv lock` records each +# workspace member's pyproject version in the lock, in uv's PEP 440 form (for +# these inputs that only means dropping the `-`), and `uv lock --check` (run by +# the verify_versions.sh self-check) fails while an entry lags. The invocation +# is logged so the test can tell whether the script called uv at all. FAKE_UV = """#!/usr/bin/env python3 import re import sys @@ -74,21 +75,36 @@ if args not in (["lock"], ["lock", "--check"]): root = Path.cwd() lock_path = root / "uv.lock" lock_text = lock_path.read_text(encoding="utf-8") -pyproject = (root / "pyproject.toml").read_text(encoding="utf-8") -declared = re.search(r'(?m)^version\\s*=\\s*"([^"]+)"', pyproject).group(1) -normalized = declared.replace("-", "") -recorded = re.search(r'(?m)^name = "deer-flow"\\nversion = "([^"]+)"', lock_text).group(1) + +members = [("deer-flow", root / "pyproject.toml")] +harness = root / "packages" / "harness" / "pyproject.toml" +if harness.exists(): + members.append(("deerflow-harness", harness)) + +for name, pyproject in members: + declared = re.search(r'(?m)^version\\s*=\\s*"([^"]+)"', pyproject.read_text(encoding="utf-8")).group(1) + normalized = declared.replace("-", "") + recorded = re.search(rf'(?m)^name = "{name}"\\nversion = "([^"]+)"', lock_text).group(1) + if args == ["lock", "--check"]: + if recorded != normalized: + sys.exit("The lockfile at `uv.lock` needs to be updated, but `--check` was provided.") + else: + lock_text = lock_text.replace( + f'name = "{name}"\\nversion = "{recorded}"', + f'name = "{name}"\\nversion = "{normalized}"', + 1, + ) if args == ["lock", "--check"]: - sys.exit(0 if recorded == normalized else "The lockfile at `uv.lock` needs to be updated, but `--check` was provided.") -lock_path.write_text(lock_text.replace(f'version = "{recorded}"', f'version = "{normalized}"', 1), encoding="utf-8") + sys.exit(0) +lock_path.write_text(lock_text, encoding="utf-8") """ def _write_sandbox(root: Path) -> Path: """Materialize a repo-shaped tree the scripts can run against.""" (root / "scripts").mkdir(parents=True) - (root / "backend").mkdir() + (root / "backend" / "packages" / "harness").mkdir(parents=True) (root / "frontend").mkdir() (root / "deploy" / "helm" / "deer-flow").mkdir(parents=True) @@ -96,6 +112,7 @@ def _write_sandbox(root: Path) -> Path: shutil.copy2(REPO_ROOT / "scripts" / name, root / "scripts" / name) (root / "backend" / "pyproject.toml").write_text(PYPROJECT, encoding="utf-8") + (root / "backend" / "packages" / "harness" / "pyproject.toml").write_text(HARNESS_PYPROJECT, encoding="utf-8") (root / "backend" / "uv.lock").write_text(UV_LOCK, encoding="utf-8") (root / "frontend" / "package.json").write_text(PACKAGE_JSON, encoding="utf-8") (root / "deploy" / "helm" / "deer-flow" / "Chart.yaml").write_text(CHART, encoding="utf-8") @@ -144,6 +161,8 @@ def test_bump_version_refreshes_the_lockfile(tmp_path: Path) -> None: assert result.returncode == 0, result.stderr assert _lock_version(root) == "2.2.0", "backend/uv.lock still pins the previous version" assert (root / "backend" / "pyproject.toml").read_text(encoding="utf-8") == PYPROJECT.replace("2.1.0-rc0", "2.2.0") + assert (root / "backend" / "packages" / "harness" / "pyproject.toml").read_text(encoding="utf-8") == HARNESS_PYPROJECT.replace("2.1.0-rc0", "2.2.0"), "the deerflow-harness workspace member must move with the release" + assert 'name = "deerflow-harness"\nversion = "2.2.0"' in (root / "backend" / "uv.lock").read_text(encoding="utf-8"), "uv rewrites the harness lock entry, and the script must let it" invocations = (bin_dir / "uv-calls.log").read_text(encoding="utf-8").splitlines() assert invocations and all(line.startswith("uv lock") for line in invocations), f"bump_version.sh never invoked uv lock: {invocations}" @@ -157,6 +176,7 @@ def test_bump_version_normalizes_a_prerelease_into_the_lock(tmp_path: Path) -> N assert result.returncode == 0, result.stderr assert (root / "backend" / "pyproject.toml").read_text(encoding="utf-8") == PYPROJECT.replace("2.1.0-rc0", "2.1.0-rc1") + assert (root / "backend" / "packages" / "harness" / "pyproject.toml").read_text(encoding="utf-8") == HARNESS_PYPROJECT.replace("2.1.0-rc0", "2.1.0-rc1") assert _lock_version(root) == "2.1.0rc1", "uv normalizes the prerelease in the lock, and the script must let it" @@ -175,4 +195,5 @@ def test_bump_version_refuses_to_run_without_uv(tmp_path: Path) -> None: assert result.returncode != 0, "a missing uv must be an error: the lock would stay stale" assert "uv" in (result.stderr + result.stdout) assert (root / "backend" / "pyproject.toml").read_text(encoding="utf-8") == PYPROJECT, "the script edited files before checking for uv" + assert (root / "backend" / "packages" / "harness" / "pyproject.toml").read_text(encoding="utf-8") == HARNESS_PYPROJECT, "the script edited files before checking for uv" assert _lock_version(root) == "2.1.0rc0" diff --git a/backend/tests/test_verify_versions_lockfile.py b/backend/tests/test_verify_versions_lockfile.py index d7905cde79..c1a451dae0 100644 --- a/backend/tests/test_verify_versions_lockfile.py +++ b/backend/tests/test_verify_versions_lockfile.py @@ -68,6 +68,12 @@ declared = re.search(r'(?m)^version\\s*=\\s*"([^"]+)"', (root / "pyproject.toml" recorded = re.search(r'(?m)^name = "deer-flow"\\nversion = "([^"]+)"', (root / "uv.lock").read_text(encoding="utf-8")) if recorded is None or recorded.group(1) != declared.replace("-", ""): sys.exit("The lockfile at `uv.lock` needs to be updated, but `--check` was provided. To update the lockfile, run `uv lock`.") +harness = root / "packages" / "harness" / "pyproject.toml" +if harness.exists(): + harness_declared = re.search(r'(?m)^version\\s*=\\s*"([^"]+)"', harness.read_text(encoding="utf-8")).group(1) + harness_recorded = re.search(r'(?m)^name = "deerflow-harness"\\nversion = "([^"]+)"', (root / "uv.lock").read_text(encoding="utf-8")) + if harness_recorded is None or harness_recorded.group(1) != harness_declared.replace("-", ""): + sys.exit("The lockfile at `uv.lock` needs to be updated, but `--check` was provided. To update the lockfile, run `uv lock`.") print("Resolved 1 package") """ @@ -75,14 +81,15 @@ print("Resolved 1 package") def _write_sandbox(root: Path, *, sources: str, lock: str, lock_name: str = "deer-flow") -> Path: """A repo-shaped tree with every version source at ``sources`` and the lock at ``lock``.""" (root / "scripts").mkdir(parents=True) - (root / "backend").mkdir() + (root / "backend" / "packages" / "harness").mkdir(parents=True) (root / "frontend").mkdir() (root / "deploy" / "helm" / "deer-flow").mkdir(parents=True) shutil.copy2(REPO_ROOT / "scripts" / "verify_versions.sh", root / "scripts" / "verify_versions.sh") (root / "backend" / "pyproject.toml").write_text(f'[project]\nname = "deer-flow"\nversion = "{sources}"\n', encoding="utf-8") + (root / "backend" / "packages" / "harness" / "pyproject.toml").write_text(f'[project]\nname = "deerflow-harness"\nversion = "{sources}"\n', encoding="utf-8") (root / "backend" / "uv.lock").write_text( - f'version = 1\n\n[[package]]\nname = "{lock_name}"\nversion = "{lock}"\nsource = {{ virtual = "." }}\n', + f'version = 1\n\n[[package]]\nname = "{lock_name}"\nversion = "{lock}"\nsource = {{ virtual = "." }}\n\n[[package]]\nname = "deerflow-harness"\nversion = "{lock}"\nsource = {{ editable = "packages/harness" }}\n', encoding="utf-8", ) (root / "frontend" / "package.json").write_text(f'{{\n "name": "deer-flow-frontend",\n "version": "{sources}"\n}}\n', encoding="utf-8") @@ -131,6 +138,7 @@ def test_gate_passes_when_the_lock_matches(tmp_path: Path, sources: str, lock: s assert result.returncode == 0, result.stderr assert f"backend/uv.lock: {lock}" in result.stdout + assert f"harness/pyproject.toml: {sources}" in result.stdout assert _uv_calls(bin_dir) == [f"{(root / 'backend').resolve()} uv lock --check"] @@ -149,6 +157,22 @@ def test_gate_rejects_a_lock_left_on_the_previous_version(tmp_path: Path, args: assert "OK —" not in result.stdout +@pytest.mark.parametrize("args", [("2.2.0",), ()], ids=["against-tag", "mutual"]) +def test_gate_rejects_a_harness_pyproject_left_on_the_previous_version(tmp_path: Path, args: tuple[str, ...]): + """The other sources were bumped; backend/packages/harness/pyproject.toml was not.""" + root = _write_sandbox(tmp_path / "repo", sources="2.2.0", lock="2.2.0") + harness = root / "backend" / "packages" / "harness" / "pyproject.toml" + harness.write_text('[project]\nname = "deerflow-harness"\nversion = "2.1.0"\n', encoding="utf-8") + bin_dir = _install_fake_uv(root) + + result = _run_verify(root, *args, path=_with_fake_uv(bin_dir)) + + assert result.returncode == 1 + assert "::error::backend/packages/harness/pyproject.toml is '2.1.0' but expected '2.2.0'." in result.stderr + assert "Tip: run scripts/bump_version.sh" in result.stderr + assert "OK —" not in result.stdout + + def test_gate_does_not_blame_the_lock_for_an_unrelated_uv_failure(tmp_path: Path): """A uv failure that is not a stale lock is still fatal, but the annotation must not claim staleness.""" root = _write_sandbox(tmp_path / "repo", sources="2.2.0", lock="2.2.0") diff --git a/scripts/bump_version.sh b/scripts/bump_version.sh index 3f342d07ac..d8fe487b98 100755 --- a/scripts/bump_version.sh +++ b/scripts/bump_version.sh @@ -5,10 +5,11 @@ # scripts/bump_version.sh # e.g. scripts/bump_version.sh 2.2.0 # # Updates: -# backend/pyproject.toml (version = "...") -# backend/uv.lock (root package version, via `uv lock`) -# frontend/package.json ("version": "...") -# deploy/helm/deer-flow/Chart.yaml (version: + appVersion:) +# backend/pyproject.toml (version = "...") +# backend/packages/harness/pyproject.toml (deerflow-harness version = "...") +# backend/uv.lock (root + harness package versions, via `uv lock`) +# frontend/package.json ("version": "...") +# deploy/helm/deer-flow/Chart.yaml (version: + appVersion:) # # Requires `uv` on PATH: backend/uv.lock pins the root package version too, and # lint CI runs `uv lock --check`, so the lock has to move with the sources. @@ -33,10 +34,11 @@ fi ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" PYPROJECT="$ROOT/backend/pyproject.toml" +HARNESS_PYPROJECT="$ROOT/backend/packages/harness/pyproject.toml" PACKAGE="$ROOT/frontend/package.json" CHART="$ROOT/deploy/helm/deer-flow/Chart.yaml" -for f in "$PYPROJECT" "$PACKAGE" "$CHART"; do +for f in "$PYPROJECT" "$HARNESS_PYPROJECT" "$PACKAGE" "$CHART"; do if [ ! -f "$f" ]; then echo "error: expected version file not found: $f" >&2 exit 1 @@ -54,20 +56,23 @@ if ! command -v uv >/dev/null 2>&1; then exit 1 fi -python3 - "$PYPROJECT" "$PACKAGE" "$CHART" "$VERSION" <<'PY' +python3 - "$PYPROJECT" "$HARNESS_PYPROJECT" "$PACKAGE" "$CHART" "$VERSION" <<'PY' import re import sys -pyproject, package, chart, version = sys.argv[1:5] +pyproject, harness, package, chart, version = sys.argv[1:6] -# backend/pyproject.toml — version = "..." -with open(pyproject) as f: - src = f.read() -new = re.sub(r'(?m)^version\s*=\s*".*?"', f'version = "{version}"', src, count=1) -if new == src: - sys.exit(f"error: no top-level 'version' field in {pyproject}") -with open(pyproject, "w") as f: - f.write(new) +# backend/pyproject.toml and backend/packages/harness/pyproject.toml — +# version = "..." (deerflow-harness is a workspace member; `uv lock` below +# moves its backend/uv.lock entry in lockstep) +for path in (pyproject, harness): + with open(path) as f: + src = f.read() + new = re.sub(r'(?m)^version\s*=\s*".*?"', f'version = "{version}"', src, count=1) + if new == src: + sys.exit(f"error: no top-level 'version' field in {path}") + with open(path, "w") as f: + f.write(new) # frontend/package.json — "version": "..." (preserve indentation; minimal diff) with open(package) as f: @@ -101,6 +106,7 @@ fi echo "Bumped version to $VERSION in:" echo " backend/pyproject.toml" +echo " backend/packages/harness/pyproject.toml" echo " backend/uv.lock" echo " frontend/package.json" echo " deploy/helm/deer-flow/Chart.yaml (version + appVersion)" diff --git a/scripts/verify_versions.sh b/scripts/verify_versions.sh index e3d1959371..bbea64eabc 100755 --- a/scripts/verify_versions.sh +++ b/scripts/verify_versions.sh @@ -2,10 +2,11 @@ # Verify that every project version source agrees. # # Sources checked: -# deploy/helm/deer-flow/Chart.yaml — version + appVersion -# backend/pyproject.toml — version -# frontend/package.json — version -# backend/uv.lock — root package version, via `uv lock --check` +# deploy/helm/deer-flow/Chart.yaml — version + appVersion +# backend/pyproject.toml — version +# backend/packages/harness/pyproject.toml — deerflow-harness version +# frontend/package.json — version +# backend/uv.lock — root + harness entries, via `uv lock --check` # # Usage: # scripts/verify_versions.sh # all sources must be mutually equal @@ -22,10 +23,11 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" CHART="$ROOT/deploy/helm/deer-flow/Chart.yaml" PYPROJECT="$ROOT/backend/pyproject.toml" +HARNESS_PYPROJECT="$ROOT/backend/packages/harness/pyproject.toml" PACKAGE="$ROOT/frontend/package.json" UV_LOCK="$ROOT/backend/uv.lock" -for f in "$CHART" "$PYPROJECT" "$PACKAGE" "$UV_LOCK"; do +for f in "$CHART" "$PYPROJECT" "$HARNESS_PYPROJECT" "$PACKAGE" "$UV_LOCK"; do if [ ! -f "$f" ]; then echo "::error::missing version file: $f" >&2 exit 1 @@ -35,6 +37,7 @@ done CHART_VERSION=$(awk '/^version:/ {print $2; exit}' "$CHART") APP_VERSION=$(awk '/^appVersion:/ {gsub(/"/, ""); print $2; exit}' "$CHART") PY_VERSION=$(awk -F'"' '/^version[[:space:]]*=/ {print $2; exit}' "$PYPROJECT") +HARNESS_VERSION=$(awk -F'"' '/^version[[:space:]]*=/ {print $2; exit}' "$HARNESS_PYPROJECT") JS_VERSION=$(grep -m1 '"version"' "$PACKAGE" | awk -F'"' '{print $4}') PY_NAME=$(awk -F'"' '/^name[[:space:]]*=/ {print $2; exit}' "$PYPROJECT") LOCK_VERSION=$(awk -v name="$PY_NAME" ' @@ -48,6 +51,7 @@ LOCK_VERSION=$(awk -v name="$PY_NAME" ' printf 'Chart.yaml version: %s\n' "$CHART_VERSION" printf 'Chart.yaml appVersion: %s\n' "$APP_VERSION" printf 'backend/pyproject.toml: %s\n' "$PY_VERSION" +printf 'harness/pyproject.toml: %s\n' "$HARNESS_VERSION" printf 'frontend/package.json: %s\n' "$JS_VERSION" printf 'backend/uv.lock: %s\n' "$LOCK_VERSION" @@ -69,11 +73,13 @@ if [ -n "$EXPECTED" ]; then mismatch "Chart.yaml version" "$CHART_VERSION" "$EXPECTED" || status=1 mismatch "Chart.yaml appVersion" "$APP_VERSION" "$EXPECTED" || status=1 mismatch "backend/pyproject.toml" "$PY_VERSION" "$EXPECTED" || status=1 + mismatch "backend/packages/harness/pyproject.toml" "$HARNESS_VERSION" "$EXPECTED" || status=1 mismatch "frontend/package.json" "$JS_VERSION" "$EXPECTED" || status=1 else echo mismatch "Chart.yaml appVersion" "$APP_VERSION" "$CHART_VERSION" || status=1 mismatch "backend/pyproject.toml" "$PY_VERSION" "$CHART_VERSION" || status=1 + mismatch "backend/packages/harness/pyproject.toml" "$HARNESS_VERSION" "$CHART_VERSION" || status=1 mismatch "frontend/package.json" "$JS_VERSION" "$CHART_VERSION" || status=1 fi