github/deer-flow
1
0
Fork 0
mirror of https://github.com/bytedance/deer-flow.git synced 2026-09-25 05:56:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(deps): bump langgraph-checkpoint to 4.2.0 / postgres 3.1.2 and delete the saver patch (#5734)

Browse Source 
* chore(checkpoint): bump to checkpoint 4.2.0 and drop the stale saver patch

langgraph-checkpoint 4.2.0 fixes the write dropped after a full -> delta
migration (upstream langchain-ai/langgraph#8526) and keeps its own
InMemorySaver override, so the compatibility patch in checkpoint_patches.py
could neither detect the fix nor stand down: it read langgraph's version
while InMemorySaver ships in langgraph-checkpoint, and it would have kept
shadowing the upstream implementation on 4.2.0. The postgres release 3.1.2
locates plain-value delta seeds (upstream #8535).

- floor langgraph-checkpoint>=4.2.0,<5.0 in the harness and raise the postgres
  extra to >=3.1.2,<3.2; langgraph's own >=4.1.0 would otherwise let a
  resolver reintroduce the bug now that the patch is gone
- delete the patch, its version guard, and the three implementation-assertion
  tests; test_full_to_delta_migration_replays_on_same_thread stays as the gate
  and fails on 4.1.1 without the patch (verified by a one-off unpatch run)
- langgraph stays 1.2.9 and langgraph-checkpoint-sqlite stays 3.1.1

Verification: make lint; delta checkpointer/cache/worker-resume suites green
with the migration gate passing on memory and sqlite (postgres params skip
without TEST_POSTGRES_URI). The full make test run before the packaging-pin
fix below was 13 failed / 18523 passed; 12 of those failures reproduce on main
with the same environment (git stash baseline, identical failure sets), 7 of
them because the local config.yaml runs checkpoint_channel_mode: delta. The
remaining one, the postgres-extra string pin in tests/test_checkpointer.py, is
updated here.

Upstream status for the surrounding gaps: cache parity across off/cold/warm
states, postgres pagination (langchain-ai/langgraph#8448) and parallel-superstep
replay order (#8382) are still open upstream, as is the abandoned-branch fork
fix (#8548) that the delta resume linearization in runtime/runs/worker.py works
around, so that path stays.

* docs(changelog): keep checkpoint upgrade in Unreleased

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
Zeren Wang 2026-09-23 03:18:26 +02:00 committed by GitHub
parent e7128dc7d6
commit 31bfd40f38
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
8 changed files with 50 additions and 141 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
CHANGELOG.md
View File

@ -2975,6 +2975,14 @@ This release closes that milestone with **765 merged pull requests**.
### Internal
- **deps:** Raise `langgraph-checkpoint` to `>=4.2.0,<5.0` and
`langgraph-checkpoint-postgres` to `>=3.1.2,<3.2`, and drop the
`InMemorySaver` delta-history compatibility patch. Upstream 4.2.0 fixes the
first write dropped after a full → delta migration
(langchain-ai/langgraph#8526) and the postgres release locates plain-value
delta seeds (langchain-ai/langgraph#8535), so the dependency floor replaces
the patch; the full → delta migration contract test remains the gate.
`langgraph` and `langgraph-checkpoint-sqlite` are unchanged. ([#5734])
- **tests:** Migrate frontend unit tests to rstest and run hook-level tests in
a DOM environment. ([#3703], [#4453])
- **tests:** Require explicit opt-in for live client tests. ([#4482])
@ -4339,3 +4347,4 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
[#5578]: https://github.com/bytedance/deer-flow/pull/5578
[#5611]: https://github.com/bytedance/deer-flow/pull/5611
[#5673]: https://github.com/bytedance/deer-flow/pull/5673
[#5734]: https://github.com/bytedance/deer-flow/pull/5734

8
CHANGELOG_zh.md
View File

@ -2225,6 +2225,13 @@
### 内部改进
- **依赖：** `langgraph-checkpoint` 下限提升到 `>=4.2.0,<5.0`，
`langgraph-checkpoint-postgres` 提升到 `>=3.1.2,<3.2`，并移除
`InMemorySaver` delta-history 兼容补丁。上游 4.2.0 修复了 full → delta
迁移后首条写入丢失（langchain-ai/langgraph#8526），postgres 新版本能定位
plain-value delta 种子（langchain-ai/langgraph#8535），因此由依赖下限取代
补丁；full → delta 迁移合约测试保留为门禁。`langgraph` 与
`langgraph-checkpoint-sqlite` 不变。 ([#5734])
- **测试：** 前端单元测试迁移到 rstest，并在 DOM 环境运行 hook 级测试。([#3703]、[#4453])
- **测试：** live client 测试要求显式 opt-in。([#4482])
- **测试：** LLM 错误测试替身不再复用共享 `FakeError`。([#4744])
@ -3527,3 +3534,4 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写，核心包含子
[#5578]: https://github.com/bytedance/deer-flow/pull/5578
[#5611]: https://github.com/bytedance/deer-flow/pull/5611
[#5673]: https://github.com/bytedance/deer-flow/pull/5673
[#5734]: https://github.com/bytedance/deer-flow/pull/5734

93
backend/packages/harness/deerflow/checkpoint_patches.py
View File

@ -6,99 +6,33 @@ imported from ``deerflow.agents.thread_state`` without pulling in the heavy
machinery). Anchored from ``deerflow.agents.thread_state`` so every process
that builds a DeerFlow graph (gateway, workers, in-process LangGraph
runtime, tests) runs with the fixes in place.
One patch remains: ``BinaryOperatorAggregate`` unwrapping an ``Overwrite``
first write into an empty (MISSING) channel (#4380). The former
``InMemorySaver`` delta-history patch was removed: upstream fixed the dropped
first post-migration write in ``langgraph-checkpoint`` 4.2.0 (upstream #8526)
while keeping its own override, so a behavioural probe cannot see the fix.
The harness dependency floor (``langgraph-checkpoint>=4.2.0``) is what keeps
that bug out, and
``tests/test_delta_channel_checkpointers.py::test_full_to_delta_migration_replays_on_same_thread``
is the regression gate that fails when the fix is absent. Do not re-add a
version-guarded saver patch: the guard that used to live here read
``langgraph``'s version, but ``InMemorySaver`` ships in the independently
released ``langgraph-checkpoint`` distribution.
"""
from __future__ import annotations
import importlib.metadata
import logging
from collections.abc import Sequence
from typing import Any
from langgraph.channels.binop import BinaryOperatorAggregate
from langgraph.checkpoint.base import BaseCheckpointSaver
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.errors import ErrorCode, InvalidUpdateError, create_error_message
from langgraph.types import Overwrite
from packaging.version import Version
logger = logging.getLogger(__name__)
_PATCH_FLAG = "_deerflow_delta_history_patched"
# The patch was authored and verified against langgraph 1.2.9
# (langgraph/checkpoint/memory/__init__.py::InMemorySaver.get_delta_channel_history).
# On any newer LangGraph the override must be re-inspected before keeping the
# patch: if upstream fixed or removed it, this module must stand down.
_PATCH_VALIDATED_LANGGRAPH_VERSION = Version("1.2.9")
def _get_delta_channel_history_via_base(self: Any, *, config: Any, channels: Any) -> Any:
return BaseCheckpointSaver.get_delta_channel_history(self, config=config, channels=channels)
async def _aget_delta_channel_history_via_base(self: Any, *, config: Any, channels: Any) -> Any:
return await BaseCheckpointSaver.aget_delta_channel_history(self, config=config, channels=channels)
def _upstream_override_present() -> bool:
"""True while InMemorySaver still ships its own (buggy) override."""
return (
getattr(InMemorySaver, "get_delta_channel_history", None) is not None
and InMemorySaver.get_delta_channel_history is not BaseCheckpointSaver.get_delta_channel_history
and InMemorySaver.aget_delta_channel_history is not BaseCheckpointSaver.aget_delta_channel_history
)
def ensure_inmemory_delta_history_patch() -> None:
"""Fix InMemorySaver dropping writes on full -> delta migrated threads.
``InMemorySaver.get_delta_channel_history`` overrides the base walk with a
single-pass version that, upon reaching the first checkpoint carrying a
non-empty plain-value blob for a channel, skips that checkpoint's *own*
pending writes as "subsumed" by the blob. That is only true when the blob
was written by that same checkpoint. When the version was carried forward
from an older ancestor - exactly the first superstep after a full -> delta
migration, where the input write lands on a checkpoint still referencing
the pre-delta blob version - those pending writes postdate the blob and
are silently dropped: the first message appended after migration vanishes
from materialized state.
Both the base implementation (used by the SQLite savers) and the Postgres
override collect the terminating checkpoint's writes *before* treating its
blob as the seed, which is the correct order. This patch delegates
InMemorySaver to the base implementation - one ``get_tuple`` per ancestor
instead of a single fused walk, which is fine for dict-backed storage.
Idempotent. Guarded: stands down when the upstream override disappears or
the assignment fails, and warns once LangGraph moves past the validated
version so the patch is re-inspected instead of silently overriding an
upstream fix. Remove once LangGraph fixes the override upstream (no
upstream issue exists yet; re-check ``InMemorySaver.get_delta_channel_history``
on every langgraph upgrade).
"""
if getattr(InMemorySaver, _PATCH_FLAG, False):
return
try:
langgraph_version = Version(importlib.metadata.version("langgraph"))
except Exception:
langgraph_version = _PATCH_VALIDATED_LANGGRAPH_VERSION
if langgraph_version > _PATCH_VALIDATED_LANGGRAPH_VERSION:
logger.warning(
"langgraph %s is newer than the version (%s) the InMemorySaver delta-history patch was validated against; re-inspect the upstream override before relying on the patch.",
langgraph_version,
_PATCH_VALIDATED_LANGGRAPH_VERSION,
)
try:
if not _upstream_override_present():
# Upstream removed its override (fixed or refactored): the base
# implementation is already in use, nothing to patch.
return
InMemorySaver.get_delta_channel_history = _get_delta_channel_history_via_base # type: ignore[method-assign]
InMemorySaver.aget_delta_channel_history = _aget_delta_channel_history_via_base # type: ignore[method-assign]
setattr(InMemorySaver, _PATCH_FLAG, True)
except (AttributeError, TypeError):
logger.warning("Failed to apply the InMemorySaver delta-history patch; leaving the upstream implementation untouched.", exc_info=True)
_BINOP_PATCH_FLAG = "_deerflow_overwrite_first_write_patched"
_unpatched_binop_update = BinaryOperatorAggregate.update
@ -188,5 +122,4 @@ def ensure_binop_overwrite_first_write_patch() -> None:
logger.warning("Failed to apply the BinaryOperatorAggregate Overwrite first-write patch; leaving the upstream implementation untouched.", exc_info=True)
ensure_inmemory_delta_history_patch()
ensure_binop_overwrite_first_write_patch()

2
backend/packages/harness/deerflow/runtime/AGENTS.md
View File

@ -241,7 +241,7 @@ the number of required IDs, whichever is larger; missing exact runs use targeted
**Where things live**:
- `runtime/checkpoint_mode.py` — mode + snapshot-frequency freeze, marker injection, delta detection, compatibility gate, both error types
- `runtime/checkpoint_state.py` — `CheckpointStateAccessor`, `build_state_mutation_graph`, `RollbackPoint`
- `checkpoint_patches.py` (package root) — checkpoint-machinery patches: delta-history folding for `InMemorySaver` (delegating to the base walk), stable message IDs across materialization, upstream first-write drop fix, and `BinaryOperatorAggregate` unwrapping an `Overwrite` first write into an empty (MISSING) channel — Union-typed reducer channels (`sandbox`/`goal`/`todos`/`promoted`) have no constructible default, so a replace-style write into a fresh branch thread or a never-written channel stored the wrapper literally and crashed the next consumer (#4380; probe-guarded, stands down if upstream fixes it)
- `checkpoint_patches.py` (package root) — the one remaining patch: `BinaryOperatorAggregate` unwrapping an `Overwrite` first write into an empty channel (#4380; probe-guarded). The `InMemorySaver` delta-history patch is gone: `langgraph-checkpoint` 4.2.0 fixed that write loss upstream (#8526) while keeping its override, so the dependency floor plus the full → delta migration contract test are the gate — never re-add a version-guarded saver patch.
- `agents/thread_state.py` — `ThreadState`/`DeltaThreadState`, `delta_messages_field` / `DELTA_MESSAGES_FIELD` (`DeltaChannel` at the configured `snapshot_frequency`, default 10), schema adaptation helpers
- `runtime/context_compaction.py` — compaction via accessor + mutation graph (reference consumer). Runs stamp their effective agent into server-owned checkpoint metadata; manual compaction uses that binding—not request `agent_name`—for memory policy and bucket. Missing/invalid legacy bindings and unreadable agent configs fail closed by skipping the optional flush while compaction may continue with the default model; a missing pre-binding checkpoint emits a warning so the skipped write is observable.
- `runtime/checkpoint_cache/` + `runtime/checkpointer/cached_saver.py` — delta-mode checkpoint history cache; checkpoint state reads MUST go through `CheckpointStateAccessor`, and the checkpointer may be a `CachedHistorySaver` wrapper — never rely on concrete saver types

10
backend/packages/harness/pyproject.toml
View File

@ -46,6 +46,12 @@ dependencies = [
"ddgs>=9.10.0",
"duckdb>=1.4.4",
"langchain-google-genai>=4.2.1",
# Floor matters: the InMemorySaver delta-history patch was removed once
# 4.2.0 fixed the dropped first post-migration write upstream (#8526) and
# kept its own override, so a 4.1.x resolution would silently reintroduce
# the bug. langgraph's own constraint (langgraph-checkpoint>=4.1.0) does
# not exclude that version.
"langgraph-checkpoint>=4.2.0,<5.0",
"langgraph-checkpoint-sqlite>=3.1.1,<3.2",
"langgraph-sdk>=0.1.51",
"sqlalchemy[asyncio]>=2.0,<3.0",
@ -73,7 +79,9 @@ groundroute = []
ollama = ["langchain-ollama>=0.3.0"]
postgres = [
"asyncpg>=0.29",
"langgraph-checkpoint-postgres>=3.1.1,<3.2",
# 3.1.2 finds plain-value delta seeds (upstream #8535); 3.1.1 walks past
# them to the thread root instead.
"langgraph-checkpoint-postgres>=3.1.2,<3.2",
"psycopg[binary]>=3.3.3",
"psycopg-pool>=3.3.0",
]

2
backend/tests/test_checkpointer.py
View File

@ -208,7 +208,7 @@ class TestHarnessPackaging:
assert "postgres" in optional_dependencies
assert optional_dependencies["postgres"] == [
"asyncpg>=0.29",
"langgraph-checkpoint-postgres>=3.1.1,<3.2",
"langgraph-checkpoint-postgres>=3.1.2,<3.2",
"psycopg[binary]>=3.3.3",
"psycopg-pool>=3.3.0",
]

51
backend/tests/test_delta_channel_checkpointers.py
View File

@ -318,54 +318,3 @@ async def test_non_delta_writers_preserve_delta_messages_and_markers(saver_env:
assert checkpoint_tuple_uses_delta(latest_tuple)
assert latest_tuple.metadata.get(CHECKPOINT_MODE_METADATA_KEY) == "delta"
assert latest_tuple.metadata.get("run_durations", {}).get("run-1") == 7
# ---------------------------------------------------------------------------
# InMemorySaver delta-history patch guards (deerflow.checkpoint_patches)
# ---------------------------------------------------------------------------
def test_inmemory_delta_history_patch_is_active() -> None:
"""The compatibility patch must be applied in every test/app process."""
from langgraph.checkpoint.memory import InMemorySaver
from deerflow import checkpoint_patches
assert getattr(InMemorySaver, checkpoint_patches._PATCH_FLAG, False) is True
assert InMemorySaver.get_delta_channel_history is checkpoint_patches._get_delta_channel_history_via_base
assert InMemorySaver.aget_delta_channel_history is checkpoint_patches._aget_delta_channel_history_via_base
def test_inmemory_delta_history_patch_stands_down_without_upstream_override(monkeypatch) -> None:
"""If upstream removes its (buggy) override, the patch must not reinstall."""
from langgraph.checkpoint.memory import InMemorySaver
from deerflow import checkpoint_patches
monkeypatch.setattr(checkpoint_patches, "_upstream_override_present", lambda: False)
monkeypatch.delattr(InMemorySaver, checkpoint_patches._PATCH_FLAG, raising=False)
sentinel = object()
monkeypatch.setattr(InMemorySaver, "get_delta_channel_history", sentinel)
checkpoint_patches.ensure_inmemory_delta_history_patch()
assert getattr(InMemorySaver, checkpoint_patches._PATCH_FLAG, False) is False
assert InMemorySaver.get_delta_channel_history is sentinel
def test_inmemory_delta_history_patch_warns_on_unvalidated_langgraph(monkeypatch, caplog) -> None:
"""A langgraph newer than the validated version must log a re-inspection warning."""
import logging
from langgraph.checkpoint.memory import InMemorySaver
from deerflow import checkpoint_patches
monkeypatch.setattr(checkpoint_patches.importlib.metadata, "version", lambda _name: "99.0.0")
monkeypatch.setattr(checkpoint_patches, "_upstream_override_present", lambda: False)
monkeypatch.delattr(InMemorySaver, checkpoint_patches._PATCH_FLAG, raising=False)
with caplog.at_level(logging.WARNING, logger=checkpoint_patches.__name__):
checkpoint_patches.ensure_inmemory_delta_history_patch()
assert any("newer than the version" in record.message for record in caplog.records)

16
backend/uv.lock generated
View File

@ -954,6 +954,7 @@ dependencies = [
{ name = "langfuse" },
{ name = "langgraph" },
{ name = "langgraph-api" },
{ name = "langgraph-checkpoint" },
{ name = "langgraph-checkpoint-sqlite" },
{ name = "langgraph-cli" },
{ name = "langgraph-runtime-inmem" },
@ -1042,7 +1043,8 @@ requires-dist = [
{ name = "langfuse", specifier = ">=3.4.1" },
{ name = "langgraph", specifier = ">=1.2.9,<1.3" },
{ name = "langgraph-api", specifier = ">=0.8.1" },
{ name = "langgraph-checkpoint-postgres", marker = "extra == 'postgres'", specifier = ">=3.1.1,<3.2" },
{ name = "langgraph-checkpoint", specifier = ">=4.2.0,<5.0" },
{ name = "langgraph-checkpoint-postgres", marker = "extra == 'postgres'", specifier = ">=3.1.2,<3.2" },
{ name = "langgraph-checkpoint-sqlite", specifier = ">=3.1.1,<3.2" },
{ name = "langgraph-cli", specifier = ">=0.4.24" },
{ name = "langgraph-runtime-inmem", specifier = ">=0.30.0" },
@ -2231,20 +2233,20 @@ wheels = [
[[package]]
name = "langgraph-checkpoint"
version = "4.1.1"
version = "4.2.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "langchain-core" },
{ name = "ormsgpack" },
]
sdist = { url = "https://files.pythonhosted.org/packages/83/47/886af6f886f0bff2273164a45f008694e48a96ff3cd25ff0228f2aa9480e/langgraph_checkpoint-4.1.1.tar.gz", hash = "sha256:6c2bdb530c91f91d7d9c1bd100925d0fc4f498d418c17f3587d1526279482a25", size = 184020, upload-time = "2026-05-22T16:57:38.503Z" }
sdist = { url = "https://files.pythonhosted.org/packages/dc/e1/089c4c9e0a2fec7f883f82ae8e6a727138d50074cfeb6644bc2d13b1019b/langgraph_checkpoint-4.2.0.tar.gz", hash = "sha256:51a593b6bee684b0818e5d6e58e28ab340c6db7794575056ce7bd1b746a84ed7", size = 180239, upload-time = "2026-08-07T20:05:03.756Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bd/b4/71425e3e38be92611300b9cc5e46a5bf98ab23f5ea8a75b73d02a2f1413c/langgraph_checkpoint-4.1.1-py3-none-any.whl", hash = "sha256:25d29144b082827218e7bc3f1e9b0566a4bb007895cd6cc26f66a8428739f56e", size = 56212, upload-time = "2026-05-22T16:57:37.203Z" },
{ url = "https://files.pythonhosted.org/packages/05/71/3b475f09bd57d3a5649792c66353312b4432afd843f301739dfcebd157f0/langgraph_checkpoint-4.2.0-py3-none-any.whl", hash = "sha256:0547fd228935a0b758865de3a3d6d7a2537c308895d0f9ab092ce9151b5da942", size = 56833, upload-time = "2026-08-07T20:05:02.655Z" },
]
[[package]]
name = "langgraph-checkpoint-postgres"
version = "3.1.1"
version = "3.1.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "langgraph-checkpoint" },
@ -2252,9 +2254,9 @@ dependencies = [
{ name = "psycopg" },
{ name = "psycopg-pool" },
]
sdist = { url = "https://files.pythonhosted.org/packages/06/92/1e8959f8cd1b56e672fde3227f6fd642be85af6c5fd662d73921074aa39d/langgraph_checkpoint_postgres-3.1.1.tar.gz", hash = "sha256:d320e147ddad8c374cd546df0b52b532dd54d0541dd9fd23fc738cbd5de76f41", size = 150413, upload-time = "2026-07-30T19:15:39.014Z" }
sdist = { url = "https://files.pythonhosted.org/packages/9d/52/6e732f7bf702ef4918b64ea4107e7da21d4276b808f9a651fe34be9b0abe/langgraph_checkpoint_postgres-3.1.2.tar.gz", hash = "sha256:1cd404803ff895a2b79f3ac04ce92b775e6b999715f8333fce674c6d927bba95", size = 155091, upload-time = "2026-08-07T20:40:00.049Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/03/32/ba457698a48a0e18d786caa770033067049fbe36d6846f8e50f13b594b51/langgraph_checkpoint_postgres-3.1.1-py3-none-any.whl", hash = "sha256:6e353aecd8150de144fef8e51a49076f58b7d6830d4cf51392b7ad4d79832ba7", size = 50778, upload-time = "2026-07-30T19:15:37.405Z" },
{ url = "https://files.pythonhosted.org/packages/0f/62/8899c9f4d9b97d5b3eb0c07c13cdf5f63342f15f5ef93eaf3477958df602/langgraph_checkpoint_postgres-3.1.2-py3-none-any.whl", hash = "sha256:6a7e38ef16985b54e356cba7bdaf447943aae33d5aaf290026c593bb6b4a6264", size = 52084, upload-time = "2026-08-07T20:39:58.994Z" },
]
[[package]]