mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-04 11:58:36 +00:00
42baed8c8c
* feat(checkpoint): dual-mode checkpoint storage with LangGraph DeltaChannel
Add a restart-required database.checkpoint_channel_mode ("full" default,
"delta") that stores the messages channel via LangGraph 1.2 DeltaChannel,
cutting checkpoint storage from O(n^2) to O(n) for append-only history.
Existing full checkpoints seed delta state transparently; no data migration.
- config: mode schema + freeze-on-first-use with
CheckpointModeReconfigurationError; mode marker persisted in checkpoint
metadata; unsafe delta->full downgrade rejected fail-closed with
CheckpointModeMismatchError (run-level error, failed state read)
- state: delta message state schema; CheckpointStateAccessor centralizes
materialized reads for all consumers (threads API, branches,
regeneration, compaction, state updates, memory, goal workers)
- runtime: raw writers (run durations, interrupted title, thread goal)
parent their checkpoints to the checkpoint they derive from, preserving
delta ancestry; rollback forks the pre-run lineage through a state
mutation graph with Overwrite restores; InMemorySaver delta-history
override delegates to the base walk (fixes dropped first write after
migration, also present upstream)
- tests: conformance suite over {memory, sqlite, postgres} covering
migration replay, stable message IDs, storage shape and writer
preservation; conftest fixture isolates the frozen mode between tests;
stale config fakes refreshed
- ci: backend unit tests gain a postgres service
* fix(checkpoint): close materialization gaps in goal flow, guard public factory
- Route goal-continuation message reads through CheckpointStateAccessor:
raw channel_values reads see the delta sentinel in delta mode, which
disabled goal continuation (stand_down=no_durable_end_of_turn) after
durable assistant turns. Raw tuples remain for tuple-only metadata
(checkpoint id, pending_writes).
- Reject checkpoint_channel_mode='delta' + checkpointer in
create_deerflow_agent at construction: factory-built persisted graphs
bypass mode-marker injection and the fail-closed gate, reproducing
silent mixed-mode state loss. Delta without persistence stays allowed.
- Import the postgres saver lazily (pytest.importorskip in the fixture)
so the documented default install collects the suite; add a CI job
running pytest --collect-only on uv sync --group dev without extras.
- Fix test_checkpointer fallback test to patch get_app_config at its
use site (provider module), making it deterministic when a local
config.yaml selects a persistent backend.
* fix(gateway): preserve extension-owned channels in state mutations, bump config version
- build_state_mutation_graph / build_checkpoint_state_mutation_accessor
accept an explicit state_schema; branch and POST /state now compile the
mutation graph from the thread's effective schema
(graph_state_schema on the assistant graph). The base-ThreadState
fallback silently discarded channels contributed by custom
AgentMiddleware.state_schema on branch (data loss) and returned a
false-success 200 on POST /state.
- POST /state validates values keys against the mutation graph's
channels and rejects unknown fields with 422 instead of ignoring
them; reducer detection covers extension channels
(BinaryOperatorAggregate or DeltaChannel) so Overwrite replace
semantics work for middleware reducers in both modes.
- Endpoint regression: custom AgentMiddleware.state_schema value
survives branch, updates through POST /state, and an unknown field
receives 422.
- config_version 26 -> 27 for the new database.checkpoint_channel_mode
(example, Helm chart values + README, support-bundle fixture), so
existing installs get the outdated-config warning and
make config-upgrade merges the field; covered by a test driving the
real example file and the real config-upgrade script.
* fix(gateway): resolve assistant schema via one boundary, copy branch reducer values with Overwrite
GET /threads/{id}/state now resolves the thread's assistant_id through a
single reusable boundary (thread metadata -> assistant_id -> effective
graph), so channels contributed by a custom AgentMiddleware.state_schema
are materialized instead of dropped by the default lead schema. POST
/state uses the same boundary instead of resolving the schema ad hoc.
Branch writes wrap every copied reducer channel in Overwrite (derived
from the effective mutation graph: BinaryOperatorAggregate + DeltaChannel),
not just messages, so already-aggregated values are never re-merged.
Regression tests use a real AgentMiddleware.state_schema with a
non-identity reducer in both full and delta modes: GET /state returns the
extension value, POST /state replaces it, branch preserves it
byte-for-byte; the unknown-field 422 is a separate assertion.
* refactor(checkpoint): collapse read-path round-trips and ship dual-mode parity tests
Address review round 4 on PR #4292:
- Push ahistory/history limit through Pregel into checkpointer.alist
(SQL LIMIT) instead of materializing all rows and breaking in Python
- Fold the read-side mode-compat gate onto the returned snapshot's
metadata; only writes keep the pre-write tuple fetch (fail-closed)
- Cache factory-built accessor graphs per (assistant_id, mode) with
factory-identity revalidation; state reads no longer build a lead
agent per request
- get_thread: one snapshot fetch + one raw pending_writes fetch on the
resolved checkpoint (post-checkpoint __error__ writes never surface
in snapshot.tasks; verified empirically)
- DeerFlowClient.get_thread: single checkpointer.list walk collects
pending_writes per checkpoint instead of N get_tuple calls
- InMemorySaver delta-history patch: stand-down when the upstream
override disappears, try/except guard, validated-version warning,
guard tests
- make_lead_agent mode precedence: first freeze is owned by app_config
(client-supplied configurable key ignored); once frozen, injected
key/app_config must match or fail closed
- Rollback: lock in non-message channel restoration via fork
inheritance with a dedicated reducer-channel test
- Add tests/test_threads_checkpoint_mode.py and
tests/test_gateway_checkpoint_mode.py referenced by AGENTS.md and
the PR validation section: lifecycle parity (memory + sqlite),
per-step blob-count storage guard, gateway endpoint parity
Counted-saver tests pin checkpoint round-trips for aget/ahistory so
these regressions cannot silently return.
* fix(checkpoint): precise mode-mismatch HTTP mapping, gate E2E, and accessor resilience
- threads router: map CheckpointModeMismatchError to 409 (with cause and
thread id) and CheckpointModeReconfigurationError to 503 across all state
endpoints instead of swallowing both into a generic 500
- gate coverage: seed a real delta checkpoint into AsyncSqliteSaver and
assert aget/aupdate/ahistory fail closed; assert 409 at the HTTP boundary
through the real route stack
- rollback: compile the restore mutation graph with the thread's effective
state schema per the build_state_mutation_graph contract
- inheritance contract locks: rollback and manual compaction preserve
middleware-contributed channels via checkpoint fork cloning
- services: revalidate the accessor-graph cache against app_config identity
so config.yaml hot-reloads never serve a stale compiled graph
- services: degrade full-mode state reads to raw checkpointer reads when the
agent factory is unavailable (delta gate still applies; delta mode has no
fallback)
- deps: override websockets==16.0 (langgraph-sdk 0.4.2's <16 pin silently
downgraded 16.0 -> 15.0.1; pin is not grounded in any API incompatibility)
and bump the langchain lower bound to what the lockfile actually resolves
* fix(checkpoint): include anchor checkpoint in degraded history walk + cover get_thread
- _RawCheckpointReadAccessor.ahistory: alist(before=...) is exclusive while
pregel's get_state_history treats config.checkpoint_id as the inclusive
start; fetch the anchor explicitly so both read paths paginate identically
- extend the degraded-path gateway test: GET /thread returns raw values, and
POST /history with before starts at the anchor checkpoint
* fix(gateway): preserve degraded checkpoint timestamps
* fix(gateway): harden degraded checkpoint access
* fix(gateway): resolve assistants for checkpoint reads
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
1078 lines
44 KiB
Python
1078 lines
44 KiB
Python
"""Unit tests for checkpointer config, packaging metadata, and factories."""
|
|
|
|
import sys
|
|
import tomllib
|
|
from concurrent.futures import ThreadPoolExecutor
|
|
from contextlib import nullcontext
|
|
from pathlib import Path
|
|
from threading import Barrier, Event, Lock
|
|
from types import SimpleNamespace
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
import pytest
|
|
|
|
import deerflow.config.app_config as app_config_module
|
|
from deerflow.config.checkpointer_config import (
|
|
CheckpointerConfig,
|
|
ensure_config_loaded,
|
|
get_checkpointer_config,
|
|
load_checkpointer_config_from_dict,
|
|
set_checkpointer_config,
|
|
)
|
|
from deerflow.config.database_config import DatabaseConfig
|
|
from deerflow.runtime.checkpointer import get_checkpointer, reset_checkpointer
|
|
from deerflow.runtime.checkpointer.provider import POSTGRES_INSTALL
|
|
from deerflow.runtime.store import get_store, reset_store
|
|
from deerflow.runtime.store.provider import POSTGRES_STORE_INSTALL
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def reset_state():
|
|
"""Reset singleton state before each test."""
|
|
app_config_module._app_config = None
|
|
set_checkpointer_config(None)
|
|
reset_checkpointer()
|
|
reset_store()
|
|
yield
|
|
app_config_module._app_config = None
|
|
set_checkpointer_config(None)
|
|
reset_checkpointer()
|
|
reset_store()
|
|
|
|
|
|
class _BlockingSingletonContext:
|
|
def __init__(self, value: object, entered: Event, release: Event, stats: dict[str, object]):
|
|
self._value = value
|
|
self._entered = entered
|
|
self._release = release
|
|
self._stats = stats
|
|
|
|
def __enter__(self):
|
|
with self._stats["lock"]:
|
|
self._stats["enters"] += 1
|
|
self._entered.set()
|
|
assert self._release.wait(timeout=3), "timed out waiting to release singleton initialization"
|
|
return self._value
|
|
|
|
def __exit__(self, exc_type, exc, tb):
|
|
with self._stats["lock"]:
|
|
self._stats["exits"] += 1
|
|
return False
|
|
|
|
|
|
class _BlockingSingletonFactory:
|
|
def __init__(self):
|
|
self.value = object()
|
|
self.entered = Event()
|
|
self.release = Event()
|
|
self.stats = {"enters": 0, "exits": 0, "lock": Lock()}
|
|
|
|
def context_manager(self, _config):
|
|
return _BlockingSingletonContext(self.value, self.entered, self.release, self.stats)
|
|
|
|
def enter_count(self) -> int:
|
|
with self.stats["lock"]:
|
|
return self.stats["enters"]
|
|
|
|
def exit_count(self) -> int:
|
|
with self.stats["lock"]:
|
|
return self.stats["exits"]
|
|
|
|
|
|
class _TrackingLock:
|
|
def __init__(self):
|
|
self._lock = Lock()
|
|
self.acquired = Event()
|
|
|
|
def acquire(self, *args, **kwargs):
|
|
acquired = self._lock.acquire(*args, **kwargs)
|
|
if acquired:
|
|
self.acquired.set()
|
|
return acquired
|
|
|
|
def release(self):
|
|
self._lock.release()
|
|
|
|
def __enter__(self):
|
|
self.acquire()
|
|
return self
|
|
|
|
def __exit__(self, exc_type, exc, tb):
|
|
self.release()
|
|
return False
|
|
|
|
def locked(self) -> bool:
|
|
return self._lock.locked()
|
|
|
|
|
|
def _call_getter_concurrently(getter, workers: int = 8) -> list[object]:
|
|
ready = Barrier(workers + 1)
|
|
|
|
def worker():
|
|
ready.wait(timeout=3)
|
|
return getter()
|
|
|
|
with ThreadPoolExecutor(max_workers=workers) as executor:
|
|
futures = [executor.submit(worker) for _ in range(workers)]
|
|
ready.wait(timeout=3)
|
|
return [future.result(timeout=3) for future in futures]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Config tests
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestCheckpointerConfig:
|
|
def test_load_memory_config(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
config = get_checkpointer_config()
|
|
assert config is not None
|
|
assert config.type == "memory"
|
|
assert config.connection_string is None
|
|
|
|
def test_load_sqlite_config(self):
|
|
load_checkpointer_config_from_dict({"type": "sqlite", "connection_string": "/tmp/test.db"})
|
|
config = get_checkpointer_config()
|
|
assert config is not None
|
|
assert config.type == "sqlite"
|
|
assert config.connection_string == "/tmp/test.db"
|
|
|
|
def test_load_postgres_config(self):
|
|
load_checkpointer_config_from_dict({"type": "postgres", "connection_string": "postgresql://localhost/db"})
|
|
config = get_checkpointer_config()
|
|
assert config is not None
|
|
assert config.type == "postgres"
|
|
assert config.connection_string == "postgresql://localhost/db"
|
|
|
|
def test_default_connection_string_is_none(self):
|
|
config = CheckpointerConfig(type="memory")
|
|
assert config.connection_string is None
|
|
|
|
def test_set_config_to_none(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
set_checkpointer_config(None)
|
|
assert get_checkpointer_config() is None
|
|
|
|
def test_ensure_config_loaded_loads_app_config_when_uninitialized(self):
|
|
def fake_get_app_config():
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
|
|
with patch("deerflow.config.app_config.get_app_config", side_effect=fake_get_app_config) as mock_get_app_config:
|
|
ensure_config_loaded()
|
|
|
|
mock_get_app_config.assert_called_once()
|
|
config = get_checkpointer_config()
|
|
assert config is not None
|
|
assert config.type == "memory"
|
|
|
|
def test_ensure_config_loaded_skips_explicit_config(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
|
|
with patch("deerflow.config.app_config.get_app_config") as mock_get_app_config:
|
|
ensure_config_loaded()
|
|
|
|
mock_get_app_config.assert_not_called()
|
|
|
|
def test_invalid_type_raises(self):
|
|
with pytest.raises(Exception):
|
|
load_checkpointer_config_from_dict({"type": "unknown"})
|
|
|
|
def test_connection_string_description_matches_runtime_defaults(self):
|
|
description = CheckpointerConfig.model_fields["connection_string"].description
|
|
|
|
assert description is not None
|
|
assert "Optional for sqlite" in description
|
|
assert "defaults to 'store.db'" in description
|
|
assert "Required for postgres" in description
|
|
|
|
|
|
class TestHarnessPackaging:
|
|
def test_pyproject_declares_postgres_extra(self):
|
|
pyproject_path = Path(__file__).resolve().parents[1] / "packages" / "harness" / "pyproject.toml"
|
|
data = tomllib.loads(pyproject_path.read_text())
|
|
|
|
optional_dependencies = data["project"]["optional-dependencies"]
|
|
assert "postgres" in optional_dependencies
|
|
assert optional_dependencies["postgres"] == [
|
|
"asyncpg>=0.29",
|
|
"langgraph-checkpoint-postgres>=3.1.0,<3.2",
|
|
"psycopg[binary]>=3.3.3",
|
|
"psycopg-pool>=3.3.0",
|
|
]
|
|
|
|
def test_workspace_pyproject_forwards_postgres_extra_to_harness(self):
|
|
pyproject_path = Path(__file__).resolve().parents[1] / "pyproject.toml"
|
|
data = tomllib.loads(pyproject_path.read_text())
|
|
|
|
optional_dependencies = data["project"]["optional-dependencies"]
|
|
assert optional_dependencies["postgres"] == ["deerflow-harness[postgres]"]
|
|
|
|
def test_postgres_missing_dependency_messages_recommend_package_extra(self):
|
|
assert "deerflow-harness[postgres]" in POSTGRES_INSTALL
|
|
assert "deerflow-harness[postgres]" in POSTGRES_STORE_INSTALL
|
|
assert "uv sync --all-packages --extra postgres" in POSTGRES_INSTALL
|
|
assert "uv sync --all-packages --extra postgres" in POSTGRES_STORE_INSTALL
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Factory tests
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestGetCheckpointer:
|
|
def test_returns_in_memory_saver_when_not_configured(self):
|
|
"""get_checkpointer should return InMemorySaver when not configured."""
|
|
from langgraph.checkpoint.memory import InMemorySaver
|
|
|
|
with patch("deerflow.runtime.checkpointer.provider.get_app_config", side_effect=FileNotFoundError):
|
|
cp = get_checkpointer()
|
|
assert cp is not None
|
|
assert isinstance(cp, InMemorySaver)
|
|
|
|
def test_memory_returns_in_memory_saver(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
from langgraph.checkpoint.memory import InMemorySaver
|
|
|
|
cp = get_checkpointer()
|
|
assert isinstance(cp, InMemorySaver)
|
|
|
|
def test_memory_singleton(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
cp1 = get_checkpointer()
|
|
cp2 = get_checkpointer()
|
|
assert cp1 is cp2
|
|
|
|
def test_reset_clears_singleton(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
cp1 = get_checkpointer()
|
|
reset_checkpointer()
|
|
cp2 = get_checkpointer()
|
|
assert cp1 is not cp2
|
|
|
|
def test_sqlite_raises_when_package_missing(self):
|
|
load_checkpointer_config_from_dict({"type": "sqlite", "connection_string": "/tmp/test.db"})
|
|
with patch.dict(sys.modules, {"langgraph.checkpoint.sqlite": None}):
|
|
reset_checkpointer()
|
|
with pytest.raises(ImportError, match="langgraph-checkpoint-sqlite"):
|
|
get_checkpointer()
|
|
|
|
def test_postgres_raises_when_package_missing(self):
|
|
load_checkpointer_config_from_dict({"type": "postgres", "connection_string": "postgresql://localhost/db"})
|
|
with patch.dict(sys.modules, {"langgraph.checkpoint.postgres": None}):
|
|
reset_checkpointer()
|
|
with pytest.raises(ImportError, match="langgraph-checkpoint-postgres"):
|
|
get_checkpointer()
|
|
|
|
def test_postgres_raises_when_connection_string_missing(self):
|
|
load_checkpointer_config_from_dict({"type": "postgres"})
|
|
mock_saver = MagicMock()
|
|
mock_module = MagicMock()
|
|
mock_module.PostgresSaver = mock_saver
|
|
with patch.dict(sys.modules, {"langgraph.checkpoint.postgres": mock_module}):
|
|
reset_checkpointer()
|
|
with pytest.raises(ValueError, match="connection_string is required"):
|
|
get_checkpointer()
|
|
|
|
def test_sqlite_creates_saver(self):
|
|
"""SQLite checkpointer is created when package is available."""
|
|
load_checkpointer_config_from_dict({"type": "sqlite", "connection_string": "/tmp/test.db"})
|
|
|
|
mock_saver_instance = MagicMock()
|
|
mock_cm = MagicMock()
|
|
mock_cm.__enter__ = MagicMock(return_value=mock_saver_instance)
|
|
mock_cm.__exit__ = MagicMock(return_value=False)
|
|
|
|
mock_saver_cls = MagicMock()
|
|
mock_saver_cls.from_conn_string = MagicMock(return_value=mock_cm)
|
|
|
|
mock_module = MagicMock()
|
|
mock_module.SqliteSaver = mock_saver_cls
|
|
|
|
with patch.dict(sys.modules, {"langgraph.checkpoint.sqlite": mock_module}):
|
|
reset_checkpointer()
|
|
cp = get_checkpointer()
|
|
|
|
assert cp is mock_saver_instance
|
|
mock_saver_cls.from_conn_string.assert_called_once()
|
|
mock_saver_instance.setup.assert_called_once()
|
|
|
|
def test_sqlite_creates_parent_dir(self):
|
|
"""Sync SQLite checkpointer should call ensure_sqlite_parent_dir before connecting.
|
|
|
|
This mirrors the async checkpointer's behaviour and prevents
|
|
'sqlite3.OperationalError: unable to open database file' when the
|
|
parent directory for the database file does not yet exist (e.g. when
|
|
using the harness package from an external virtualenv where the
|
|
.deer-flow directory has not been created).
|
|
"""
|
|
load_checkpointer_config_from_dict({"type": "sqlite", "connection_string": "relative/test.db"})
|
|
|
|
mock_saver_instance = MagicMock()
|
|
mock_cm = MagicMock()
|
|
mock_cm.__enter__ = MagicMock(return_value=mock_saver_instance)
|
|
mock_cm.__exit__ = MagicMock(return_value=False)
|
|
|
|
mock_saver_cls = MagicMock()
|
|
mock_saver_cls.from_conn_string = MagicMock(return_value=mock_cm)
|
|
|
|
mock_module = MagicMock()
|
|
mock_module.SqliteSaver = mock_saver_cls
|
|
|
|
with (
|
|
patch.dict(sys.modules, {"langgraph.checkpoint.sqlite": mock_module}),
|
|
patch("deerflow.runtime.checkpointer.provider.ensure_sqlite_parent_dir") as mock_ensure,
|
|
patch(
|
|
"deerflow.runtime.checkpointer.provider.resolve_sqlite_conn_str",
|
|
return_value="/tmp/resolved/relative/test.db",
|
|
),
|
|
):
|
|
reset_checkpointer()
|
|
cp = get_checkpointer()
|
|
|
|
assert cp is mock_saver_instance
|
|
mock_ensure.assert_called_once_with("/tmp/resolved/relative/test.db")
|
|
mock_saver_cls.from_conn_string.assert_called_once_with("/tmp/resolved/relative/test.db")
|
|
|
|
def test_sqlite_ensure_parent_dir_before_connect(self):
|
|
"""ensure_sqlite_parent_dir must be called before from_conn_string."""
|
|
load_checkpointer_config_from_dict({"type": "sqlite", "connection_string": "relative/test.db"})
|
|
|
|
call_order = []
|
|
|
|
mock_saver_instance = MagicMock()
|
|
mock_cm = MagicMock()
|
|
mock_cm.__enter__ = MagicMock(return_value=mock_saver_instance)
|
|
mock_cm.__exit__ = MagicMock(return_value=False)
|
|
|
|
mock_saver_cls = MagicMock()
|
|
mock_saver_cls.from_conn_string = MagicMock(side_effect=lambda *a, **kw: (call_order.append("connect"), mock_cm)[1])
|
|
|
|
mock_module = MagicMock()
|
|
mock_module.SqliteSaver = mock_saver_cls
|
|
|
|
def record_ensure(*a, **kw):
|
|
call_order.append("ensure")
|
|
|
|
with (
|
|
patch.dict(sys.modules, {"langgraph.checkpoint.sqlite": mock_module}),
|
|
patch(
|
|
"deerflow.runtime.checkpointer.provider.ensure_sqlite_parent_dir",
|
|
side_effect=record_ensure,
|
|
),
|
|
patch(
|
|
"deerflow.runtime.checkpointer.provider.resolve_sqlite_conn_str",
|
|
return_value="/tmp/resolved/relative/test.db",
|
|
),
|
|
):
|
|
reset_checkpointer()
|
|
get_checkpointer()
|
|
|
|
assert call_order == ["ensure", "connect"]
|
|
|
|
def test_postgres_creates_saver(self):
|
|
"""Postgres checkpointer is created when packages are available."""
|
|
load_checkpointer_config_from_dict({"type": "postgres", "connection_string": "postgresql://localhost/db"})
|
|
|
|
mock_saver_instance = MagicMock()
|
|
mock_cm = MagicMock()
|
|
mock_cm.__enter__ = MagicMock(return_value=mock_saver_instance)
|
|
mock_cm.__exit__ = MagicMock(return_value=False)
|
|
|
|
mock_saver_cls = MagicMock()
|
|
mock_saver_cls.from_conn_string = MagicMock(return_value=mock_cm)
|
|
|
|
mock_pg_module = MagicMock()
|
|
mock_pg_module.PostgresSaver = mock_saver_cls
|
|
|
|
with patch.dict(sys.modules, {"langgraph.checkpoint.postgres": mock_pg_module}):
|
|
reset_checkpointer()
|
|
cp = get_checkpointer()
|
|
|
|
assert cp is mock_saver_instance
|
|
mock_saver_cls.from_conn_string.assert_called_once_with("postgresql://localhost/db")
|
|
mock_saver_instance.setup.assert_called_once()
|
|
|
|
|
|
class TestSyncSingletonThreadSafety:
|
|
def test_store_reset_clears_singleton(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
store1 = get_store()
|
|
reset_store()
|
|
store2 = get_store()
|
|
assert store1 is not store2
|
|
|
|
def test_concurrent_checkpointer_getter_creates_one_instance(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
factory = _BlockingSingletonFactory()
|
|
|
|
with patch("deerflow.runtime.checkpointer.provider._sync_checkpointer_cm", side_effect=factory.context_manager):
|
|
futures_started = ThreadPoolExecutor(max_workers=1)
|
|
try:
|
|
result_future = futures_started.submit(_call_getter_concurrently, get_checkpointer)
|
|
assert factory.entered.wait(timeout=3)
|
|
factory.release.wait(timeout=0.05)
|
|
factory.release.set()
|
|
results = result_future.result(timeout=3)
|
|
finally:
|
|
futures_started.shutdown(wait=True)
|
|
|
|
assert all(result is factory.value for result in results)
|
|
assert factory.enter_count() == 1
|
|
|
|
def test_concurrent_store_getter_creates_one_instance(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
factory = _BlockingSingletonFactory()
|
|
|
|
with patch("deerflow.runtime.store.provider._sync_store_cm", side_effect=factory.context_manager):
|
|
futures_started = ThreadPoolExecutor(max_workers=1)
|
|
try:
|
|
result_future = futures_started.submit(_call_getter_concurrently, get_store)
|
|
assert factory.entered.wait(timeout=3)
|
|
factory.release.wait(timeout=0.05)
|
|
factory.release.set()
|
|
results = result_future.result(timeout=3)
|
|
finally:
|
|
futures_started.shutdown(wait=True)
|
|
|
|
assert all(result is factory.value for result in results)
|
|
assert factory.enter_count() == 1
|
|
|
|
def test_checkpointer_loads_config_outside_singleton_lock(self):
|
|
tracking_lock = _TrackingLock()
|
|
|
|
def fake_ensure_config_loaded():
|
|
assert not tracking_lock.locked()
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.provider._checkpointer_lock", tracking_lock),
|
|
patch("deerflow.runtime.checkpointer.provider.ensure_config_loaded", side_effect=fake_ensure_config_loaded),
|
|
):
|
|
checkpointer = get_checkpointer()
|
|
|
|
assert checkpointer is not None
|
|
assert tracking_lock.acquired.is_set()
|
|
|
|
def test_store_loads_config_outside_singleton_lock(self):
|
|
tracking_lock = _TrackingLock()
|
|
|
|
def fake_ensure_config_loaded():
|
|
assert not tracking_lock.locked()
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
|
|
with (
|
|
patch("deerflow.runtime.store.provider._store_lock", tracking_lock),
|
|
patch("deerflow.runtime.store.provider.ensure_config_loaded", side_effect=fake_ensure_config_loaded),
|
|
):
|
|
store = get_store()
|
|
|
|
assert store is not None
|
|
assert tracking_lock.acquired.is_set()
|
|
|
|
def test_checkpointer_reset_waits_for_initialization(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
factory = _BlockingSingletonFactory()
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.provider._sync_checkpointer_cm", side_effect=factory.context_manager),
|
|
ThreadPoolExecutor(max_workers=2) as executor,
|
|
):
|
|
get_future = executor.submit(get_checkpointer)
|
|
assert factory.entered.wait(timeout=3)
|
|
|
|
reset_started = Event()
|
|
|
|
def reset_worker():
|
|
reset_started.set()
|
|
reset_checkpointer()
|
|
|
|
reset_future = executor.submit(reset_worker)
|
|
assert reset_started.wait(timeout=3)
|
|
factory.release.wait(timeout=0.05)
|
|
|
|
assert not reset_future.done()
|
|
assert factory.exit_count() == 0
|
|
|
|
factory.release.set()
|
|
assert get_future.result(timeout=3) is factory.value
|
|
reset_future.result(timeout=3)
|
|
|
|
assert factory.exit_count() == 1
|
|
|
|
def test_store_reset_waits_for_initialization(self):
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
factory = _BlockingSingletonFactory()
|
|
|
|
with (
|
|
patch("deerflow.runtime.store.provider._sync_store_cm", side_effect=factory.context_manager),
|
|
ThreadPoolExecutor(max_workers=2) as executor,
|
|
):
|
|
get_future = executor.submit(get_store)
|
|
assert factory.entered.wait(timeout=3)
|
|
|
|
reset_started = Event()
|
|
|
|
def reset_worker():
|
|
reset_started.set()
|
|
reset_store()
|
|
|
|
reset_future = executor.submit(reset_worker)
|
|
assert reset_started.wait(timeout=3)
|
|
factory.release.wait(timeout=0.05)
|
|
|
|
assert not reset_future.done()
|
|
assert factory.exit_count() == 0
|
|
|
|
factory.release.set()
|
|
assert get_future.result(timeout=3) is factory.value
|
|
reset_future.result(timeout=3)
|
|
|
|
assert factory.exit_count() == 1
|
|
|
|
|
|
class TestAsyncCheckpointer:
|
|
@pytest.mark.anyio
|
|
async def test_sqlite_creates_parent_dir_via_to_thread(self):
|
|
"""Async SQLite setup should move mkdir off the event loop."""
|
|
from deerflow.runtime.checkpointer.async_provider import _prepare_sqlite_checkpointer_path, make_checkpointer
|
|
|
|
mock_config = MagicMock()
|
|
mock_config.checkpointer = CheckpointerConfig(type="sqlite", connection_string="relative/test.db")
|
|
|
|
mock_saver = AsyncMock()
|
|
mock_cm = AsyncMock()
|
|
mock_cm.__aenter__.return_value = mock_saver
|
|
mock_cm.__aexit__.return_value = False
|
|
|
|
mock_saver_cls = MagicMock()
|
|
mock_saver_cls.from_conn_string.return_value = mock_cm
|
|
|
|
mock_module = MagicMock()
|
|
mock_module.AsyncSqliteSaver = mock_saver_cls
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.async_provider.get_app_config", return_value=mock_config),
|
|
patch.dict(sys.modules, {"langgraph.checkpoint.sqlite.aio": mock_module}),
|
|
patch(
|
|
"deerflow.runtime.checkpointer.async_provider.asyncio.to_thread",
|
|
new_callable=AsyncMock,
|
|
return_value="/tmp/resolved/test.db",
|
|
) as mock_to_thread,
|
|
):
|
|
async with make_checkpointer() as saver:
|
|
assert saver is mock_saver
|
|
|
|
mock_to_thread.assert_awaited_once()
|
|
called_fn, called_path = mock_to_thread.await_args.args
|
|
assert called_fn is _prepare_sqlite_checkpointer_path
|
|
assert called_path == "relative/test.db"
|
|
mock_saver_cls.from_conn_string.assert_called_once_with("/tmp/resolved/test.db")
|
|
mock_saver.setup.assert_awaited_once()
|
|
|
|
@pytest.mark.anyio
|
|
async def test_postgres_uses_connection_pool(self):
|
|
"""Async postgres checkpointer should use AsyncConnectionPool, not a single connection."""
|
|
from deerflow.runtime.checkpointer.async_provider import make_checkpointer
|
|
|
|
mock_config = MagicMock()
|
|
mock_config.checkpointer = CheckpointerConfig(type="postgres", connection_string="postgresql://localhost/db")
|
|
|
|
mock_saver = AsyncMock()
|
|
|
|
mock_saver_cls = MagicMock(return_value=mock_saver)
|
|
|
|
mock_pool_instance = AsyncMock()
|
|
mock_pool_instance.__aenter__.return_value = mock_pool_instance
|
|
mock_pool_instance.__aexit__.return_value = False
|
|
|
|
mock_pool_cls = MagicMock(return_value=mock_pool_instance)
|
|
mock_pool_cls.check_connection = AsyncMock()
|
|
mock_dict_row = MagicMock()
|
|
|
|
mock_pg_module = MagicMock()
|
|
mock_pg_module.AsyncPostgresSaver = mock_saver_cls
|
|
|
|
mock_psycopg_rows = MagicMock()
|
|
mock_psycopg_rows.dict_row = mock_dict_row
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.async_provider.get_app_config", return_value=mock_config),
|
|
patch.dict(sys.modules, {"langgraph.checkpoint.postgres.aio": mock_pg_module}),
|
|
patch.dict(sys.modules, {"psycopg.rows": mock_psycopg_rows}),
|
|
patch.dict(sys.modules, {"psycopg_pool": MagicMock(AsyncConnectionPool=mock_pool_cls)}),
|
|
):
|
|
# AsyncConnectionPool() is a callable that returns mock_pool_instance
|
|
# We need the constructor to be an async context manager
|
|
async with make_checkpointer() as saver:
|
|
assert saver is mock_saver
|
|
|
|
# Verify the pool was constructed with check Connection
|
|
mock_pool_cls.assert_called_once()
|
|
call_kwargs = mock_pool_cls.call_args
|
|
assert call_kwargs[0][0] == "postgresql://localhost/db"
|
|
assert call_kwargs[1]["check"] is mock_pool_cls.check_connection
|
|
|
|
# Verify saver was constructed with the pool (not via from_conn_string)
|
|
mock_saver_cls.assert_called_once_with(conn=mock_pool_instance)
|
|
mock_saver.setup.assert_awaited_once()
|
|
|
|
@pytest.mark.anyio
|
|
async def test_database_postgres_uses_connection_pool(self):
|
|
"""Unified database postgres path should use AsyncConnectionPool with keepalive."""
|
|
from deerflow.config.database_config import DatabaseConfig
|
|
from deerflow.runtime.checkpointer.async_provider import make_checkpointer
|
|
|
|
db_config = DatabaseConfig(backend="postgres", postgres_url="postgresql://localhost/db")
|
|
mock_config = MagicMock()
|
|
mock_config.checkpointer = None
|
|
mock_config.database = db_config
|
|
|
|
mock_saver = AsyncMock()
|
|
|
|
mock_saver_cls = MagicMock(return_value=mock_saver)
|
|
|
|
mock_pool_instance = AsyncMock()
|
|
mock_pool_instance.__aenter__.return_value = mock_pool_instance
|
|
mock_pool_instance.__aexit__.return_value = False
|
|
|
|
mock_pool_cls = MagicMock(return_value=mock_pool_instance)
|
|
mock_pool_cls.check_connection = AsyncMock()
|
|
mock_dict_row = MagicMock()
|
|
|
|
mock_pg_module = MagicMock()
|
|
mock_pg_module.AsyncPostgresSaver = mock_saver_cls
|
|
|
|
mock_psycopg_rows = MagicMock()
|
|
mock_psycopg_rows.dict_row = mock_dict_row
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.async_provider.get_app_config", return_value=mock_config),
|
|
patch.dict(sys.modules, {"langgraph.checkpoint.postgres.aio": mock_pg_module}),
|
|
patch.dict(sys.modules, {"psycopg.rows": mock_psycopg_rows}),
|
|
patch.dict(sys.modules, {"psycopg_pool": MagicMock(AsyncConnectionPool=mock_pool_cls)}),
|
|
):
|
|
async with make_checkpointer() as saver:
|
|
assert saver is mock_saver
|
|
|
|
mock_pool_cls.assert_called_once()
|
|
call_kwargs = mock_pool_cls.call_args
|
|
assert call_kwargs[0][0] == "postgresql://localhost/db"
|
|
assert call_kwargs[1]["check"] is mock_pool_cls.check_connection
|
|
|
|
mock_saver_cls.assert_called_once_with(conn=mock_pool_instance)
|
|
mock_saver.setup.assert_awaited_once()
|
|
|
|
@pytest.mark.anyio
|
|
async def test_database_sqlite_creates_parent_dir_via_to_thread(self):
|
|
"""Unified database SQLite setup should also move path IO off the event loop."""
|
|
from deerflow.config.database_config import DatabaseConfig
|
|
from deerflow.runtime.checkpointer.async_provider import _prepare_database_sqlite_checkpointer_path, make_checkpointer
|
|
|
|
db_config = DatabaseConfig(backend="sqlite", sqlite_dir="relative-data")
|
|
mock_config = MagicMock()
|
|
mock_config.checkpointer = None
|
|
mock_config.database = db_config
|
|
|
|
mock_saver = AsyncMock()
|
|
mock_cm = AsyncMock()
|
|
mock_cm.__aenter__.return_value = mock_saver
|
|
mock_cm.__aexit__.return_value = False
|
|
|
|
mock_saver_cls = MagicMock()
|
|
mock_saver_cls.from_conn_string.return_value = mock_cm
|
|
|
|
mock_module = MagicMock()
|
|
mock_module.AsyncSqliteSaver = mock_saver_cls
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.async_provider.get_app_config", return_value=mock_config),
|
|
patch.dict(sys.modules, {"langgraph.checkpoint.sqlite.aio": mock_module}),
|
|
patch(
|
|
"deerflow.runtime.checkpointer.async_provider.asyncio.to_thread",
|
|
new_callable=AsyncMock,
|
|
return_value="/tmp/data/deerflow.db",
|
|
) as mock_to_thread,
|
|
):
|
|
async with make_checkpointer() as saver:
|
|
assert saver is mock_saver
|
|
|
|
mock_to_thread.assert_awaited_once()
|
|
called_fn, called_db_config = mock_to_thread.await_args.args
|
|
assert called_fn is _prepare_database_sqlite_checkpointer_path
|
|
assert called_db_config is db_config
|
|
mock_saver_cls.from_conn_string.assert_called_once_with("/tmp/data/deerflow.db")
|
|
mock_saver.setup.assert_awaited_once()
|
|
|
|
|
|
class TestCheckpointerDatabaseConfig:
|
|
"""The sync checkpointer must follow the unified ``database`` section when no
|
|
legacy ``checkpointer`` section is configured — matching the async
|
|
``make_checkpointer`` factory and the sync Store provider.
|
|
|
|
Regression: ``get_checkpointer`` / ``checkpointer_context`` previously read
|
|
only the legacy ``checkpointer`` section and fell back to ``InMemorySaver``,
|
|
silently ignoring ``database``. Embedded callers (``DeerFlowClient``) and the
|
|
TUI then persisted Store rows to sqlite/postgres while checkpoints went to an
|
|
in-memory saver and were lost on exit.
|
|
"""
|
|
|
|
def test_sync_checkpointer_context_uses_database_config(self):
|
|
"""The one-shot sync checkpointer factory must follow unified database config."""
|
|
from deerflow.runtime.checkpointer.provider import checkpointer_context
|
|
|
|
app_config = SimpleNamespace(
|
|
checkpointer=None,
|
|
database=DatabaseConfig(backend="postgres", postgres_url="postgresql://localhost/db"),
|
|
)
|
|
expected = object()
|
|
factory = MagicMock(return_value=nullcontext(expected))
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.provider.get_app_config", return_value=app_config),
|
|
patch("deerflow.runtime.checkpointer.provider._sync_checkpointer_cm", factory),
|
|
checkpointer_context() as cp,
|
|
):
|
|
assert cp is expected
|
|
|
|
resolved = factory.call_args.args[0]
|
|
assert resolved.type == "postgres"
|
|
assert resolved.connection_string == "postgresql://localhost/db"
|
|
|
|
def test_sync_checkpointer_context_uses_sqlite_database_config(self, tmp_path):
|
|
"""The one-shot sync checkpointer factory must resolve the sqlite branch too, not just postgres."""
|
|
from deerflow.runtime.checkpointer.provider import checkpointer_context
|
|
|
|
db_config = DatabaseConfig(backend="sqlite", sqlite_dir=str(tmp_path))
|
|
app_config = SimpleNamespace(checkpointer=None, database=db_config)
|
|
expected = object()
|
|
factory = MagicMock(return_value=nullcontext(expected))
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.provider.get_app_config", return_value=app_config),
|
|
patch("deerflow.runtime.checkpointer.provider._sync_checkpointer_cm", factory),
|
|
checkpointer_context() as cp,
|
|
):
|
|
assert cp is expected
|
|
|
|
resolved = factory.call_args.args[0]
|
|
assert resolved.type == "sqlite"
|
|
assert resolved.connection_string == db_config.checkpointer_sqlite_path
|
|
|
|
def test_sync_checkpointer_singleton_uses_database_config(self):
|
|
"""The cached sync checkpointer factory must resolve database config before locking."""
|
|
app_config = SimpleNamespace(
|
|
checkpointer=None,
|
|
database=DatabaseConfig(backend="postgres", postgres_url="postgresql://localhost/db"),
|
|
)
|
|
expected = object()
|
|
factory = MagicMock(return_value=nullcontext(expected))
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.provider.ensure_config_loaded"),
|
|
patch("deerflow.runtime.checkpointer.provider.get_app_config", return_value=app_config),
|
|
patch("deerflow.runtime.checkpointer.provider._sync_checkpointer_cm", factory),
|
|
):
|
|
assert get_checkpointer() is expected
|
|
|
|
resolved = factory.call_args.args[0]
|
|
assert resolved.type == "postgres"
|
|
assert resolved.connection_string == "postgresql://localhost/db"
|
|
|
|
def test_sync_checkpointer_falls_back_to_memory_when_config_file_is_missing(self):
|
|
"""The sync checkpointer keeps its no-config fallback for embedded callers."""
|
|
from langgraph.checkpoint.memory import InMemorySaver
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.provider.ensure_config_loaded"),
|
|
patch("deerflow.runtime.checkpointer.provider.get_checkpointer_config", return_value=None),
|
|
patch("deerflow.runtime.checkpointer.provider.get_app_config", side_effect=FileNotFoundError),
|
|
):
|
|
assert isinstance(get_checkpointer(), InMemorySaver)
|
|
|
|
def test_legacy_checkpointer_config_takes_precedence(self):
|
|
"""Backward-compatible checkpointer config must override database."""
|
|
from deerflow.runtime.checkpointer.provider import checkpointer_context
|
|
|
|
app_config = SimpleNamespace(
|
|
checkpointer=CheckpointerConfig(type="memory"),
|
|
database=DatabaseConfig(backend="postgres", postgres_url="postgresql://localhost/db"),
|
|
)
|
|
expected = object()
|
|
factory = MagicMock(return_value=nullcontext(expected))
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.provider.get_app_config", return_value=app_config),
|
|
patch("deerflow.runtime.checkpointer.provider._sync_checkpointer_cm", factory),
|
|
checkpointer_context() as cp,
|
|
):
|
|
assert cp is expected
|
|
|
|
resolved = factory.call_args.args[0]
|
|
assert resolved.type == "memory"
|
|
assert resolved.connection_string is None
|
|
|
|
def test_explicit_memory_database_uses_in_memory_saver(self):
|
|
"""Explicit memory mode remains an intentional non-persistent checkpointer."""
|
|
from langgraph.checkpoint.memory import InMemorySaver
|
|
|
|
from deerflow.runtime.checkpointer.provider import checkpointer_context
|
|
|
|
app_config = SimpleNamespace(checkpointer=None, database=DatabaseConfig(backend="memory"))
|
|
|
|
with (
|
|
patch("deerflow.runtime.checkpointer.provider.get_app_config", return_value=app_config),
|
|
checkpointer_context() as cp,
|
|
):
|
|
assert isinstance(cp, InMemorySaver)
|
|
|
|
|
|
class TestStoreDatabaseConfig:
|
|
def test_sync_store_falls_back_to_memory_when_config_file_is_missing(self):
|
|
"""The sync Store keeps its no-config fallback for embedded callers."""
|
|
from langgraph.store.memory import InMemoryStore
|
|
|
|
with (
|
|
patch("deerflow.runtime.store.provider.ensure_config_loaded"),
|
|
patch("deerflow.runtime.store.provider.get_checkpointer_config", return_value=None),
|
|
patch("deerflow.runtime.store.provider.get_app_config", side_effect=FileNotFoundError),
|
|
):
|
|
assert isinstance(get_store(), InMemoryStore)
|
|
|
|
@pytest.mark.anyio
|
|
async def test_async_postgres_store_uses_database_config(self, caplog):
|
|
"""Unified database postgres config must not fall back to InMemoryStore."""
|
|
from deerflow.runtime.store.async_provider import make_store
|
|
|
|
caplog.set_level("WARNING", logger="deerflow.runtime.store.async_provider")
|
|
app_config = SimpleNamespace(
|
|
checkpointer=None,
|
|
database=DatabaseConfig(backend="postgres", postgres_url="postgresql://localhost/db"),
|
|
)
|
|
mock_store = AsyncMock()
|
|
mock_cm = AsyncMock()
|
|
mock_cm.__aenter__.return_value = mock_store
|
|
mock_cm.__aexit__.return_value = False
|
|
mock_store_cls = MagicMock()
|
|
mock_store_cls.from_conn_string.return_value = mock_cm
|
|
mock_module = MagicMock(AsyncPostgresStore=mock_store_cls)
|
|
|
|
with patch.dict(sys.modules, {"langgraph.store.postgres.aio": mock_module}):
|
|
async with make_store(app_config) as store:
|
|
assert store is mock_store
|
|
|
|
mock_store_cls.from_conn_string.assert_called_once_with("postgresql://localhost/db")
|
|
mock_store.setup.assert_awaited_once()
|
|
assert "No 'checkpointer' section" not in caplog.text
|
|
|
|
@pytest.mark.anyio
|
|
async def test_async_sqlite_store_uses_unified_database_path(self, tmp_path):
|
|
"""Unified database SQLite config must use the shared deerflow.db path."""
|
|
from deerflow.runtime.store.async_provider import make_store
|
|
from deerflow.runtime.store.provider import ensure_sqlite_parent_dir
|
|
|
|
db_config = DatabaseConfig(backend="sqlite", sqlite_dir=str(tmp_path))
|
|
app_config = SimpleNamespace(checkpointer=None, database=db_config)
|
|
mock_store = AsyncMock()
|
|
mock_cm = AsyncMock()
|
|
mock_cm.__aenter__.return_value = mock_store
|
|
mock_cm.__aexit__.return_value = False
|
|
mock_store_cls = MagicMock()
|
|
mock_store_cls.from_conn_string.return_value = mock_cm
|
|
mock_module = MagicMock(AsyncSqliteStore=mock_store_cls)
|
|
|
|
with (
|
|
patch.dict(sys.modules, {"langgraph.store.sqlite.aio": mock_module}),
|
|
patch(
|
|
"deerflow.runtime.store.async_provider.asyncio.to_thread",
|
|
new_callable=AsyncMock,
|
|
) as mock_to_thread,
|
|
):
|
|
async with make_store(app_config) as store:
|
|
assert store is mock_store
|
|
|
|
mock_to_thread.assert_awaited_once()
|
|
called_fn, called_path = mock_to_thread.await_args.args
|
|
assert called_fn is ensure_sqlite_parent_dir
|
|
assert called_path == db_config.checkpointer_sqlite_path
|
|
mock_store_cls.from_conn_string.assert_called_once_with(db_config.checkpointer_sqlite_path)
|
|
mock_store.setup.assert_awaited_once()
|
|
|
|
def test_sync_store_context_uses_database_config(self):
|
|
"""The one-shot sync Store factory must follow unified database config."""
|
|
from deerflow.runtime.store.provider import store_context
|
|
|
|
app_config = SimpleNamespace(
|
|
checkpointer=None,
|
|
database=DatabaseConfig(backend="postgres", postgres_url="postgresql://localhost/db"),
|
|
)
|
|
expected_store = object()
|
|
factory = MagicMock(return_value=nullcontext(expected_store))
|
|
|
|
with (
|
|
patch("deerflow.runtime.store.provider.get_app_config", return_value=app_config),
|
|
patch("deerflow.runtime.store.provider._sync_store_cm", factory),
|
|
store_context() as store,
|
|
):
|
|
assert store is expected_store
|
|
|
|
resolved = factory.call_args.args[0]
|
|
assert resolved.type == "postgres"
|
|
assert resolved.connection_string == "postgresql://localhost/db"
|
|
|
|
def test_sync_store_singleton_uses_database_config(self):
|
|
"""The cached sync Store factory must resolve database config before locking."""
|
|
app_config = SimpleNamespace(
|
|
checkpointer=None,
|
|
database=DatabaseConfig(backend="postgres", postgres_url="postgresql://localhost/db"),
|
|
)
|
|
expected_store = object()
|
|
factory = MagicMock(return_value=nullcontext(expected_store))
|
|
|
|
with (
|
|
patch("deerflow.runtime.store.provider.ensure_config_loaded"),
|
|
patch("deerflow.runtime.store.provider.get_app_config", return_value=app_config),
|
|
patch("deerflow.runtime.store.provider._sync_store_cm", factory),
|
|
):
|
|
assert get_store() is expected_store
|
|
|
|
resolved = factory.call_args.args[0]
|
|
assert resolved.type == "postgres"
|
|
assert resolved.connection_string == "postgresql://localhost/db"
|
|
|
|
def test_legacy_checkpointer_config_takes_precedence_for_store(self):
|
|
"""Backward-compatible checkpointer config must override database for Store."""
|
|
from deerflow.runtime.store.provider import store_context
|
|
|
|
app_config = SimpleNamespace(
|
|
checkpointer=CheckpointerConfig(type="memory"),
|
|
database=DatabaseConfig(backend="postgres", postgres_url="postgresql://localhost/db"),
|
|
)
|
|
expected_store = object()
|
|
factory = MagicMock(return_value=nullcontext(expected_store))
|
|
|
|
with (
|
|
patch("deerflow.runtime.store.provider.get_app_config", return_value=app_config),
|
|
patch("deerflow.runtime.store.provider._sync_store_cm", factory),
|
|
store_context() as store,
|
|
):
|
|
assert store is expected_store
|
|
|
|
resolved = factory.call_args.args[0]
|
|
assert resolved.type == "memory"
|
|
assert resolved.connection_string is None
|
|
|
|
def test_explicit_memory_database_uses_in_memory_store(self):
|
|
"""Explicit memory mode remains an intentional non-persistent Store."""
|
|
from langgraph.store.memory import InMemoryStore
|
|
|
|
from deerflow.runtime.store.provider import store_context
|
|
|
|
app_config = SimpleNamespace(
|
|
checkpointer=None,
|
|
database=DatabaseConfig(backend="memory"),
|
|
)
|
|
with patch("deerflow.runtime.store.provider.get_app_config", return_value=app_config):
|
|
with store_context() as store:
|
|
assert isinstance(store, InMemoryStore)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# app_config.py integration
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestAppConfigLoadsCheckpointer:
|
|
def test_load_checkpointer_section(self):
|
|
"""load_checkpointer_config_from_dict populates the global config."""
|
|
set_checkpointer_config(None)
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
cfg = get_checkpointer_config()
|
|
assert cfg is not None
|
|
assert cfg.type == "memory"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# DeerFlowClient falls back to config checkpointer
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestClientCheckpointerFallback:
|
|
def test_client_uses_config_checkpointer_when_none_provided(self):
|
|
"""DeerFlowClient._ensure_agent falls back to get_checkpointer() when checkpointer=None."""
|
|
from langgraph.checkpoint.memory import InMemorySaver
|
|
|
|
from deerflow.client import DeerFlowClient
|
|
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
|
|
captured_kwargs = {}
|
|
|
|
def fake_create_agent(**kwargs):
|
|
captured_kwargs.update(kwargs)
|
|
return MagicMock()
|
|
|
|
model_mock = MagicMock()
|
|
config_mock = MagicMock()
|
|
config_mock.models = [model_mock]
|
|
config_mock.get_model_config.return_value = MagicMock(supports_vision=False)
|
|
config_mock.checkpointer = None
|
|
|
|
config_mock.skills.deferred_discovery = False
|
|
config_mock.skills.container_path = "/mnt/skills"
|
|
config_mock.tool_search.enabled = False
|
|
|
|
with (
|
|
patch("deerflow.client.get_app_config", return_value=config_mock),
|
|
patch("deerflow.client.create_agent", side_effect=fake_create_agent),
|
|
patch("deerflow.client.create_chat_model", return_value=MagicMock()),
|
|
patch("deerflow.client.build_middlewares", return_value=[]),
|
|
patch("deerflow.client.apply_prompt_template", return_value=""),
|
|
patch("deerflow.client.get_enabled_skills_for_config", return_value=[]),
|
|
patch("deerflow.client.DeerFlowClient._get_tools", return_value=[]),
|
|
):
|
|
client = DeerFlowClient(checkpointer=None)
|
|
config = client._get_runnable_config("test-thread")
|
|
client._ensure_agent(config)
|
|
|
|
assert "checkpointer" in captured_kwargs
|
|
assert isinstance(captured_kwargs["checkpointer"], InMemorySaver)
|
|
|
|
def test_client_explicit_checkpointer_takes_precedence(self):
|
|
"""An explicitly provided checkpointer is used even when config checkpointer is set."""
|
|
from deerflow.client import DeerFlowClient
|
|
|
|
load_checkpointer_config_from_dict({"type": "memory"})
|
|
|
|
explicit_cp = MagicMock()
|
|
captured_kwargs = {}
|
|
|
|
def fake_create_agent(**kwargs):
|
|
captured_kwargs.update(kwargs)
|
|
return MagicMock()
|
|
|
|
model_mock = MagicMock()
|
|
config_mock = MagicMock()
|
|
config_mock.models = [model_mock]
|
|
config_mock.get_model_config.return_value = MagicMock(supports_vision=False)
|
|
config_mock.checkpointer = None
|
|
|
|
config_mock.skills.deferred_discovery = False
|
|
config_mock.skills.container_path = "/mnt/skills"
|
|
config_mock.tool_search.enabled = False
|
|
|
|
with (
|
|
patch("deerflow.client.get_app_config", return_value=config_mock),
|
|
patch("deerflow.client.create_agent", side_effect=fake_create_agent),
|
|
patch("deerflow.client.create_chat_model", return_value=MagicMock()),
|
|
patch("deerflow.client.build_middlewares", return_value=[]),
|
|
patch("deerflow.client.apply_prompt_template", return_value=""),
|
|
patch("deerflow.client.get_enabled_skills_for_config", return_value=[]),
|
|
patch("deerflow.client.DeerFlowClient._get_tools", return_value=[]),
|
|
):
|
|
client = DeerFlowClient(checkpointer=explicit_cp)
|
|
config = client._get_runnable_config("test-thread")
|
|
client._ensure_agent(config)
|
|
|
|
assert captured_kwargs["checkpointer"] is explicit_cp
|