mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-17 02:09:15 +00:00
* feat(checkpoint): dual-mode checkpoint storage with LangGraph DeltaChannel
Add a restart-required database.checkpoint_channel_mode ("full" default,
"delta") that stores the messages channel via LangGraph 1.2 DeltaChannel,
cutting checkpoint storage from O(n^2) to O(n) for append-only history.
Existing full checkpoints seed delta state transparently; no data migration.
- config: mode schema + freeze-on-first-use with
CheckpointModeReconfigurationError; mode marker persisted in checkpoint
metadata; unsafe delta->full downgrade rejected fail-closed with
CheckpointModeMismatchError (run-level error, failed state read)
- state: delta message state schema; CheckpointStateAccessor centralizes
materialized reads for all consumers (threads API, branches,
regeneration, compaction, state updates, memory, goal workers)
- runtime: raw writers (run durations, interrupted title, thread goal)
parent their checkpoints to the checkpoint they derive from, preserving
delta ancestry; rollback forks the pre-run lineage through a state
mutation graph with Overwrite restores; InMemorySaver delta-history
override delegates to the base walk (fixes dropped first write after
migration, also present upstream)
- tests: conformance suite over {memory, sqlite, postgres} covering
migration replay, stable message IDs, storage shape and writer
preservation; conftest fixture isolates the frozen mode between tests;
stale config fakes refreshed
- ci: backend unit tests gain a postgres service
* fix(checkpoint): close materialization gaps in goal flow, guard public factory
- Route goal-continuation message reads through CheckpointStateAccessor:
raw channel_values reads see the delta sentinel in delta mode, which
disabled goal continuation (stand_down=no_durable_end_of_turn) after
durable assistant turns. Raw tuples remain for tuple-only metadata
(checkpoint id, pending_writes).
- Reject checkpoint_channel_mode='delta' + checkpointer in
create_deerflow_agent at construction: factory-built persisted graphs
bypass mode-marker injection and the fail-closed gate, reproducing
silent mixed-mode state loss. Delta without persistence stays allowed.
- Import the postgres saver lazily (pytest.importorskip in the fixture)
so the documented default install collects the suite; add a CI job
running pytest --collect-only on uv sync --group dev without extras.
- Fix test_checkpointer fallback test to patch get_app_config at its
use site (provider module), making it deterministic when a local
config.yaml selects a persistent backend.
* fix(gateway): preserve extension-owned channels in state mutations, bump config version
- build_state_mutation_graph / build_checkpoint_state_mutation_accessor
accept an explicit state_schema; branch and POST /state now compile the
mutation graph from the thread's effective schema
(graph_state_schema on the assistant graph). The base-ThreadState
fallback silently discarded channels contributed by custom
AgentMiddleware.state_schema on branch (data loss) and returned a
false-success 200 on POST /state.
- POST /state validates values keys against the mutation graph's
channels and rejects unknown fields with 422 instead of ignoring
them; reducer detection covers extension channels
(BinaryOperatorAggregate or DeltaChannel) so Overwrite replace
semantics work for middleware reducers in both modes.
- Endpoint regression: custom AgentMiddleware.state_schema value
survives branch, updates through POST /state, and an unknown field
receives 422.
- config_version 26 -> 27 for the new database.checkpoint_channel_mode
(example, Helm chart values + README, support-bundle fixture), so
existing installs get the outdated-config warning and
make config-upgrade merges the field; covered by a test driving the
real example file and the real config-upgrade script.
* fix(gateway): resolve assistant schema via one boundary, copy branch reducer values with Overwrite
GET /threads/{id}/state now resolves the thread's assistant_id through a
single reusable boundary (thread metadata -> assistant_id -> effective
graph), so channels contributed by a custom AgentMiddleware.state_schema
are materialized instead of dropped by the default lead schema. POST
/state uses the same boundary instead of resolving the schema ad hoc.
Branch writes wrap every copied reducer channel in Overwrite (derived
from the effective mutation graph: BinaryOperatorAggregate + DeltaChannel),
not just messages, so already-aggregated values are never re-merged.
Regression tests use a real AgentMiddleware.state_schema with a
non-identity reducer in both full and delta modes: GET /state returns the
extension value, POST /state replaces it, branch preserves it
byte-for-byte; the unknown-field 422 is a separate assertion.
* refactor(checkpoint): collapse read-path round-trips and ship dual-mode parity tests
Address review round 4 on PR #4292:
- Push ahistory/history limit through Pregel into checkpointer.alist
(SQL LIMIT) instead of materializing all rows and breaking in Python
- Fold the read-side mode-compat gate onto the returned snapshot's
metadata; only writes keep the pre-write tuple fetch (fail-closed)
- Cache factory-built accessor graphs per (assistant_id, mode) with
factory-identity revalidation; state reads no longer build a lead
agent per request
- get_thread: one snapshot fetch + one raw pending_writes fetch on the
resolved checkpoint (post-checkpoint __error__ writes never surface
in snapshot.tasks; verified empirically)
- DeerFlowClient.get_thread: single checkpointer.list walk collects
pending_writes per checkpoint instead of N get_tuple calls
- InMemorySaver delta-history patch: stand-down when the upstream
override disappears, try/except guard, validated-version warning,
guard tests
- make_lead_agent mode precedence: first freeze is owned by app_config
(client-supplied configurable key ignored); once frozen, injected
key/app_config must match or fail closed
- Rollback: lock in non-message channel restoration via fork
inheritance with a dedicated reducer-channel test
- Add tests/test_threads_checkpoint_mode.py and
tests/test_gateway_checkpoint_mode.py referenced by AGENTS.md and
the PR validation section: lifecycle parity (memory + sqlite),
per-step blob-count storage guard, gateway endpoint parity
Counted-saver tests pin checkpoint round-trips for aget/ahistory so
these regressions cannot silently return.
* fix(checkpoint): precise mode-mismatch HTTP mapping, gate E2E, and accessor resilience
- threads router: map CheckpointModeMismatchError to 409 (with cause and
thread id) and CheckpointModeReconfigurationError to 503 across all state
endpoints instead of swallowing both into a generic 500
- gate coverage: seed a real delta checkpoint into AsyncSqliteSaver and
assert aget/aupdate/ahistory fail closed; assert 409 at the HTTP boundary
through the real route stack
- rollback: compile the restore mutation graph with the thread's effective
state schema per the build_state_mutation_graph contract
- inheritance contract locks: rollback and manual compaction preserve
middleware-contributed channels via checkpoint fork cloning
- services: revalidate the accessor-graph cache against app_config identity
so config.yaml hot-reloads never serve a stale compiled graph
- services: degrade full-mode state reads to raw checkpointer reads when the
agent factory is unavailable (delta gate still applies; delta mode has no
fallback)
- deps: override websockets==16.0 (langgraph-sdk 0.4.2's <16 pin silently
downgraded 16.0 -> 15.0.1; pin is not grounded in any API incompatibility)
and bump the langchain lower bound to what the lockfile actually resolves
* fix(checkpoint): include anchor checkpoint in degraded history walk + cover get_thread
- _RawCheckpointReadAccessor.ahistory: alist(before=...) is exclusive while
pregel's get_state_history treats config.checkpoint_id as the inclusive
start; fetch the anchor explicitly so both read paths paginate identically
- extend the degraded-path gateway test: GET /thread returns raw values, and
POST /history with before starts at the anchor checkpoint
* fix(gateway): preserve degraded checkpoint timestamps
* fix(gateway): harden degraded checkpoint access
* fix(gateway): resolve assistants for checkpoint reads
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
146 lines
5.3 KiB
Python
146 lines
5.3 KiB
Python
"""Stateless runs endpoints -- stream and wait without a pre-existing thread.
|
|
|
|
These endpoints auto-create a temporary thread when no ``thread_id`` is
|
|
supplied in the request body. When a ``thread_id`` **is** provided, it
|
|
is reused so that conversation history is preserved across calls.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import uuid
|
|
|
|
from fastapi import APIRouter, HTTPException, Query, Request
|
|
from fastapi.responses import StreamingResponse
|
|
|
|
from app.gateway.authz import require_permission
|
|
from app.gateway.deps import get_feedback_repo, get_run_event_store, get_run_manager, get_run_store, get_stream_bridge
|
|
from app.gateway.pagination import trim_run_message_page
|
|
from app.gateway.routers.thread_runs import RunCreateRequest
|
|
from app.gateway.services import build_checkpoint_state_accessor, sse_consumer, start_run, wait_for_run_completion
|
|
from deerflow.runtime import serialize_channel_values_for_api
|
|
|
|
logger = logging.getLogger(__name__)
|
|
router = APIRouter(prefix="/api/runs", tags=["runs"])
|
|
|
|
|
|
def _resolve_thread_id(body: RunCreateRequest) -> str:
|
|
"""Return the thread_id from the request body, or generate a new one."""
|
|
thread_id = ((body.config or {}).get("configurable") or {}).get("thread_id")
|
|
if thread_id:
|
|
return str(thread_id)
|
|
return str(uuid.uuid4())
|
|
|
|
|
|
@router.post("/stream")
|
|
async def stateless_stream(body: RunCreateRequest, request: Request) -> StreamingResponse:
|
|
"""Create a run and stream events via SSE.
|
|
|
|
If ``config.configurable.thread_id`` is provided, the run is created
|
|
on the given thread so that conversation history is preserved.
|
|
Otherwise a new temporary thread is created.
|
|
"""
|
|
thread_id = _resolve_thread_id(body)
|
|
bridge = get_stream_bridge(request)
|
|
run_mgr = get_run_manager(request)
|
|
record = await start_run(body, thread_id, request)
|
|
|
|
return StreamingResponse(
|
|
sse_consumer(bridge, record, request, run_mgr),
|
|
media_type="text/event-stream",
|
|
headers={
|
|
"Cache-Control": "no-cache",
|
|
"Connection": "keep-alive",
|
|
"X-Accel-Buffering": "no",
|
|
"Content-Location": f"/api/threads/{thread_id}/runs/{record.run_id}",
|
|
},
|
|
)
|
|
|
|
|
|
@router.post("/wait", response_model=dict)
|
|
async def stateless_wait(body: RunCreateRequest, request: Request) -> dict:
|
|
"""Create a run and block until completion.
|
|
|
|
If ``config.configurable.thread_id`` is provided, the run is created
|
|
on the given thread so that conversation history is preserved.
|
|
Otherwise a new temporary thread is created.
|
|
"""
|
|
thread_id = _resolve_thread_id(body)
|
|
bridge = get_stream_bridge(request)
|
|
run_mgr = get_run_manager(request)
|
|
record = await start_run(body, thread_id, request)
|
|
|
|
completed = True
|
|
if record.task is not None:
|
|
completed = await wait_for_run_completion(bridge, record, request, run_mgr)
|
|
|
|
if completed:
|
|
try:
|
|
accessor, config = build_checkpoint_state_accessor(
|
|
request,
|
|
thread_id=thread_id,
|
|
assistant_id=body.assistant_id,
|
|
)
|
|
snapshot = await accessor.aget(config)
|
|
snapshot_config = snapshot.config or {}
|
|
if snapshot_config.get("configurable", {}).get("checkpoint_id"):
|
|
return serialize_channel_values_for_api(snapshot.values)
|
|
except Exception:
|
|
logger.exception("Failed to fetch final state for run %s", record.run_id)
|
|
|
|
return {"status": record.status.value, "error": record.error}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Run-scoped read endpoints
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
async def _resolve_run(run_id: str, request: Request) -> dict:
|
|
"""Fetch run by run_id with user ownership check. Raises 404 if not found."""
|
|
run_store = get_run_store(request)
|
|
record = await run_store.get(run_id) # user_id=AUTO filters by contextvar
|
|
if record is None:
|
|
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
|
|
return record
|
|
|
|
|
|
@router.get("/{run_id}/messages")
|
|
@require_permission("runs", "read")
|
|
async def run_messages(
|
|
run_id: str,
|
|
request: Request,
|
|
limit: int = Query(default=50, le=200, ge=1),
|
|
before_seq: int | None = Query(default=None, ge=1),
|
|
after_seq: int | None = Query(default=None, ge=1),
|
|
) -> dict:
|
|
"""Return paginated messages for a run (cursor-based).
|
|
|
|
Pagination:
|
|
- after_seq: messages with seq > after_seq (forward)
|
|
- before_seq: messages with seq < before_seq (backward)
|
|
- neither: latest messages
|
|
|
|
Response: { data: [...], has_more: bool }
|
|
"""
|
|
run = await _resolve_run(run_id, request)
|
|
event_store = get_run_event_store(request)
|
|
rows = await event_store.list_messages_by_run(
|
|
run["thread_id"],
|
|
run_id,
|
|
limit=limit + 1,
|
|
before_seq=before_seq,
|
|
after_seq=after_seq,
|
|
)
|
|
data, has_more = trim_run_message_page(rows, limit=limit, after_seq=after_seq)
|
|
return {"data": data, "has_more": has_more}
|
|
|
|
|
|
@router.get("/{run_id}/feedback")
|
|
@require_permission("runs", "read")
|
|
async def run_feedback(run_id: str, request: Request) -> list[dict]:
|
|
"""Return all feedback for a run."""
|
|
run = await _resolve_run(run_id, request)
|
|
feedback_repo = get_feedback_repo(request)
|
|
return await feedback_repo.list_by_run(run["thread_id"], run_id)
|