deer-flow/backend/tests/test_thread_regenerate_prepare.py
Totoro 0fe6e63a68
feat(goal): show why a chat goal stopped and when it was met, and redact the goal evaluator and continuation input (#6556)
* feat(goal): record the met goal and keep goal state on the history head

- Add a goal_outcome LastValue channel (GoalOutcomeState). Only the
  guarded clear of a satisfied ordinary goal writes it, in the same
  checkpoint as the clear; the reply anchor is the latest visible AI
  message with text, the reply the visible signature keys on. A goal
  stored without created_at still clears and gets a record.
- write_thread_goal removes the record on every other goal write and
  lists it in channel_versions, new_versions and metadata writes when
  it changes. Threads that never had a record keep their checkpoint
  shape. The sync-only SqliteSaver and CachedHistorySaver, which take
  the #6448 sync fallback, store and drop it the same way.
- goal_outcome is server-owned: POST /state and non-internal run input
  that carry it get 400. A POST /state goal write, or a goal in
  external run input, clears it.
- A branch head written onto its replay base resets the channels the
  head lacks, so a goal or record that write_thread_goal removed does
  not come back. Branching from the met turn keeps the record without
  the met goal.
- The POST /history head now carries an active goal and an achieved
  goal_outcome, also before any message exists, so the post-run
  history refetch keeps them. Older entries stay title/thread_data.
- contracts/thread_goal_contract.json pins the chat stand-down and
  check-failure codes, the record shape and status, the head keys and
  the host-written reasons.

* feat(frontend): show why a goal stopped and when it was met

- The goal bar derives its state from the active goal, the goal_outcome
  record and the messages: set, in progress, continuing, stopped (limit,
  auto-continue off, no progress, token budget, missing evidence, failed
  run, unknown code), waiting for you, waiting, couldn't check the goal,
  paused and goal met. An 8/8 stand-down now reads "Stopped" with the
  next step instead of "Continuing 8/8", and a count-0 stand-down no
  longer looks like a fresh goal.
- "Goal met" shows while its anchor reply is the latest visible message
  (a visible human message, or a visible AI message with text, moves the
  chat on), so a reload agrees with the live view. An active goal wins
  over the record, and this tab's /goal set or clear hides it.
- useActiveGoal drops the optimistic goal when the record of that same
  goal arrives, which fixes the bar that stayed after a goal met on the
  first run (the clear frame has no goal key), also for a goal stored
  without created_at. Server goal values pass through isGoalState; a
  /goal status read uses the backend's own active check, so the bar, the
  toast and the edit lock agree with the server.
- Row 2 gives the reason and what to do next; Details holds the full
  goal, the checker's own note (never host-written text, a check failure
  or a paused goal's older check), the assumption note and an unknown
  code, and starts collapsed in each new state. The bar is a labelled
  section with a live region that announces state changes only (another
  goal's same state counts) and re-reads its text after a language switch.
- The edit pencil on the latest turn is shown disabled with "Editing is
  off while a goal is set" instead of hidden; the message toolbar shows
  on focus-within, so keyboard users can reach it.
- A /goal set or clear rejected with 409 keeps the draft and says a run
  is still going; other failures keep the server's message and the draft.
  A /goal request replaced by a newer one leaves the composer alone, so
  a double submit no longer empties it before the second one fails.
- Copy lives in inputBox.goalBar (en-US, zh-CN); a contract test reads
  contracts/thread_goal_contract.json, and the raw-identifier guard now
  catches stand-down codes and goal field names.

* test(frontend): cover the goal bar on both chat pages

- tests/e2e/goal-status.spec.ts runs every case on the chat page and the
  agent chat page, since each wires the goal bar and the edit lock itself:
  a goal stopped at 8/8, one waiting for details and one waiting on an
  open question card show their reason and next step, also after a
  reload; the locked edit pencil gets keyboard focus, shows its toolbar
  and tooltip, does not open the editor, and hides while a file uploads;
  Goal met after one automatic follow-up (In progress, Continuing 1/8,
  then "Auto-continued once") survives a reload and hides after the next
  visible message; a goal met on the first run of a new chat replaces the
  set bar and frees the pencil, and a new /goal replaces Goal met while
  its own run goes on; a /goal set or clear refused with 409 keeps the
  draft for the retry and shows its own busy toast.
- The mock POST /history head carries only the contract's
  history_head_keys, with goal only while active and goal_outcome only
  when achieved; thread state and run values omit an unset goal instead
  of sending goal: null. PUT and DELETE /goal drop the met record,
  setGoalWritesBusy returns the Gateway's 409, and a repeat create of a
  requested thread id returns the thread as is, like the Gateway.
- The raw-identifier guard also catches checker blockers and the met
  record's field names.
- frontend/src/AGENTS.md: the GoalStatus sentence now names goal,
  goal_outcome and useActiveGoal, reworded in place.

* test(frontend): keep the draft through a failed upload and a busy /goal

#6412 fires onSent only once a send is dispatched, so a failed upload
keeps the text, files and stored draft. A /goal the Gateway refuses with
409 never reaches sendMessage, and the composer rejects it so PromptInput
keeps the text. Drive both through the real sendMessage in one composer:
a failed upload keeps "summarize it" stored, a busy /goal keeps its text
and stored draft with no run, and the retry saves the goal and starts
one run with the objective, not the "/goal" text. After that run the
composer is empty and no stored draft comes back.

* fix(frontend): drop the stop condition's closing period before the pause clause

A stored stop condition often ends with a period, so the schedule card
read "…remain)., pauses itself" (and "…。,满足后自动暂停" in Chinese).
Trim the condition's closing sentence punctuation before the clause that
follows it. The event line, where the condition ends the sentence, is
unchanged.

* fix(frontend): point "evidence missing" at the agent, not a user reply

"Evidence missing" asked the user to reply with what's left, which is
the push a goal is meant to save. It now asks the agent to show the
result or explain what's missing, and offers /goal clear only for a goal that
no longer applies: clearing drops the met record, so work that is right
would never show "Goal met".

zh-CN follows. Copy only: no key, state or mapping changes, and the
scheduled page's reason labels are untouched.

tests/unit/components/workspace/goal-status-next-steps.test.ts checks
the row in en-US and zh-CN, and that no stop row offers /goal clear
without that condition. goal-status.dom.test.tsx and
goal-stream.dom.test.tsx now expect the new copy.

* fix(frontend): say "Couldn't check the goal" once in the goal bar

After a goal check that could not run, the bar's second line borrowed
the scheduled page's whole reason, so it repeated the chip: "Couldn't
check the goal · Couldn't check the goal; the run itself may be fine.
It's checked again after your next message." Chinese had the same
repeat.

The line now gives only why the check failed ("The run itself may be
fine", "No final reply was saved" or "The chat changed during the
check"), then when it is checked again. The screen-reader announcement
follows. The chip and the scheduled page's reason labels are unchanged.

tests/unit/components/workspace/goal-status-unchecked.test.ts checks
every check-failure code in en-US and zh-CN: the label appears once,
and the line still says why the check failed.

* feat(goal): redact the evaluator input when pii_redaction is enabled

The goal evaluator calls its model directly, outside
PiiRedactionMiddleware, so it sent raw user lines, tool arguments, tool
results and Human Input Card answers even with pii_redaction enabled.

With pii_redaction enabled:
- the messages in the evidence window are redacted whole before the
  evidence caps can cut an identifier in two, as TitleMiddleware does.
  This reuses redact_queued_messages, which already covers content,
  call arguments and card answers. Earlier messages are left alone: the
  evaluator never sees them, and the pass is synchronous work on the
  event loop (0.84 s for an 800-message, 4 MB thread, 0.06 s for its
  window);
- the assembled input, the goal objective included, is redacted again.

A redaction error fails the check with evaluator_failed; raw text is
never sent instead. When redaction is off, the input is byte-identical.
Thread state keeps the raw text. README.md, config.example.yaml and the
middleware docstring list the evaluator with the other redacted inputs.

tests/test_goal_pii.py puts an address where the evidence caps cut a
tool argument, a tool result and a card answer, and checks the
objective, the unchanged input when redaction is off, that only the
evidence window reaches the redactor and gives the same evidence as
redacting the whole thread, that a redaction error stands down with
evaluator_failed before the model is called, and the worker path.

* fix(goal): redact the hidden continuation when pii_redaction is enabled

The hidden /goal continuation repeats the goal objective and the
evaluator's reason and evidence summary. PiiRedactionMiddleware skips
it, since it is a framework message and not a user message, so with
pii_redaction enabled each continuation turn sent them raw to the
agent's model, and the message stayed in the thread for later model
calls. The durable-context block and the evaluator input already redact
the objective.

With pii_redaction enabled, make_goal_continuation_message now redacts
the objective, the reason and the evidence summary with redact_text
before the message is built. The thread keeps the redacted message,
which the UI hides; the goal state keeps the raw objective.

The worker builds the message before it counts the continuation. A
redaction error fails the check the way an evaluator redaction error
does: run_failed with stand_down_reason evaluator_failed, the goal stays
active, nothing is sent and the continuation budget is not spent. Only
the exception type is stored. With redaction off, the message is
byte-identical.

README.md, config.example.yaml and the middleware docstring list the
continuation with the other redacted inputs.

tests/test_goal_continuation_pii.py checks the redacted objective,
reason and evidence summary against the tokens redact_text gives, the
unchanged message with redaction off or no config, the worker path
under the run config, and a redaction error.

* docs(changelog): link the goal outcome entries to #6556

Fill the four ([#PR]) placeholders in CHANGELOG.md and CHANGELOG_zh.md with this PR's number and add the matching link references.

* fix(frontend): keep an active goal stored without timestamps visible

POST /threads/{id}/state and run input can store an active goal without
created_at or updated_at. The backend still evaluates it and rejects an
edit with 409 (is_active_goal checks only status "active"), but
isGoalState required both timestamps, so history and live values
dropped it: both chat pages hid the goal bar and unlocked the last-turn
pencil, and only a /goal status read, which used a looser predicate,
brought them back.

isGoalState now accepts any record with a string objective and status
"active", and normalizeGoalStatusRead uses it, so thread values, live
updates and status reads share one rule. Malformed or inactive values
are still dropped.

Tests cover a sparse active goal from the history head and from a live
update before the post-run refetch (bar shown, hasGoal true), and a
malformed goal that stays hidden.

* test(goal): check the outcome write metadata the same way on every saver

PostgresSaver stores checkpoint metadata through
get_serializable_checkpoint_metadata, which drops "writes", so the
[postgres] case of test_goal_outcome_round_trips_through_the_sync_fallback
failed with KeyError: 'writes' when CI ran it against a live database.

The test now records the metadata write_thread_goal passes to put and
asserts the clear's and the next goal's "writes" there, for every saver.
The stored channel values are checked as before, and on sqlite and
cached-sqlite the stored "writes" must still equal what was passed.

* docs(agents): keep goal_outcome out of the agents guidance chain

The middlewares AGENTS.md chain is already over the 98,304-byte AG002
hard limit on main, and the PR-mode check fails any chain a PR touches,
so adding goal_outcome to the ThreadState field list in
deerflow/agents/AGENTS.md failed the agent-guidance job. goal_outcome
stays documented in runtime/AGENTS.md and backend/app/gateway/AGENTS.md,
which are outside that chain.

* refactor(goal): read the evaluator's evidence and redaction from one window

format_visible_conversation recomputed the window start that the redaction
pass takes from _evidence_window. Both now use _evidence_window, so a later
change to the cap or to what counts as visible moves both.

The window test now fills the window with tool-only replies and tool results.
Before, it caught a cap change in one copy but not a change to what counts
as visible.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-10 00:20:45 +08:00

1740 lines
62 KiB
Python

from __future__ import annotations
import asyncio
import copy
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from _router_auth_helpers import call_unwrapped
from fastapi import HTTPException
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage
from langgraph.checkpoint.base import empty_checkpoint, uuid6
from langgraph.checkpoint.memory import InMemorySaver
from deerflow.runtime import DisconnectMode, RunRecord, RunStatus
from deerflow.utils.messages import ORIGINAL_USER_CONTENT_KEY
def _checkpoint(
checkpoint_id: str,
messages: list[object],
*,
metadata: dict | None = None,
goal: dict | None = None,
next_tasks: tuple[str, ...] = (),
):
channel_values = {"messages": messages}
if goal is not None:
channel_values["goal"] = goal
return SimpleNamespace(
config={
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
"checkpoint_id": checkpoint_id,
"checkpoint_map": None,
}
},
checkpoint={"channel_values": channel_values},
metadata=metadata or {},
next=next_tasks,
)
async def _put_memory_checkpoint(
checkpointer: InMemorySaver,
thread_id: str,
messages: list[object],
*,
step: int,
parent_config: dict | None = None,
metadata: dict | None = None,
) -> dict:
checkpoint = empty_checkpoint()
checkpoint["id"] = str(uuid6())
checkpoint["channel_values"] = {"messages": messages}
checkpoint["channel_versions"] = {"messages": step}
checkpoint_metadata = {
"step": step,
"source": "loop",
"writes": {"test": {"messages": messages}},
"parents": {},
}
checkpoint_metadata.update(metadata or {})
return await checkpointer.aput(
parent_config or {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}},
checkpoint,
checkpoint_metadata,
{"messages": step},
)
async def _collect_checkpoints(checkpointer: InMemorySaver, config: dict) -> list:
return [checkpoint async for checkpoint in checkpointer.alist(config)]
class FakeCheckpointer:
def __init__(self, history, *, latest=None, materialized_history=None, materialized_latest=None):
self.history = history
self.latest = latest
self.materialized_history = materialized_history
self.materialized_latest = materialized_latest
self.alist_limits = []
async def aget_tuple(self, config):
checkpoint_id = config.get("configurable", {}).get("checkpoint_id")
if checkpoint_id:
return next((item for item in self.history if item.config["configurable"]["checkpoint_id"] == checkpoint_id), None)
return self.latest or (self.history[0] if self.history else None)
async def alist(self, config, limit=200):
self.alist_limits.append(limit)
for item in self.history[:limit]:
yield item
def _snapshot(checkpoint_id: str, messages: list[object], *, metadata: dict | None = None):
return SimpleNamespace(
values={"messages": messages},
config={
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
"checkpoint_id": checkpoint_id,
"checkpoint_map": None,
}
},
metadata=metadata or {},
)
def _human_input_response(
value: str,
*,
request_id: str,
source: str = "ask_clarification",
) -> dict:
return {
"version": 1,
"kind": "human_input_response",
"source": source,
"request_id": request_id,
"response_kind": "text",
"value": value,
}
class FakeAccessor:
def __init__(self, checkpointer: FakeCheckpointer):
self.checkpointer = checkpointer
@staticmethod
def _from_raw(checkpoint):
return SimpleNamespace(
values=dict(checkpoint.checkpoint.get("channel_values", {})),
config=checkpoint.config,
metadata=checkpoint.metadata,
parent_config=getattr(checkpoint, "parent_config", None),
next=getattr(checkpoint, "next", ()),
)
async def aget(self, config):
materialized_latest = getattr(self.checkpointer, "materialized_latest", None)
if materialized_latest is not None and not config.get("configurable", {}).get("checkpoint_id"):
return materialized_latest
raw = await self.checkpointer.aget_tuple(config)
return self._from_raw(raw) if raw is not None else SimpleNamespace(values={}, config={}, metadata={})
async def ahistory(self, config, *, limit=None):
alist_limits = getattr(self.checkpointer, "alist_limits", None)
if alist_limits is not None:
alist_limits.append(limit)
history = getattr(self.checkpointer, "materialized_history", None)
if history is None:
if hasattr(self.checkpointer, "history"):
history = [self._from_raw(item) for item in self.checkpointer.history]
else:
history = [self._from_raw(item) async for item in self.checkpointer.alist(config, limit=limit)]
return history[:limit]
@pytest.fixture(autouse=True)
def _patch_checkpoint_accessor(monkeypatch):
from app.gateway import services
from app.gateway.routers import thread_runs
def build_accessor(request, *, thread_id, assistant_id=None, checkpoint_id=None):
config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
if checkpoint_id is not None:
config["configurable"]["checkpoint_id"] = checkpoint_id
return FakeAccessor(request.app.state.checkpointer), config
async def build_thread_accessor(request, *, thread_id, checkpoint_id=None):
return build_accessor(request, thread_id=thread_id, checkpoint_id=checkpoint_id)
monkeypatch.setattr(services, "build_checkpoint_state_accessor", build_accessor)
monkeypatch.setattr(thread_runs, "build_thread_checkpoint_state_accessor", build_thread_accessor)
class FakeEventStore:
def __init__(self, rows):
self.rows = rows
async def list_messages(self, thread_id, *, limit=50, before_seq=None, after_seq=None, user_id=None):
return self.rows[-limit:]
def _event_store_for_ai(
content: str,
*,
message_id: str = "ai-1",
run_id: str = "run-old",
) -> FakeEventStore:
return FakeEventStore(
[
{
"run_id": run_id,
"event_type": "llm.ai.response",
"category": "message",
"content": {
"id": message_id,
"type": "ai",
"content": content,
},
"metadata": {"caller": "lead_agent"},
}
]
)
class FakeRunManager:
def __init__(self, records):
self.records = records
async def list_by_thread(self, thread_id, *, user_id=None, limit=100):
return self.records[:limit]
async def get(self, run_id, *, user_id=None):
return next((record for record in self.records if record.run_id == run_id), None)
def _request(checkpointer, event_store, *, run_manager=None, user_id="user-1"):
from app.gateway.auth_disabled import AUTH_SOURCE_SESSION
return SimpleNamespace(
app=SimpleNamespace(
state=SimpleNamespace(
checkpointer=checkpointer,
run_event_store=event_store,
run_manager=run_manager or FakeRunManager([]),
)
),
state=SimpleNamespace(user=SimpleNamespace(id=user_id), auth_source=AUTH_SOURCE_SESSION),
)
def test_run_wait_readers_return_materialized_final_values() -> None:
from app.gateway import services
from app.gateway.routers import runs, thread_runs
snapshot = SimpleNamespace(
values={
"messages": [
HumanMessage(id="h1", content="question"),
AIMessage(id="a1", content="answer"),
]
},
config={
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
"checkpoint_id": "ckpt-2",
}
},
parent_config={"configurable": {"checkpoint_id": "ckpt-1"}},
metadata={"step": 2},
next=(),
tasks=(),
created_at=None,
)
accessor = SimpleNamespace(aget=AsyncMock(return_value=snapshot))
record = SimpleNamespace(
run_id="run-1",
thread_id="thread-1",
task=None,
status=RunStatus.success,
error=None,
)
request = SimpleNamespace()
body = thread_runs.RunCreateRequest(
assistant_id="lead-agent",
config={"configurable": {"thread_id": "thread-1"}},
)
async def _scenario() -> tuple[dict, dict]:
with (
patch.object(thread_runs, "get_stream_bridge", return_value=object()),
patch.object(thread_runs, "get_run_manager", return_value=object()),
patch.object(thread_runs, "start_run", AsyncMock(return_value=record)),
patch.object(
services,
"build_checkpoint_state_accessor",
create=True,
new=MagicMock(return_value=(accessor, snapshot.config)),
),
patch.object(runs, "get_stream_bridge", return_value=object()),
patch.object(runs, "get_run_manager", return_value=object()),
patch.object(runs, "start_run", AsyncMock(return_value=record)),
patch.object(
services,
"build_checkpoint_state_accessor",
create=True,
new=MagicMock(return_value=(accessor, snapshot.config)),
),
):
thread_result = await call_unwrapped(thread_runs.wait_run, "thread-1", body, request)
stateless_result = await call_unwrapped(runs.stateless_wait, body, request)
return thread_result, stateless_result
thread_result, stateless_result = asyncio.run(_scenario())
assert [message["id"] for message in thread_result["messages"]] == ["h1", "a1"]
assert [message["id"] for message in stateless_result["messages"]] == ["h1", "a1"]
def test_run_wait_readers_preserve_terminal_error_without_checkpoint() -> None:
from app.gateway import services
from app.gateway.routers import runs, thread_runs
snapshot = SimpleNamespace(
values={},
config={"configurable": {"thread_id": "thread-1", "checkpoint_ns": ""}},
parent_config=None,
metadata={},
next=(),
tasks=(),
created_at=None,
)
accessor = SimpleNamespace(aget=AsyncMock(return_value=snapshot))
record = SimpleNamespace(
run_id="run-1",
thread_id="thread-1",
task=None,
status=RunStatus.error,
error="run failed before checkpoint",
)
request = SimpleNamespace()
body = thread_runs.RunCreateRequest(config={"configurable": {"thread_id": "thread-1"}})
async def _scenario() -> tuple[dict, dict]:
with (
patch.object(thread_runs, "get_stream_bridge", return_value=object()),
patch.object(thread_runs, "get_run_manager", return_value=object()),
patch.object(thread_runs, "start_run", AsyncMock(return_value=record)),
patch.object(
services,
"build_checkpoint_state_accessor",
new=MagicMock(return_value=(accessor, snapshot.config)),
),
patch.object(runs, "get_stream_bridge", return_value=object()),
patch.object(runs, "get_run_manager", return_value=object()),
patch.object(runs, "start_run", AsyncMock(return_value=record)),
patch.object(
services,
"build_checkpoint_state_accessor",
new=MagicMock(return_value=(accessor, snapshot.config)),
),
):
thread_result = await call_unwrapped(thread_runs.wait_run, "thread-1", body, request)
stateless_result = await call_unwrapped(runs.stateless_wait, body, request)
return thread_result, stateless_result
thread_result, stateless_result = asyncio.run(_scenario())
expected = {
"status": "error",
"error": "run failed before checkpoint",
"__error__": {"error": "RunError", "message": "run failed before checkpoint"},
}
assert thread_result == expected
assert stateless_result == expected
@pytest.mark.parametrize("route_name", ["thread", "stateless"])
def test_run_wait_readers_preserve_terminal_error_when_accessor_builder_fails(route_name: str) -> None:
from app.gateway import services
from app.gateway.routers import runs, thread_runs
record = SimpleNamespace(
run_id="run-1",
thread_id="thread-1",
task=None,
status=RunStatus.error,
error="run failed before checkpoint",
)
request = SimpleNamespace()
body = thread_runs.RunCreateRequest(config={"configurable": {"thread_id": "thread-1"}})
async def _scenario() -> dict:
if route_name == "thread":
with (
patch.object(thread_runs, "get_stream_bridge", return_value=object()),
patch.object(thread_runs, "get_run_manager", return_value=object()),
patch.object(thread_runs, "start_run", AsyncMock(return_value=record)),
patch.object(
services,
"build_checkpoint_state_accessor",
new=MagicMock(side_effect=RuntimeError("graph construction failed")),
),
):
return await call_unwrapped(thread_runs.wait_run, "thread-1", body, request)
with (
patch.object(runs, "get_stream_bridge", return_value=object()),
patch.object(runs, "get_run_manager", return_value=object()),
patch.object(runs, "start_run", AsyncMock(return_value=record)),
patch.object(
services,
"build_checkpoint_state_accessor",
new=MagicMock(side_effect=RuntimeError("graph construction failed")),
),
):
return await call_unwrapped(runs.stateless_wait, body, request)
result = asyncio.run(_scenario())
expected = {
"status": "error",
"error": "run failed before checkpoint",
"__error__": {"error": "RunError", "message": "run failed before checkpoint"},
}
assert result == expected
def test_prepare_regenerate_payload_returns_clean_input_and_base_checkpoint():
from app.gateway.routers import thread_runs
human = HumanMessage(
id="human-1",
content="<current_uploads>injected</current_uploads>\n\n/data-analysis analyze data.csv",
additional_kwargs={
ORIGINAL_USER_CONTENT_KEY: "/data-analysis analyze data.csv",
"files": [{"filename": "data.csv", "path": "/mnt/user-data/uploads/data.csv"}],
},
)
ai = AIMessage(id="ai-1", content="answer v1")
base = _checkpoint("ckpt-base", [])
after_human = _checkpoint("ckpt-human", [human])
latest = _checkpoint("ckpt-ai", [human, ai])
checkpointer = FakeCheckpointer([latest, after_human, base])
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer v1"},
"metadata": {"caller": "lead_agent"},
}
]
)
original_builder = thread_runs.build_thread_checkpoint_state_accessor
thread_builder = AsyncMock(side_effect=original_builder)
with patch.object(thread_runs, "build_thread_checkpoint_state_accessor", thread_builder):
response = asyncio.run(thread_runs._prepare_regenerate_payload("thread-1", "ai-1", _request(checkpointer, event_store)))
assert [call.kwargs["thread_id"] for call in thread_builder.await_args_list] == ["thread-1", "thread-1"]
assert response.checkpoint == {
"checkpoint_ns": "",
"checkpoint_id": "ckpt-base",
"checkpoint_map": None,
}
assert response.target_run_id == "run-old"
assert response.metadata == {
"regenerate_from_message_id": "ai-1",
"regenerate_from_run_id": "run-old",
"regenerate_checkpoint_id": "ckpt-base",
}
assert "title" not in response.input
regenerated_human = response.input["messages"][0]
assert regenerated_human["id"] == "human-1"
assert regenerated_human["content"] == [{"type": "text", "text": "/data-analysis analyze data.csv"}]
assert regenerated_human["additional_kwargs"] == {"files": [{"filename": "data.csv", "path": "/mnt/user-data/uploads/data.csv"}]}
@pytest.mark.parametrize("materialized_delta", [False, True], ids=["full", "delta"])
def test_prepare_regenerate_payload_replays_latest_confirmed_human_input_response(
materialized_delta: bool,
):
from app.gateway.routers import thread_runs
original_human = HumanMessage(id="human-1", content="Help me plan a trip")
first_clarification = ToolMessage(
id="tool-city",
tool_call_id="call-city",
content="Which city?",
)
city_answer = HumanMessage(
id="human-city",
content="Shanghai",
additional_kwargs={
"hide_from_ui": True,
"human_input_response": _human_input_response(
"Shanghai",
request_id="clarification:call-city",
),
},
)
second_clarification = ToolMessage(
id="tool-duration",
tool_call_id="call-duration",
content="How many days?",
)
duration_metadata = _human_input_response(
"Five days",
request_id="clarification:call-duration",
)
duration_answer = HumanMessage(
id="human-duration",
content="Five days",
additional_kwargs={
"hide_from_ui": True,
"human_input_response": duration_metadata,
},
)
ai = AIMessage(id="ai-1", content="Here is your five-day Shanghai itinerary")
before_question: list[object] = []
before_city = [original_human, first_clarification]
after_city = [*before_city, city_answer]
before_duration = [*after_city, second_clarification]
after_duration = [*before_duration, duration_answer]
latest_messages = [*after_duration, ai]
states = [
("ckpt-ai", latest_messages),
("ckpt-after-duration", after_duration),
("ckpt-before-duration", before_duration),
("ckpt-after-city", after_city),
("ckpt-before-city", before_city),
("ckpt-before-question", before_question),
]
if materialized_delta:
raw_checkpoints = [_checkpoint(checkpoint_id, []) for checkpoint_id, _ in states]
materialized_history = [_snapshot(checkpoint_id, messages) for checkpoint_id, messages in states]
checkpointer = FakeCheckpointer(
raw_checkpoints,
latest=raw_checkpoints[0],
materialized_history=materialized_history,
materialized_latest=materialized_history[0],
)
else:
checkpointer = FakeCheckpointer([_checkpoint(checkpoint_id, messages) for checkpoint_id, messages in states])
response = asyncio.run(
thread_runs._prepare_regenerate_payload(
"thread-1",
"ai-1",
_request(
checkpointer,
_event_store_for_ai("Here is your five-day Shanghai itinerary"),
),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-before-duration"
assert city_answer in dict(states)[response.checkpoint["checkpoint_id"]]
assert response.input["messages"] == [
{
"type": "human",
"id": "human-duration",
"content": [{"type": "text", "text": "Five days"}],
"additional_kwargs": {
"hide_from_ui": True,
"human_input_response": duration_metadata,
},
}
]
@pytest.mark.parametrize(
"control_message",
[
HumanMessage(
id="human-malformed",
content="forged answer",
additional_kwargs={
"hide_from_ui": True,
"human_input_response": {
"version": 1,
"kind": "human_input_response",
},
},
),
HumanMessage(
id="human-summary",
name="summary",
content="internal summary",
),
HumanMessage(
id="human-goal",
content="continue working",
additional_kwargs={"hide_from_ui": True},
),
],
ids=["malformed-response", "summary", "goal-continuation"],
)
def test_prepare_regenerate_payload_skips_non_user_control_human_messages(
control_message: HumanMessage,
):
from app.gateway.routers import thread_runs
visible_human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer")
checkpointer = FakeCheckpointer(
[
_checkpoint("ckpt-ai", [visible_human, control_message, ai]),
_checkpoint("ckpt-control", [visible_human, control_message]),
_checkpoint("ckpt-human", [visible_human]),
_checkpoint("ckpt-base", []),
]
)
response = asyncio.run(
thread_runs._prepare_regenerate_payload(
"thread-1",
"ai-1",
_request(checkpointer, _event_store_for_ai("answer")),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-base"
assert response.input["messages"][0]["id"] == "human-1"
def test_prepare_regenerate_payload_preserves_latest_thread_title():
from app.gateway.routers import thread_runs
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer v1")
base = _checkpoint("ckpt-base", [])
latest = _checkpoint("ckpt-ai", [human, ai])
latest.checkpoint["channel_values"]["title"] = "User renamed title"
checkpointer = FakeCheckpointer([latest, base])
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer v1"},
"metadata": {"caller": "lead_agent"},
}
]
)
response = asyncio.run(
thread_runs._prepare_regenerate_payload(
"thread-1",
"ai-1",
_request(checkpointer, event_store),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-base"
assert response.input["title"] == "User renamed title"
def test_prepare_regenerate_payload_supports_latest_interrupted_response_missing_from_checkpoint():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
human = HumanMessage(
id="human-1",
content="question",
additional_kwargs={"run_id": "run-interrupted"},
)
base = _checkpoint("ckpt-base", [])
latest = _checkpoint("ckpt-human", [human])
checkpointer = FakeCheckpointer([latest, base])
run_manager = FakeRunManager(
[
SimpleNamespace(
run_id="run-interrupted",
thread_id="thread-1",
status=RunStatus.interrupted,
)
]
)
response = asyncio.run(
_prepare_regenerate_payload(
"thread-1",
"lc_run--partial-response",
_request(
checkpointer,
FakeEventStore([]),
run_manager=run_manager,
),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-base"
assert response.target_run_id == "run-interrupted"
assert response.metadata == {
"regenerate_from_message_id": "lc_run--partial-response",
"regenerate_from_run_id": "run-interrupted",
"regenerate_checkpoint_id": "ckpt-base",
}
assert response.input["messages"][0]["id"] == "human-1"
@pytest.mark.parametrize(
("status", "record_thread_id"),
[
(RunStatus.success, "thread-1"),
(RunStatus.interrupted, "another-thread"),
],
)
def test_prepare_regenerate_payload_does_not_accept_unverified_missing_response(
status: RunStatus,
record_thread_id: str,
):
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
human = HumanMessage(
id="human-1",
content="question",
additional_kwargs={"run_id": "source-run"},
)
checkpointer = FakeCheckpointer(
[
_checkpoint("ckpt-human", [human]),
_checkpoint("ckpt-base", []),
]
)
run_manager = FakeRunManager(
[
SimpleNamespace(
run_id="source-run",
thread_id=record_thread_id,
status=status,
)
]
)
with pytest.raises(HTTPException) as exc:
asyncio.run(
_prepare_regenerate_payload(
"thread-1",
"missing-response",
_request(
checkpointer,
FakeEventStore([]),
run_manager=run_manager,
),
)
)
assert exc.value.status_code == 404
assert exc.value.detail == "Message missing-response not found"
def test_prepare_regenerate_payload_does_not_mutate_legacy_single_checkpoint_branch():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
checkpointer = InMemorySaver()
source_thread_id = "source-thread"
branch_thread_id = "legacy-branch"
source_run_id = "source-run"
human = HumanMessage(id="human-1", content="question", additional_kwargs={"run_id": source_run_id})
ai = AIMessage(id="ai-1", content="answer")
async def _seed() -> str:
source_base_config = await _put_memory_checkpoint(checkpointer, source_thread_id, [], step=0)
after_human = await _put_memory_checkpoint(
checkpointer,
source_thread_id,
[human],
step=1,
parent_config=source_base_config,
)
source_head_config = await _put_memory_checkpoint(
checkpointer,
source_thread_id,
[human, ai],
step=2,
parent_config=after_human,
)
source_head = await checkpointer.aget_tuple(source_head_config)
assert source_head is not None
legacy_head = copy.deepcopy(source_head.checkpoint)
legacy_head_id = str(uuid6())
legacy_head["id"] = legacy_head_id
legacy_metadata = copy.deepcopy(source_head.metadata)
legacy_metadata.update(
{
"source": "branch",
"deerflow_branch": True,
"branch_parent_thread_id": source_thread_id,
"branch_parent_checkpoint_id": source_head_config["configurable"]["checkpoint_id"],
"branch_parent_message_id": "ai-1",
}
)
await checkpointer.aput(
{"configurable": {"thread_id": branch_thread_id, "checkpoint_ns": ""}},
legacy_head,
legacy_metadata,
dict(legacy_head["channel_versions"]),
)
return legacy_head_id
legacy_head_id = asyncio.run(_seed())
request = _request(checkpointer, FakeEventStore([]))
with pytest.raises(HTTPException) as exc:
asyncio.run(_prepare_regenerate_payload(branch_thread_id, "ai-1", request))
assert exc.value.status_code == 409
assert exc.value.detail == "Could not find an addressable checkpoint before the target user message"
latest = asyncio.run(checkpointer.aget_tuple({"configurable": {"thread_id": branch_thread_id, "checkpoint_ns": ""}}))
assert latest is not None
assert latest.config["configurable"]["checkpoint_id"] == legacy_head_id
branch_history = asyncio.run(_collect_checkpoints(checkpointer, {"configurable": {"thread_id": branch_thread_id, "checkpoint_ns": ""}}))
assert [item.config["configurable"]["checkpoint_id"] for item in branch_history] == [legacy_head_id]
def test_prepare_regenerate_payload_rejects_legacy_branch_when_source_checkpoint_is_missing():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
checkpointer = InMemorySaver()
branch_thread_id = "legacy-orphan"
human = HumanMessage(id="human-1", content="question", additional_kwargs={"run_id": "source-run"})
ai = AIMessage(id="ai-1", content="answer")
async def _seed() -> None:
await _put_memory_checkpoint(
checkpointer,
branch_thread_id,
[human, ai],
step=1,
metadata={
"source": "branch",
"deerflow_branch": True,
"branch_parent_thread_id": "deleted-source",
"branch_parent_checkpoint_id": "missing-checkpoint",
"branch_parent_message_id": "ai-1",
},
)
asyncio.run(_seed())
with pytest.raises(HTTPException) as exc:
asyncio.run(
_prepare_regenerate_payload(
branch_thread_id,
"ai-1",
_request(checkpointer, FakeEventStore([])),
)
)
assert exc.value.status_code == 409
assert exc.value.detail == "Could not find an addressable checkpoint before the target user message"
branch_history = asyncio.run(_collect_checkpoints(checkpointer, {"configurable": {"thread_id": branch_thread_id, "checkpoint_ns": ""}}))
assert len(branch_history) == 1
def test_prepare_edit_regenerate_payload_returns_new_human_and_edit_metadata():
from app.gateway.routers import thread_runs
human = HumanMessage(
id="human-1",
content="<current_uploads>injected</current_uploads>\n\noriginal question",
name="researcher",
additional_kwargs={
ORIGINAL_USER_CONTENT_KEY: "original question",
"files": [{"filename": "data.csv", "path": "/mnt/user-data/uploads/data.csv"}],
"referenced_message_contexts": [{"message_id": "ai-prev", "quote": "quoted"}],
"hide_from_ui": False,
"run_id": "old-run",
"timestamp": "2026-07-22T00:00:00Z",
"middleware_private": "do-not-copy",
},
)
ai = AIMessage(id="ai-1", content="answer v1")
base = _checkpoint("ckpt-base", [])
after_human = _checkpoint("ckpt-human", [human])
latest = _checkpoint("ckpt-ai", [human, ai])
checkpointer = FakeCheckpointer([latest, after_human, base])
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer v1"},
"metadata": {"caller": "lead_agent"},
}
]
)
run_manager = FakeRunManager(
[
SimpleNamespace(
run_id="run-old",
status=RunStatus.success,
metadata={},
last_ai_message="answer v1",
)
]
)
response = asyncio.run(
thread_runs._prepare_edit_regenerate_payload(
"thread-1",
"human-1",
" updated question\nwith details ",
_request(checkpointer, event_store, run_manager=run_manager),
)
)
assert response.checkpoint == {
"checkpoint_ns": "",
"checkpoint_id": "ckpt-base",
"checkpoint_map": None,
}
assert response.target_run_id == "run-old"
assert response.replacement_human_message_id != "human-1"
assert response.source_message_ids == ["human-1", "ai-1"]
assert response.metadata == {
"replay_kind": "edit",
"regenerate_from_message_id": "ai-1",
"regenerate_from_run_id": "run-old",
"regenerate_checkpoint_id": "ckpt-base",
"edit_from_message_id": "human-1",
"edit_message_id": response.replacement_human_message_id,
"edit_version_group_id": "human-1",
}
replacement = response.input["messages"][0]
assert replacement == {
"type": "human",
"id": response.replacement_human_message_id,
"name": "researcher",
"content": [{"type": "text", "text": "updated question\nwith details"}],
"additional_kwargs": {
"files": [{"filename": "data.csv", "path": "/mnt/user-data/uploads/data.csv"}],
"referenced_message_contexts": [{"message_id": "ai-prev", "quote": "quoted"}],
},
}
def _first_turn_checkpointer() -> FakeCheckpointer:
"""First-turn history where the user message's id is swapped mid-run.
``DynamicContextMiddleware`` moves the first user message to ``{id}__user``
and gives ``{id}`` to the injected reminder, so every checkpoint written
before that node ran holds the same prompt under an id the replay-base
lookup cannot match (#4531).
"""
system = SystemMessage(id="human-1", content="<system-reminder>date</system-reminder>")
swapped_human = HumanMessage(id="human-1__user", content="original question")
raw_human = HumanMessage(id="human-1", content="original question")
ai = AIMessage(id="ai-1", content="answer v1")
return FakeCheckpointer(
[
_checkpoint("ckpt-head", [system, swapped_human, ai]),
_checkpoint("ckpt-after-inject", [system, swapped_human], next_tasks=("LoopDetectionMiddleware.before_agent",)),
_checkpoint("ckpt-mid", [raw_human], next_tasks=("DynamicContextMiddleware.before_agent",)),
_checkpoint("ckpt-input", [], next_tasks=("__start__",)),
_checkpoint("ckpt-empty", []),
]
)
def _answer_run_fixtures() -> tuple[FakeEventStore, FakeRunManager]:
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer v1"},
"metadata": {"caller": "lead_agent"},
}
]
)
run_manager = FakeRunManager([SimpleNamespace(run_id="run-old", status=RunStatus.success, metadata={}, last_ai_message="answer v1")])
return event_store, run_manager
def test_prepare_edit_regenerate_payload_preserves_a_rename_the_replay_base_predates():
"""Replaying a turn must not roll the thread back to an older title (#4457)."""
from app.gateway.routers import thread_runs
human = HumanMessage(id="human-1", content="original question")
ai = AIMessage(id="ai-1", content="answer v1")
base = _checkpoint("ckpt-base", [])
base.checkpoint["channel_values"]["title"] = "auto generated title"
latest = _checkpoint("ckpt-ai", [human, ai])
latest.checkpoint["channel_values"]["title"] = "User renamed title"
event_store, run_manager = _answer_run_fixtures()
response = asyncio.run(
thread_runs._prepare_edit_regenerate_payload(
"thread-1",
"human-1",
"edited question",
_request(FakeCheckpointer([latest, base]), event_store, run_manager=run_manager),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-base"
assert response.input["title"] == "User renamed title"
def test_prepare_edit_regenerate_payload_lets_an_untitled_base_name_the_edited_turn():
"""A base with no title belongs to a thread that has not been named yet.
Pinning the current title there would keep a name generated from the prompt
the edit just replaced, so leave the channel empty and let the title
middleware name the rewritten turn.
"""
from app.gateway.routers import thread_runs
human = HumanMessage(id="human-1", content="original question")
ai = AIMessage(id="ai-1", content="answer v1")
latest = _checkpoint("ckpt-ai", [human, ai])
latest.checkpoint["channel_values"]["title"] = "title of the replaced question"
event_store, run_manager = _answer_run_fixtures()
response = asyncio.run(
thread_runs._prepare_edit_regenerate_payload(
"thread-1",
"human-1",
"edited question",
_request(FakeCheckpointer([latest, _checkpoint("ckpt-base", [])]), event_store, run_manager=run_manager),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-base"
assert "title" not in response.input
def test_prepare_regenerate_payload_replays_the_pre_swap_user_message_id():
"""Replaying `{id}__user` would make the reminder middleware skip the turn.
The replay base predates the injection, so the turn must re-enter the graph
under the id the client originally sent or it loses its date/memory block.
"""
from app.gateway.routers import thread_runs
event_store, run_manager = _answer_run_fixtures()
response = asyncio.run(
thread_runs._prepare_regenerate_payload(
"thread-1",
"ai-1",
_request(_first_turn_checkpointer(), event_store, run_manager=run_manager),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-empty"
assert response.input["messages"][0]["id"] == "human-1"
def test_prepare_edit_regenerate_payload_skips_mid_run_replay_base_on_first_turn():
"""The replay base must predate the turn, not sit inside the run that produced it.
``ckpt-mid`` still holds the original prompt (under its pre-swap id) and owns
the injection node's pending writes, so replaying from it re-adds the prompt
the edit is meant to replace.
"""
from app.gateway.routers import thread_runs
event_store, run_manager = _answer_run_fixtures()
response = asyncio.run(
thread_runs._prepare_edit_regenerate_payload(
"thread-1",
"human-1__user",
"edited question",
_request(_first_turn_checkpointer(), event_store, run_manager=run_manager),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-empty"
assert response.metadata["regenerate_checkpoint_id"] == "ckpt-empty"
def test_prepare_edit_regenerate_payload_prefers_checkpoint_lineage():
"""Edit replay must resolve its base the same lineage-first way regenerate does.
A chronological scan cannot tell sibling branches apart (#4358), so the head
checkpoint has to reach the lineage walk.
"""
from app.gateway.routers import thread_runs
event_store, run_manager = _answer_run_fixtures()
checkpointer = _first_turn_checkpointer()
from app.gateway.checkpoint_lineage import CheckpointParentMissingError
walk = AsyncMock(side_effect=CheckpointParentMissingError("no parent link"))
with patch.object(thread_runs, "find_checkpoint_before_message", walk):
response = asyncio.run(
thread_runs._prepare_edit_regenerate_payload(
"thread-1",
"human-1__user",
"edited question",
_request(checkpointer, event_store, run_manager=run_manager),
)
)
assert walk.await_count == 1
head_checkpoint = walk.await_args.args[1]
assert head_checkpoint.config["configurable"]["checkpoint_id"] == "ckpt-head"
# Legacy checkpoints without parent links still degrade to the bounded scan.
assert response.checkpoint["checkpoint_id"] == "ckpt-empty"
@pytest.mark.parametrize(
("replacement_text", "detail"),
[
(" \n\t", "Edited message cannot be empty"),
(" original question ", "Edited message is unchanged"),
],
)
def test_prepare_edit_regenerate_payload_rejects_empty_or_unchanged_text(replacement_text: str, detail: str):
from app.gateway.routers.thread_runs import _prepare_edit_regenerate_payload
human = HumanMessage(id="human-1", content="original question")
ai = AIMessage(id="ai-1", content="answer")
checkpointer = FakeCheckpointer(
[
_checkpoint("ckpt-ai", [human, ai]),
_checkpoint("ckpt-human", [human]),
_checkpoint("ckpt-base", []),
]
)
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer"},
"metadata": {"caller": "lead_agent"},
}
]
)
run_manager = FakeRunManager([SimpleNamespace(run_id="run-old", status=RunStatus.success, metadata={}, last_ai_message="answer")])
with pytest.raises(HTTPException) as exc:
asyncio.run(
_prepare_edit_regenerate_payload(
"thread-1",
"human-1",
replacement_text,
_request(checkpointer, event_store, run_manager=run_manager),
)
)
assert exc.value.status_code == 409
assert exc.value.detail == detail
def test_prepare_edit_regenerate_payload_requires_latest_human_turn():
from app.gateway.routers.thread_runs import _prepare_edit_regenerate_payload
old_human = HumanMessage(id="human-old", content="old question")
old_ai = AIMessage(id="ai-old", content="old answer")
latest_human = HumanMessage(id="human-latest", content="latest question")
latest_ai = AIMessage(id="ai-latest", content="latest answer")
checkpointer = FakeCheckpointer(
[
_checkpoint("ckpt-latest", [old_human, old_ai, latest_human, latest_ai]),
_checkpoint("ckpt-base", []),
]
)
with pytest.raises(HTTPException) as exc:
asyncio.run(
_prepare_edit_regenerate_payload(
"thread-1",
"human-old",
"edited old question",
_request(checkpointer, FakeEventStore([])),
)
)
assert exc.value.status_code == 409
assert exc.value.detail == "Only the latest completed user turn can be edited"
def test_prepare_edit_regenerate_payload_allows_answered_historical_clarification():
from app.gateway.routers import thread_runs
old_human = HumanMessage(id="human-old", content="ambiguous request")
clarification = ToolMessage(
id="tool-clarify",
tool_call_id="call-clarify",
content="need input",
artifact={
"human_input": {
"version": 1,
"kind": "human_input_request",
"request_id": "clarify-1",
"prompt": "Which format?",
}
},
)
clarification_answer = HumanMessage(
id="human-clarify-answer",
content="Use Markdown",
additional_kwargs={
"hide_from_ui": True,
"human_input_response": {
"version": 1,
"kind": "human_input_response",
"request_id": "clarify-1",
"source": "user",
"value": "Use Markdown",
},
},
)
latest_human = HumanMessage(id="human-latest", content="latest question")
latest_ai = AIMessage(id="ai-latest", content="latest answer")
historical_messages = [old_human, clarification, clarification_answer]
checkpointer = FakeCheckpointer(
[
_checkpoint("ckpt-ai", [*historical_messages, latest_human, latest_ai]),
_checkpoint("ckpt-human", [*historical_messages, latest_human]),
_checkpoint("ckpt-base", historical_messages),
]
)
event_store = FakeEventStore(
[
{
"run_id": "run-latest",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-latest", "type": "ai", "content": "latest answer"},
"metadata": {"caller": "lead_agent"},
}
]
)
run_manager = FakeRunManager(
[
SimpleNamespace(
run_id="run-latest",
status=RunStatus.success,
metadata={},
last_ai_message="latest answer",
)
]
)
response = asyncio.run(
thread_runs._prepare_edit_regenerate_payload(
"thread-1",
"human-latest",
"updated latest question",
_request(checkpointer, event_store, run_manager=run_manager),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-base"
assert response.target_run_id == "run-latest"
assert response.source_message_ids == ["human-latest", "ai-latest"]
def test_prepare_edit_regenerate_payload_rejects_open_clarification_turn():
from app.gateway.routers.thread_runs import _prepare_edit_regenerate_payload
human = HumanMessage(id="human-1", content="question")
tool = ToolMessage(
id="tool-1",
tool_call_id="call-1",
content="need input",
artifact={"human_input": {"request_id": "clarify-1", "status": "pending"}},
)
checkpointer = FakeCheckpointer(
[
_checkpoint("ckpt-tool", [human, tool]),
_checkpoint("ckpt-human", [human]),
_checkpoint("ckpt-base", []),
]
)
with pytest.raises(HTTPException) as exc:
asyncio.run(
_prepare_edit_regenerate_payload(
"thread-1",
"human-1",
"updated question",
_request(checkpointer, FakeEventStore([])),
)
)
assert exc.value.status_code == 409
assert exc.value.detail == "Only completed assistant text turns can be edited"
def test_prepare_edit_regenerate_payload_rejects_active_goal():
from app.gateway.routers.thread_runs import _prepare_edit_regenerate_payload
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer")
checkpointer = FakeCheckpointer(
[
_checkpoint("ckpt-ai", [human, ai], goal={"status": "active", "objective": "finish"}),
_checkpoint("ckpt-human", [human]),
_checkpoint("ckpt-base", []),
]
)
with pytest.raises(HTTPException) as exc:
asyncio.run(
_prepare_edit_regenerate_payload(
"thread-1",
"human-1",
"updated question",
_request(checkpointer, FakeEventStore([])),
)
)
assert exc.value.status_code == 409
assert exc.value.detail == "Cannot edit while a goal is active"
@pytest.mark.parametrize(
"latest_values",
[
# Clearing a met goal leaves only its record.
{"goal_outcome": {"status": "achieved", "objective": "finish", "reply_message_id": "ai-1"}},
# POST /state stores an unbuilt goal the history head does not show as active.
{"goal": {"objective": "finish"}},
],
ids=["met-goal-record", "goal-without-status"],
)
def test_prepare_edit_regenerate_payload_allows_editing_without_an_active_goal(latest_values):
from app.gateway.routers.thread_runs import _prepare_edit_regenerate_payload
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer v1")
latest = _checkpoint("ckpt-ai", [human, ai])
latest.checkpoint["channel_values"].update(latest_values)
event_store, run_manager = _answer_run_fixtures()
response = asyncio.run(
_prepare_edit_regenerate_payload(
"thread-1",
"human-1",
"updated question",
_request(FakeCheckpointer([latest, _checkpoint("ckpt-human", [human]), _checkpoint("ckpt-base", [])]), event_store, run_manager=run_manager),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-base"
def test_prepare_edit_regenerate_payload_requires_successful_source_run():
from app.gateway.routers.thread_runs import _prepare_edit_regenerate_payload
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer")
checkpointer = FakeCheckpointer(
[
_checkpoint("ckpt-ai", [human, ai]),
_checkpoint("ckpt-human", [human]),
_checkpoint("ckpt-base", []),
]
)
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer"},
"metadata": {"caller": "lead_agent"},
}
]
)
run_manager = FakeRunManager([SimpleNamespace(run_id="run-old", status=RunStatus.error, metadata={}, last_ai_message="answer")])
with pytest.raises(HTTPException) as exc:
asyncio.run(
_prepare_edit_regenerate_payload(
"thread-1",
"human-1",
"updated question",
_request(checkpointer, event_store, run_manager=run_manager),
)
)
assert exc.value.status_code == 409
assert exc.value.detail == "Only successful assistant runs can be edited and rerun"
def test_prepare_regenerate_uses_materialized_history_when_raw_messages_are_omitted():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
earlier_human = HumanMessage(id="human-0", content="earlier question")
earlier_ai = AIMessage(id="ai-0", content="earlier answer")
target_human = HumanMessage(id="human-1", content="question")
target_ai = AIMessage(id="ai-1", content="answer")
raw_latest = _checkpoint("ckpt-ai", [])
raw_after_human = _checkpoint("ckpt-human", [])
raw_base = _checkpoint("ckpt-base", [])
materialized_history = [
_snapshot("ckpt-ai", [earlier_human, earlier_ai, target_human, target_ai]),
_snapshot("ckpt-human", [earlier_human, earlier_ai, target_human]),
_snapshot("ckpt-base", [earlier_human, earlier_ai]),
]
checkpointer = FakeCheckpointer(
[raw_latest, raw_after_human, raw_base],
latest=raw_latest,
materialized_history=materialized_history,
materialized_latest=materialized_history[0],
)
event_store = FakeEventStore(
[
{
"run_id": "run-target",
"event_type": "ai_message",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer"},
"metadata": {"caller": "lead_agent"},
}
]
)
response = asyncio.run(
_prepare_regenerate_payload(
"thread-1",
"ai-1",
_request(checkpointer, event_store),
)
)
assert response.checkpoint["checkpoint_id"] == "ckpt-base"
assert response.metadata["regenerate_checkpoint_id"] == "ckpt-base"
assert response.input["messages"][0]["id"] == "human-1"
assert checkpointer.alist_limits == [400]
def test_prepare_regenerate_rejects_cyclic_lineage_without_chronological_fallback():
from app.gateway.routers import thread_runs
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer")
def linked_snapshot(checkpoint_id: str, messages: list[object], parent_id: str) -> SimpleNamespace:
return SimpleNamespace(
values={"messages": messages},
config={
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
"checkpoint_id": checkpoint_id,
}
},
metadata={},
parent_config={
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
"checkpoint_id": parent_id,
}
},
)
head = linked_snapshot("head", [human, ai], "cycle")
cycle = linked_snapshot("cycle", [human], "head")
wrong_sibling_base = linked_snapshot("wrong-sibling", [], "root")
by_id = {"head": head, "cycle": cycle}
async def aget(config):
return by_id[config["configurable"]["checkpoint_id"]]
accessor = SimpleNamespace(
aget=aget,
ahistory=AsyncMock(return_value=[head, wrong_sibling_base]),
)
builder = AsyncMock(
return_value=(
accessor,
{"configurable": {"thread_id": "thread-1", "checkpoint_ns": ""}},
)
)
with patch.object(thread_runs, "build_thread_checkpoint_state_accessor", builder):
with pytest.raises(HTTPException) as exc:
asyncio.run(
thread_runs._find_base_checkpoint_before_human(
"thread-1",
"human-1",
_request(FakeCheckpointer([]), FakeEventStore([])),
head_checkpoint=head,
)
)
assert exc.value.status_code == 409
assert exc.value.detail == "Could not safely resolve the checkpoint before the target user message"
accessor.ahistory.assert_not_awaited()
def test_prepare_regenerate_rejects_dangling_parent_without_chronological_fallback():
from app.gateway.routers import thread_runs
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer")
head = SimpleNamespace(
values={"messages": [human, ai]},
config={
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
"checkpoint_id": "head",
}
},
metadata={},
parent_config={
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
"checkpoint_id": "missing",
}
},
)
missing = SimpleNamespace(
values={},
config=head.parent_config,
metadata=None,
created_at=None,
parent_config=None,
)
accessor = SimpleNamespace(
aget=AsyncMock(return_value=missing),
ahistory=AsyncMock(return_value=[head, _snapshot("wrong-sibling", [])]),
)
builder = AsyncMock(
return_value=(
accessor,
{"configurable": {"thread_id": "thread-1", "checkpoint_ns": ""}},
)
)
with patch.object(thread_runs, "build_thread_checkpoint_state_accessor", builder):
with pytest.raises(HTTPException) as exc:
asyncio.run(
thread_runs._find_base_checkpoint_before_human(
"thread-1",
"human-1",
_request(FakeCheckpointer([]), FakeEventStore([])),
head_checkpoint=head,
)
)
assert exc.value.status_code == 409
assert exc.value.detail == "Could not safely resolve the checkpoint before the target user message"
accessor.ahistory.assert_not_awaited()
def test_prepare_regenerate_payload_rejects_non_latest_assistant():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
human = HumanMessage(id="human-1", content="question")
old_ai = AIMessage(id="ai-old", content="old")
latest_ai = AIMessage(id="ai-latest", content="latest")
base = _checkpoint("ckpt-base", [])
after_human = _checkpoint("ckpt-human", [human])
latest = _checkpoint("ckpt-latest", [human, old_ai, latest_ai])
checkpointer = FakeCheckpointer([latest, after_human, base])
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "ai_message",
"category": "message",
"content": {"id": "ai-old", "type": "ai", "content": "old"},
"metadata": {"caller": "lead_agent"},
}
]
)
with pytest.raises(HTTPException) as exc:
asyncio.run(_prepare_regenerate_payload("thread-1", "ai-old", _request(checkpointer, event_store)))
assert exc.value.status_code == 409
assert exc.value.detail == "Only the latest assistant message can be regenerated"
@pytest.mark.parametrize(
("content", "summary"),
[
("answer", "answer"),
("<think>Reasoning.</think>answer", "<think>Reasoning.</think>answer"),
("<think>Reasoning.</think>answer", "answer"),
("<think>" + "r" * 2400 + "</think>" + "a" * 2500, "a" * 2000),
],
ids=["plain", "legacy-summary", "visible-summary", "long-reasoning"],
)
def test_prepare_regenerate_payload_falls_back_to_matching_run_when_events_are_missing(content, summary):
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content=content)
base = _checkpoint("ckpt-base", [])
after_human = _checkpoint("ckpt-human", [human])
latest = _checkpoint("ckpt-ai", [human, ai])
checkpointer = FakeCheckpointer([latest, after_human, base])
run_manager = FakeRunManager(
[
RunRecord(run_id="run-latest", thread_id="thread-1", assistant_id="lead_agent", status=RunStatus.success, on_disconnect=DisconnectMode.continue_, last_ai_message=summary),
SimpleNamespace(run_id="run-older", status=RunStatus.error, last_ai_message="answer"),
]
)
response = asyncio.run(
_prepare_regenerate_payload(
"thread-1",
"ai-1",
_request(checkpointer, FakeEventStore([]), run_manager=run_manager),
)
)
assert response.target_run_id == "run-latest"
assert response.metadata["regenerate_from_run_id"] == "run-latest"
def test_prepare_regenerate_payload_uses_server_stamped_human_run_id_without_parent_events():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
human = HumanMessage(id="human-1", content="question", additional_kwargs={"run_id": "parent-run"})
ai = AIMessage(id="ai-1", content="answer")
base = _checkpoint("ckpt-base", [])
after_human = _checkpoint("ckpt-human", [human])
latest = _checkpoint(
"ckpt-ai",
[human, ai],
metadata={
"deerflow_branch": True,
"branch_parent_thread_id": "parent-thread",
"branch_parent_checkpoint_id": "parent-checkpoint",
},
)
checkpointer = FakeCheckpointer([latest, after_human, base])
event_store = FakeEventStore([])
response = asyncio.run(_prepare_regenerate_payload("thread-1", "ai-1", _request(checkpointer, event_store)))
assert response.target_run_id == "parent-run"
def test_prepare_regenerate_payload_rejects_unverified_run_fallback_when_events_are_missing():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer")
base = _checkpoint("ckpt-base", [])
after_human = _checkpoint("ckpt-human", [human])
latest = _checkpoint("ckpt-ai", [human, ai])
checkpointer = FakeCheckpointer([latest, after_human, base])
run_manager = FakeRunManager(
[
SimpleNamespace(run_id="run-latest", status=RunStatus.success, last_ai_message="different"),
]
)
with pytest.raises(HTTPException) as exc:
asyncio.run(
_prepare_regenerate_payload(
"thread-1",
"ai-1",
_request(checkpointer, FakeEventStore([]), run_manager=run_manager),
)
)
assert exc.value.status_code == 409
assert exc.value.detail == "Could not find source run for assistant message"
def test_prepare_regenerate_payload_requires_addressable_checkpoint_before_human():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer")
latest = _checkpoint("ckpt-ai", [human, ai])
checkpointer = FakeCheckpointer([latest])
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer"},
"metadata": {"caller": "lead_agent"},
}
]
)
with pytest.raises(HTTPException) as exc:
asyncio.run(_prepare_regenerate_payload("thread-1", "ai-1", _request(checkpointer, event_store)))
assert exc.value.status_code == 409
assert exc.value.detail == "Could not find an addressable checkpoint before the target user message"
assert checkpointer.alist_limits == [400]
def test_prepare_regenerate_payload_reports_recent_checkpoint_scan_limit():
from app.gateway.routers.thread_runs import _prepare_regenerate_payload
human = HumanMessage(id="human-1", content="question")
ai = AIMessage(id="ai-1", content="answer")
latest = _checkpoint("ckpt-latest", [human, ai])
history_without_human = [_checkpoint(f"ckpt-{index}", []) for index in range(201)]
checkpointer = FakeCheckpointer(history_without_human, latest=latest)
event_store = FakeEventStore(
[
{
"run_id": "run-old",
"event_type": "llm.ai.response",
"category": "message",
"content": {"id": "ai-1", "type": "ai", "content": "answer"},
"metadata": {"caller": "lead_agent"},
}
]
)
with pytest.raises(HTTPException) as exc:
asyncio.run(_prepare_regenerate_payload("thread-1", "ai-1", _request(checkpointer, event_store)))
assert exc.value.status_code == 409
assert exc.value.detail == "Could not locate target user message in recent checkpoint history (limit=200)"
assert checkpointer.alist_limits == [400]
def test_find_base_checkpoint_ignores_duration_only_checkpoints() -> None:
from app.gateway.routers.thread_runs import _find_base_checkpoint_before_human
human = HumanMessage(id="human-1", content="question")
duration_checkpoints = [
_checkpoint(
f"duration-{index}",
[],
metadata={"writes": {"runtime_run_duration": {"run_ids": [f"run-{index}"]}}},
)
for index in range(200)
]
base = _checkpoint("ckpt-base", [])
after_human = _checkpoint("ckpt-human", [human])
checkpointer = FakeCheckpointer([*duration_checkpoints, after_human, base])
result = asyncio.run(_find_base_checkpoint_before_human("thread-1", "human-1", _request(checkpointer, FakeEventStore([]))))
assert result.config == base.config
assert checkpointer.alist_limits == [400]