github/deer-flow
1
0
Fork 0
mirror of https://github.com/bytedance/deer-flow.git synced 2026-09-19 11:06:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(agents): close delegations a stopped run left in progress (#5507)

Browse Source 
* fix(agents): close delegations a stopped run left in progress

Every task call is recorded in the delegation ledger as in_progress and
only moves on when its ToolMessage arrives. When the user stops a run
while a subagent is running, the task tool re-raises the cancellation
and no ToolMessage is written, so the entry stayed in_progress for the
rest of the thread and every later model call was told "already
delegated; do NOT delegate again; wait for or build on the result".

When a run starts with a new user message, mark entries that an earlier
run left in_progress and that have no ToolMessage as cancelled. Resumed
runs, which have no new user message, keep the current behaviour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(agents): share the run-opening boundary between capture and closure

Also pin that in_progress entries without a run_id are never closed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(agents): clarify legacy delegation reply handling

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
alanhuangyoo 2026-09-19 10:40:22 +08:00 committed by GitHub
parent 208ba7cc65
commit 23cc4f4d00
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 208 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@ -1450,6 +1450,11 @@ same optional field is supported in the agent's `config.yaml`.
Sub-agents are an optimization, not the default response to a complex request.
After Stop interrupts a delegated task before it returns a reply, the next user
turn marks that earlier task as cancelled in the agent's durable context so it
can retry. Existing replies are preserved. Older replies without status metadata
may still appear in progress; their outcome is not inferred from their text.
The lead agent can spawn sub-agents on the fly — each with its own scoped context, tools, and termination conditions — when delegation has clear net benefit from real parallel latency, specialist capability, or context isolation. It keeps interdependent scopes and overlapping side effects out of parallel dispatch; a bounded sequential chain can still run in one sub-agent when specialist or context-isolation benefit clearly wins. The lead uses the fewest useful sub-agents and re-evaluates later batches instead of fanning out solely because a task is large or multi-step. Sub-agents report back structured results, and the lead agent verifies and synthesizes them into a coherent output. Deterministic tool receipts cover both direct tool messages and state-updating `Command` results such as delegated `task` responses; when the receipt ledger reaches its context budget, it retains the newest actions and their original receipt IDs. Operators can disable this provenance layer with `verification.receipts_enabled: false`. Their configured skills are resolved from the same user-scoped catalog as the lead agent, so user-owned custom skills remain available without exposing another user's version. Their internal AI and tool messages stay scoped to the delegated graph instead of entering the parent chat stream. Reloaded thread history enforces the same boundary: callback-captured sub-agent AI responses remain available in run-event diagnostics but are excluded from the parent transcript, while the parent `task` result remains attached to its subtask card. Long-running sub-agents compact older history when summarization is enabled and re-inject the summary as guarded, hidden durable context before continuing, so recent assistant/tool activity remains grounded in the task. Their system instructions, including the role and report contract, survive compaction; if only those instructions and the current request would be summarized, compaction is skipped. Provider/model request failures are reported as failed sub-agent tasks rather than successful results, so the lead agent and Web UI can react to them correctly. Concurrent parent runs also receive independent server-side sub-agent execution IDs, so a provider that reuses a tool-call ID cannot make one run poll, cancel, or clean up another run's background task. Collapsed sub-agent cards show the effective model and, when the provider returns usage metadata, a cumulative token total that updates after each completed sub-agent LLM call and persists after a reload. When token usage tracking is enabled, completed sub-agent usage is attributed back to the dispatching step from that run's terminal tool-message metadata rather than a process-global provider-ID cache.
An ordinary `task` also receives a defensive snapshot of the dispatching run's current uploads. This lets eligible sub-agents use `list_uploaded_files` to find earlier-turn files without returning same-turn attachments as historical. Delayed or recovered `batch_task` workers leave this tool disabled because they have no valid turn-local upload boundary.

5
backend/packages/harness/deerflow/agents/middlewares/AGENTS.md
View File

@ -17,6 +17,11 @@ omission restores LangChain's 4000-token default.
Persisted delegation verdicts are untrusted durable context; ledger rendering revalidates them and ignores malformed values.
Completed is not accepted; retain useful work and address acceptance gaps.
On new user turns, DurableContext cancels earlier-run unanswered delegations.
It preserves resumes, same-run continuations, and entries without `run_id`.
Any reply prevents cancellation; legacy replies without status metadata may
stay `in_progress`. Never infer status from reply text.
Assembly order: `tool_error_handling_middleware.py::_build_runtime_middlewares` (exposed as `build_lead_runtime_middlewares`), then `../lead_agent/agent.py::build_middlewares` appends lead-only entries. Optional entries require their config/runtime condition.
**Message provenance.** At injection/rewrite, always stamp `additional_kwargs`

64
backend/packages/harness/deerflow/agents/middlewares/durable_context_middleware.py
View File

@ -19,7 +19,7 @@ from deerflow_extension_api import ContentKind, provenance_kwargs
from langchain.agents import AgentState
from langchain.agents.middleware import AgentMiddleware
from langchain.agents.middleware.types import ModelCallResult, ModelRequest, ModelResponse
from langchain_core.messages import AnyMessage, HumanMessage, SystemMessage
from langchain_core.messages import AnyMessage, HumanMessage, SystemMessage, ToolMessage
from langgraph.runtime import Runtime
from deerflow.agents.middlewares.delegation_ledger import extract_delegations, render_delegation_ledger
@ -151,31 +151,61 @@ def _messages_after_pre_existing_boundary(messages: list[AnyMessage], pre_existi
return []
def _current_run_messages(messages: list[AnyMessage], run_id: str | None, pre_existing_message_ids: frozenset[str]) -> list[AnyMessage]:
"""Return the message tail where this invocation may have emitted tasks.
def _run_opening_human_index(messages: list[AnyMessage], run_id: str, pre_existing_message_ids: frozenset[str]) -> int | None:
"""Index of the HumanMessage that opened this run, or None for a resumed run.
A resumed run may not append a new HumanMessage marker. In that case the
latest HumanMessage can belong to an older run. The worker supplies the
message ids that existed before this run so we can capture only newly
appended messages instead of re-tagging old task calls.
The latest HumanMessage opened this run when it carries this run's
``run_id``, or carries none and was not in the thread before the run
started. A resumed run may not append one, so the latest HumanMessage can
belong to an older run. Both the capture window and the decision to close
earlier runs' delegations read this, so they cannot disagree.
"""
if run_id is None:
return messages
for index in range(len(messages) - 1, -1, -1):
message = messages[index]
if not isinstance(message, HumanMessage):
continue
message_run_id = message.additional_kwargs.get("run_id")
if message_run_id == run_id:
return messages[index + 1 :]
if message_run_id is None:
message_id = _message_id(message)
if not pre_existing_message_ids or (message_id is not None and message_id not in pre_existing_message_ids):
return messages[index + 1 :]
return _messages_after_pre_existing_boundary(messages, pre_existing_message_ids)
if message_run_id is not None:
return index if message_run_id == run_id else None
message_id = _message_id(message)
opened = not pre_existing_message_ids or (message_id is not None and message_id not in pre_existing_message_ids)
return index if opened else None
return None
def _current_run_messages(messages: list[AnyMessage], run_id: str | None, pre_existing_message_ids: frozenset[str]) -> list[AnyMessage]:
"""Return the message tail where this invocation may have emitted tasks.
The worker supplies the message ids that existed before this run, so a
resumed run captures only newly appended messages instead of re-tagging
old task calls.
"""
if run_id is None:
return messages
index = _run_opening_human_index(messages, run_id, pre_existing_message_ids)
if index is not None:
return messages[index + 1 :]
return _messages_after_pre_existing_boundary(messages, pre_existing_message_ids)
def _close_delegations_left_by_earlier_runs(messages: list[AnyMessage], existing: list[dict], run_id: str) -> list[dict]:
"""Mark delegations that an earlier run left in_progress without a result as cancelled.
A ``task`` call waits for its subagent, so an entry that is still
in_progress with no ToolMessage once a later user turn starts belongs to a
run that was stopped while the subagent ran. Nothing else will ever update
it, and the ledger would keep telling the model not to delegate again.
Any recorded reply excludes this inference, including legacy ToolMessages
without subagent status metadata. Their outcome is unknown, not evidence
of cancellation. Conservatively leave those entries unchanged, even if
they remain in_progress: this repairs missing replies, not legacy results.
Current task producers stamp metadata for extract_delegations to capture.
"""
answered = {str(message.tool_call_id) for message in messages if isinstance(message, ToolMessage) and message.tool_call_id}
return [{**entry, "status": "cancelled"} for entry in existing if isinstance(entry, dict) and entry.get("status") == "in_progress" and entry.get("run_id") not in (None, run_id) and entry.get("id") not in answered]
def _with_run_id(delegations: list[dict], run_id: str | None, existing: list[dict]) -> list[dict]:
"""Tag only new delegation ids with the current run_id."""
if run_id is None:
@ -243,6 +273,8 @@ class DurableContextMiddleware(AgentMiddleware[AgentState]):
_with_run_id(extract_delegations(messages), run_id, existing),
existing,
)
if run_id is not None and _run_opening_human_index(state["messages"], run_id, pre_existing_message_ids) is not None:
delegations = [*delegations, *_close_delegations_left_by_earlier_runs(state["messages"], existing, run_id)]
if delegations:
return {"delegations": delegations}
return None

150
backend/tests/test_durable_context_middleware.py
View File

@ -1,6 +1,8 @@
import asyncio
from types import SimpleNamespace
from typing import Annotated
import pytest
from _agent_e2e_helpers import FakeToolCallingModel
from langchain.agents import create_agent
from langchain.tools import InjectedToolCallId
@ -413,6 +415,118 @@ class TestBeforeModelCapture:
assert [entry["id"] for entry in out["delegations"]] == ["new-call"]
assert out["delegations"][0]["run_id"] == "run-new"
def test_new_user_turn_closes_delegation_an_earlier_run_left_in_progress(self):
"""A run stopped while its subagent ran never records a result; the next user turn must not keep saying "already delegated"."""
middleware = DurableContextMiddleware()
runtime = SimpleNamespace(context={"run_id": "run-new"})
messages = [
HumanMessage(content="old request", additional_kwargs={"run_id": "run-old"}),
AIMessage(
content="",
tool_calls=[
{
"name": "task",
"args": {"description": "research auth", "prompt": "do it", "subagent_type": "general-purpose"},
"id": "old-call",
"type": "tool_call",
}
],
),
HumanMessage(content="please continue", additional_kwargs={"run_id": "run-new"}),
]
existing = [
{
"id": "old-call",
"run_id": "run-old",
"description": "research auth",
"subagent_type": "general-purpose",
"status": "in_progress",
"created_at": "2026-07-11T00:00:00Z",
}
]
out = middleware.before_model({"messages": messages, "delegations": existing}, runtime)
assert out is not None
assert [(entry["id"], entry["status"], entry["run_id"]) for entry in out["delegations"]] == [("old-call", "cancelled", "run-old")]
ledger = merge_delegations(existing, out["delegations"])
request = middleware._inject(SimpleNamespace(messages=messages, state={"delegations": ledger}, override=lambda **kwargs: SimpleNamespace(**kwargs)))
data = next(message.content for message in request.messages if isinstance(message, HumanMessage) and "<durable_context_data>" in message.content)
assert "do NOT delegate again" not in data
assert "[cancelled] research auth" in data
def test_new_user_turn_keeps_in_progress_delegation_without_run_id(self):
"""Ledger entries written before delegations carried a run_id can't be tied to a run, so they stay as they are."""
middleware = DurableContextMiddleware()
runtime = SimpleNamespace(context={"run_id": "run-new"})
messages = [
HumanMessage(content="old request"),
AIMessage(
content="",
tool_calls=[
{
"name": "task",
"args": {"description": "research auth", "prompt": "do it", "subagent_type": "general-purpose"},
"id": "legacy-call",
"type": "tool_call",
}
],
),
HumanMessage(content="please continue", additional_kwargs={"run_id": "run-new"}),
]
existing = [
{
"id": "legacy-call",
"description": "research auth",
"subagent_type": "general-purpose",
"status": "in_progress",
"created_at": "2026-07-11T00:00:00Z",
}
]
assert middleware.before_model({"messages": messages, "delegations": existing}, runtime) is None
@pytest.mark.parametrize(
"result_metadata",
[
pytest.param(make_subagent_additional_kwargs("completed", result="partial notes"), id="structured-completed"),
pytest.param(make_subagent_additional_kwargs("failed", error="task failed"), id="structured-failed"),
pytest.param({}, id="legacy-without-status-metadata"),
],
)
def test_new_user_turn_keeps_earlier_delegation_that_has_a_result(self, result_metadata):
"""A recorded reply rules out inferring cancellation, even when its legacy status is unknown."""
middleware = DurableContextMiddleware()
runtime = SimpleNamespace(context={"run_id": "run-new"})
messages = [
HumanMessage(content="old request", additional_kwargs={"run_id": "run-old"}),
AIMessage(
content="",
tool_calls=[
{
"name": "task",
"args": {"description": "research auth", "prompt": "do it", "subagent_type": "general-purpose"},
"id": "old-call",
"type": "tool_call",
}
],
),
ToolMessage(content="partial notes", tool_call_id="old-call", name="task", additional_kwargs=result_metadata),
HumanMessage(content="please continue", additional_kwargs={"run_id": "run-new"}),
]
existing = [
{
"id": "old-call",
"run_id": "run-old",
"description": "research auth",
"subagent_type": "general-purpose",
"status": "in_progress",
"created_at": "2026-07-11T00:00:00Z",
}
]
assert middleware.before_model({"messages": messages, "delegations": existing}, runtime) is None
def test_returns_none_when_no_delegations(self):
middleware = DurableContextMiddleware()
@ -568,6 +682,42 @@ def fake_read_file(path: str) -> str:
class TestGraphIntegration:
@pytest.mark.asyncio
async def test_stopped_delegation_is_not_reported_in_progress_to_the_next_turn(self):
"""Stop cancels the run while the task tool waits, so no ToolMessage is ever written for it."""
task_started = asyncio.Event()
@tool("task")
async def slow_task(description: str, prompt: str, subagent_type: str) -> str:
"""Delegate to a subagent."""
task_started.set()
await asyncio.sleep(3600)
return "never"
model = RecordingFakeModel(
responses=[
AIMessage(content="", id="ai-1", tool_calls=[{"name": "task", "id": "call-1", "args": {"description": "research auth", "prompt": "do it", "subagent_type": "general-purpose"}}]),
AIMessage(content="ok", id="ai-2"),
]
)
graph = create_agent(model=model, tools=[slow_task], middleware=[DurableContextMiddleware()], state_schema=ThreadState, checkpointer=InMemorySaver())
config = {"configurable": {"thread_id": "stopped-delegation"}}
run_1 = asyncio.create_task(graph.ainvoke({"messages": [HumanMessage("research auth", additional_kwargs={"run_id": "run-1"})]}, config=config, context={"run_id": "run-1"}))
await asyncio.wait_for(task_started.wait(), timeout=10)
run_1.cancel()
with pytest.raises(asyncio.CancelledError):
await run_1
await graph.ainvoke({"messages": [HumanMessage("please continue", additional_kwargs={"run_id": "run-2"})]}, config=config, context={"run_id": "run-2"})
state = (await graph.aget_state(config)).values
assert [(entry["id"], entry["status"]) for entry in state["delegations"]] == [("call-1", "cancelled")]
ledger_text = "\n".join(str(message.content) for message in model.received[-1] if isinstance(message, HumanMessage))
assert "[cancelled] research auth" in ledger_text
assert "do NOT delegate again" not in ledger_text
def test_subagent_limit_counts_only_prior_delegations_in_real_middleware_chain(self):
model = RecordingFakeModel(
responses=[