* feat(extensions): pass a task's final status, error, goal verdict and run metadata to on_task_stop
A lead run's final status is decided after the graph has ended: a model
that fails after retries ends the graph normally, so callback-based
tracing reports the run as OK. on_task_stop runs after the status is
persisted but only received completed | failed | aborted.
Additive, so existing contributors keep working:
- TaskInfo.metadata: a read-only copy of the run's metadata (scheduled
runs carry scheduled_task_id, scheduled_task_run_id, scheduled_trigger);
empty for subagents.
- TaskInfo.stop: a new TaskStopDetail (status, error, goal_verdict), set
only on the info passed to on_task_stop.
- deerflow-extension-api 0.2.13.
Refs #8303
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(subagents): compare stop identity without the stop detail; assert its status
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(extensions): keep TaskInfo identity and serialization; flag unconfirmed stop status
Review follow-up for #8304:
- Exclude TaskInfo.stop and .metadata from equality and hashing
(compare=False), so the stop info finds state keyed by the start info.
- Replace MappingProxyType with ReadOnlyDict, a dict subclass that rejects
mutation and freezes nested mappings (lists become tuples). asdict(),
deepcopy, pickle and json.dumps work again; observers sharing one
snapshot can no longer change nested values for the next observer.
- TaskStopDetail.persisted: False when the run store did not confirm the
terminal write or the worker lost ownership, so a staged status is not
reported as final.
- Document waiting_for_input as a resumable pause (outcome failed) and the
persisted flag (en/zh observers and reference).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nan Gao <thunderggnn@gmail.com>
* fix(subagents): keep a steered subagent report in the parent snapshot
A Steer frees the parent's turn before its subagent finishes: wait_tool
returns a 'result is pending' task result and the real report arrives
later as a hidden subagent-result HumanMessage.
ParentContextSnapshot only kept genuine user messages, so it dropped that
report and kept the pending placeholder. A later context_mode=snapshot
delegation therefore inherited a pending result and could not use quotes
the parent had already received.
Keep the hidden subagent-result message as historical conversation data.
* fix(subagents): share the delayed result message name
---------
Co-authored-by: 2SinceJuly <202121119+2SinceJuly@users.noreply.github.com>
Co-authored-by: AochenShen99 <sac1063467113@hotmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
read_batch_item trusted list_items(offset=position) to land on position.
That only holds while item rows are dense: OFFSET-based paging skips
surviving rows over any gap (e.g. a future retention delete), so the
reader would serve the next item's report attributed to the requested
item key, and the tool layer would project next_position from the wrong
base. The browser projection (read_browser_batch_item) already verifies
items[0].position; align the model-facing reader with it and add the
same negative-position guard. Reading past a gap now returns not_found
instead of a shifted row, matching the browser path's semantics.
* fix(subagents): apply tool policies to middleware-declared tools
* fix(subagents): address tool-policy review feedback
Normalize declaration conversion errors with middleware context, document
generated discovery exemptions, and add the Unreleased changelog entry.
Audit final explicit and middleware tool sets before graph creation, with
identity-based exemptions for generated discovery helpers and regressions.
* fix(subagents): reuse tool name resolution and clarify policy errors
* docs: turn README into an index and move reference detail to docs/guide
The README had grown to 393 KB because every change appended its notes there. The text moves verbatim into 18 topic guides under docs/guide/; only headings are re-leveled and relative links re-pointed. The README keeps the introduction, a quick start and a guide index. Contributor test notes move to CONTRIBUTING.md.
* docs: send user docs to topic guides and budget the root README
The rule 'update README.md for user-facing changes' is what grew the README. Guidance now points to docs/guide/, and check_agent_guidance.py fails when the root README exceeds 16 KiB (AG003).
* chore: remove unreferenced screenshots and finished plan notes
pr-build/ holds two PR screenshots (3 MB) from #1986. plans/ holds the plan for the merged #4049. docs/SKILL_NAME_CONFLICT_FIX.md is a February change note. Nothing references them.
* docs: route always-loaded AGENTS.md files to scoped guides
* docs(agents): cut scoped guides to scope, verify, traps, decisions
Each AGENTS.md keeps four kinds of content: the scope of the directory, the
verification commands, traps a local test run does not show, and decisions the
code cannot state. Values, field lists, function inventories, test names, issue
numbers and history are deleted, because the code, the tests and git state them.
Search and fetch providers share one community guide. Four guides with no
remaining content are removed. The size check now allows 6 KiB per file (hard
limit 8 KiB) and 24 KiB per chain (hard limit 32 KiB).
* style: format frontend guide table and guidance test imports
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(subagents): keep tests_passed evidence per call when a provider reuses its id
Tool-call ids are unique per response, not per conversation. Some providers
number calls per response, so two turns can both issue call_0.
The bash evidence for tests_passed acceptance assumed one call per id in two
places:
- _harvest_bash_executions built one id -> command map over the whole
conversation, so every result under a reused id was labelled with the
last call's command. A passing subset run (pytest tests/unit) became
evidence for the full suite when a later call_0 ran pytest tests and had
no result yet.
- update_bash_executions merged values chunks by tool_call_id. A dict keeps
a key's first position, so a later run under a reused id overwrote the
earlier entry in place. With call_0 failing, call_1 passing and call_0
failing again, the merged list ended on the stale pass and the leaf held.
Pair results through pair_tool_call_results, which answers a call only from
its own response, and merge chunks by result message id, keeping the call id
for entries without one. Each run stays a separate entry in execution order,
so the latest run decides the leaf again.
* fix(subagents): key the summary cache per run and document id-less calls
- Cache full-output test summaries by result message id (call id only
without one), so reused call ids keep one cache slot per run.
- Update the bash_executions field doc and the subagents guide to the
result-id merge, and state that calls without an id record no
evidence (the leaf stays UNVERIFIED).
- Tests pin both: one classification per run under a reused id, and no
evidence for a call with a missing or empty id.
* chore(subagents): drop the stray empty deerflow/executor.py
The previous commit added a 0-byte module at the package root by mistake.
Nothing imports deerflow.executor; the real module is deerflow/subagents/executor.py.
* fix(subagents): build the chat model off the event loop
`SubagentExecutor._create_agent` is awaited from `_aexecute` on the shared
event loop, and it constructed the chat model inline. Provider construction can
read local credentials -- the Codex provider loads the Codex CLI `auth.json`
while it is built -- so every delegated task did synchronous file IO on the loop
before its first model call, which stalls every concurrent turn sharing that
loop.
Build the model through `run_assembly`, the dedicated assembly pool the other
provider-construction sites already use, and leave the rest of the middleware
and graph assembly where it is. The helper preserves request context, and a
cancelled construction still never starts an inference.
`tests/blocking_io/test_subagent_model_offloop.py` drives the real
`_create_agent` with a real `CodexChatModel` under the strict Blockbuster gate:
on `main` the credential load fails with `BlockingError(os.stat)`, and reverting
the `run_assembly` call reproduces the same failure.
* docs(subagents): name the isolated subagent loop in the off-loop notes
The blocked loop is the persistent isolated subagent loop every concurrent
delegated task shares, not the Gateway request loop; say so in the boundary
audit, the regression docstring, and the assertion message.
---------
Co-authored-by: Totoro <msy626836554@gmail.com>
* feat(subagents): continue native batches and honor item dependencies
* docs: link native batch follow-up changelog entries
* test(migrations): pin isolated history-seed upgrade to 0050
* fix(subagents): preserve follow-up scope, blockers and fair scheduling
* test(memory): freeze capacity evaluation time
* fix(subagents): guard follow-up admission and retain run controls
* fix(subagents): enforce runs:cancel on tool cancellation of follow-up batches
* style(tests): format test_batch_followup_host with ruff
* fix(subagents): drop the admission checkpoint check and keep SDK cancel compatible
The checkpoint check inside run admission waited on the checkpoint saver's
lock while holding the SQLite write lock, which could make another chat's
checkpoint write fail. Remove it; dispatch still checks the conversation
before admission.
A follow-up waiting for an unanswered question now keeps its claim until the
lease expires instead of rereading the conversation every poll. Cancel passes
the caller's run authority only to submitters that take it, so existing SDK
submitters keep the two-argument call. Document that retrying a prerequisite
by hand does not requeue its blocked items, and list 0052 as the migration head.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(extension-api): add task identity fields to CompactionEvent (0.2.10)
CompactionEvent carried the transform, its hashes and its counts, but
nothing that names the task, run or thread the compaction happened in.
An observer that wants to relate a compaction to the run that produced it
had to match hashes across sessions, which is unreliable.
Append three optional fields, task_id, run_id and thread_id (str | None,
default None). Existing keyword and positional construction is unchanged,
so already-released extensions keep working. The host fills them in the
next commit.
Bump deerflow-extension-api to 0.2.10 and the harness's exact pin with it.
* fix(extensions): hand compaction observers the live task store and its identity
The summarization middleware already recovers the live task store from
the runtime to observe its summary system call, and the registry
allocates that store precisely because a compaction observer is
registered. notify_context_compacted() then discarded it and handed every
observer a detached store, with a docstring claiming there was no live
task.
- The lead run worker and the subagent executor now seed every task store
they allocate with the task's TaskInfo, not only when a lifecycle
contributor is registered; the lifecycle notification reuses the seeded
object. A subagent without a parent run_id still gets no TaskInfo.
- compact_state() and acompact_state() pass the store from
task_store_from_runtime() to _record_compaction().
- notify_context_compacted() takes an optional task_store. A live store
goes to the observers and fills the event's task_id/run_id/thread_id
from its TaskInfo; without one (manual /compact runs outside any run)
observers still receive a detached store and the ids stay None.
For a background subagent the store keeps its public scope id (the
provider tool_call_id) while TaskInfo.task_id stays the execution id, as
lifecycle contributors already see it; the event follows TaskInfo.
* docs(extensions): document compaction event identity and seeded TaskInfo
The extensions guide and the developer manual still said compaction
observers always receive a detached store because no task is live at the
call site. Describe what the host does now: lead runs and subagents with a
parent run seed every task store with TaskInfo, compaction observers get
the live store, and CompactionEvent's task_id/run_id/thread_id come from
it; only manual /compact falls back to a detached store with None ids.
Adds the 0.2.10 row to the contract version table (PR number pending).
* docs(changelog): note compaction observers receive the live task identity
* docs(extensions): list compaction observers among task-store triggers
* docs: reference #7831 for the compaction event identity change
* docs(extensions): backfill contract history rows 0.2.5 to 0.2.9
The version table jumped from 0.2.4 to 0.2.10 although 0.2.5-0.2.9
shipped. Add those rows, attributing names merged without a bump to
the first release that carried them (#5796 in 0.2.5, #6920 and #5645
in 0.2.8, #7729 in 0.2.10). The page intro now says it documents
0.2.4 plus the 0.2.10 CompactionEvent fields, and points to the
history for the rest.
* fix(subagents): terminalize a background cancel that beat the loop
request_cancel_background_task() calls Future.cancel() on the future handed
back by _submit_to_isolated_loop_in_context(). That call only wins while the
coroutine has not been picked up yet; when it does win, run_with_timeout never
starts, so no one writes a terminal status and no one opens the cleanup fence.
The registry entry then stayed PENDING forever: task_tool polled its full
budget for a result that could not arrive, and cleanup_background_task()
refused to drop a non-terminal entry.
Publish CANCELLED from the future's own completion callback when the coroutine
body never ran. A cancellation that did reach a running coroutine keeps its
existing owner -- the future reports cancelled() for that case as well, which
is why the callback checks a start marker instead of cancelled() alone.
* fix(subagents): publish a pre-start cancel from the isolated loop
`Future.cancel()` on the future returned by
`_submit_to_isolated_loop_in_context()` fires `forget_future` in the calling
thread, but the loop keeps owning the coroutine until its first step runs. A
cancel that lands in that window read the start marker as clear and published
CANCELLED — opening the cleanup fence — while the worker could still take its
capacity slot.
Hand the decision to the isolated loop and re-check the marker there, which is
the only place serialized against that first step. A loop that is gone cannot
start the coroutine, so the inline fallback keeps the original guarantee.
* test(subagents): accept both terminal owners in the task-created cancel window
Task.__init__ queues the first step inside the task factory, so the step can
run before the deferred pre-start decision that was handed to the loop. When
it does, the coroutine's own CancelledError branch legitimately publishes
"Cancelled by user"; the harness had pinned the pre-start message instead.
Assert the invariants both owners share -- terminal cancelled, fence opened
only after the loop settled the body, cleanup removes the entry -- and let
either error message pass.
* fix(subagents): refuse to start a cancelled execution before taking a slot
A cancel that wins the race against the loop is decided on the loop, one turn
after the submission callback queues the wrapper's first step. That turn is not
a guarantee: the wrapper could still start, create the execution task, and take
a capacity slot after the terminal status and the cleanup fence had already been
published from that decision.
request_cancel_background_task sets cancel_event before it cancels the
future, so the wrapper can treat that event as the startup half of the handshake
and decline to create the execution task. The fence then cannot be outrun by a
worker that acquires capacity afterwards.
* test(subagents): hold the cancel window open with an event barrier
The factory blocked the loop for a fixed 200 ms and then resumed on its own. A
caller delayed longer than that -- a loaded CI worker is enough -- let execution
start before the cancel landed, and `assert not execution_ran.is_set()` failed
even though cancellation was handled correctly.
Block the factory on an event instead and release it from the test's `finally`
after the cancel request and the closed-fence assertions, so the window stays
open exactly as long as the test needs it.
---------
Co-authored-by: gouyu-hou <213296172+gouyu-hou@users.noreply.github.com>
* feat(runtime): add Queue and Steer for active Web runs
* style(frontend): format Queue and Steer controls
* fix(runtime): retain user Steer controls and order active inputs
* docs: add Chinese changelog entry for Queue and Steer
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(subagents): add thinking_enabled and reasoning_effort config for sub-agents (#4875)
(cherry picked from commit b1083728b23504cb0f1d1a463f8db70c5ec1c5a7)
* fix: resolve CI failures for subagent thinking config PR
- Fix UnboundLocalError in task_tool.py: initialize parent_thinking_enabled
and parent_reasoning_effort before the if runtime block
- Remove unused imports from test_subagent_thinking_config.py
(SimpleNamespace, pytest, resolve_subagent_model_name)
- Fix import sorting in subagents_config.py and test file
- Add reasoning_effort: None to executor test assertion
(cherry picked from commit 187851e307b871937e48bb67ae1b34b8b69bf92d)
* fix: address review comments - remove parent inheritance, supports_thinking clamp, Literal type
- Remove parent_thinking_enabled/parent_reasoning_effort from metadata
extraction in task_tool.py (backward compatible: subagents only think
when explicitly configured)
- Add supports_thinking clamp in _create_agent to prevent cross-model
mismatch errors
- Change SubagentConfig.reasoning_effort to Literal type
- Fix sandbox_state assignment that was accidentally removed
- Fix syntax errors in subagents_config.py (missing closing parens)
- Update all docstrings/comments from "inherit" to "no override"
- Simplify test helper and remove parent inheritance tests
(cherry picked from commit 74f9c659ef4a98bd1756109929ab0748fab82317)
* feat(subagents): adapt reasoning settings to current native workers
* fix(subagents): retain model context on reasoning rejection
* fix(subagents): preserve model-factory reasoning diagnostics
---------
Co-authored-by: Neon549 <neon549@users.noreply.github.com>
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* fix(subagents): drain final cleanup across caller cancellation
* fix(subagents): preserve stop outcomes during cancellation
Follow up on PR #7546 discussion_r4374235193 and Issue #7535.
Capture the unwinding cancellation before the drained cleanup tail and use it
only while the result remains non-terminal. Preserve completed, failed and
timed-out decisions. Keep cancellation propagation and cleanup ownership.
Share the synchronous result holder and publish its existing FAILED outcome
before requesting cancellation. This necessary timeout boundary overlaps #7483;
no active-cancel terminal write is added to the cleanup tail.
Validation: 12 targeted regressions pass on Windows and Linux; make test
37876 passed / 808 skipped, make test-blocking-io 292 passed / 1 skipped.
Format, lint, diff checks and four-file incremental OCR delegated review pass.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(subagents): preserve test summaries before evidence truncation
* fix(subagents): honor Node counters and cache harvested summaries
* fix(subagents): retain warning counts and Go case evidence
* fix(subagents): retain long pytest sibling summaries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(subagents): report model_length_capped when a subagent hits the output limit
ModelLengthFinishReasonMiddleware only ran on the lead chain. A subagent
whose response was cut at the provider output limit (finish_reason=length,
stop_reason=max_tokens, MAX_TOKENS, Responses max_output_tokens) reached
the lead as a clean `completed`, and a tool call truncated at that
boundary still executed with partial arguments.
Register the same middleware on the subagent chain, in the lead's
relative position (after configured extensions, before Safety). It now
also records each detected cap per context run_id and exposes
`consume_stop_reason`, so the executor's existing duck-typed guard
collection reports `stop_reason=model_length_capped`. Add the value to
the subagent status contract (Python enum, contract fixture, frontend
whitelist) with the label "model output length limit", and update the
subagent docs.
Tests cover the middleware channel, chain placement, the contract, and
an end-to-end executor run with a fake model.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* test(tasks): cover cap reasons on completed and failed runs
Extend the existing shared-contract round-trip matrix to preserve both terminal statuses alongside every supported stop reason, including model_length_capped and loop_capped.
---------
Co-authored-by: huangzz <110116@nd.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* refactor(skills): keep allowed-tools as compatibility metadata
* docs(skills): align current guides with agent tool authority
* test(skills): simplify authority regressions and sync Korean docs
A `go test -v` run whose every case is skipped exits 0 and prints the same
`ok` package banner a passing run does, so the banner alone supplied positive
evidence for `tests_passed:` while no test executed. unittest (#6916) and Cargo
(#6891) already enforce the non-skipped bar; Go's per-case `--- SKIP` lines are
the only record content that separates the two here.
Non-verbose `go test` output is byte-identical for an all-skipped and a passing
package, so those records stay judged by the banner alone.
* fix(sandbox): append the Exit Code: 124 marker on E2B/BoxLite bash timeouts
When a bash command exceeds its timeout on the E2B or BoxLite providers,
the SDK raises (TimeoutException / TimeoutError) and the generic except
returned a bare "Error: <message>" with no exit marker; a bridge-future
timeout on BoxLite stringifies to an empty message, so the model saw just
"Error: ". Local, AIO, and Tenki all end timed-out output with the
"Exit Code: 124" marker that _truncate_bash_output preserves and the
acceptance-checklist harvest parses to recover the real shell status.
Catch the timeout exceptions ahead of the generic handler in both
providers and return the Tenki-style "Error: command timed out" +
"Exit Code: 124" ending. With #6782 forwarding bash_command_timeout to
every provider, this path is now a first-class configured behavior.
OpenSandbox routes timeouts through the same generic except, but its SDK
is an optional extra whose timeout behavior I could not verify locally,
so it is left out of this change.
Fixes#6847
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* test(boxlite): clarify bridge timeout coverage and SDK failure limits
---------
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(subagents): release batch executions the worker stops supervising
_execute_item cleaned up its execution in a finally block, but the
executor registry only removes terminal entries. When stop() cancelled
an item task, the subagent was still running at that point: the
cancellation request only signals it, so cleanup kept the entry, and
nothing removed it once the run turned cancelled. Each in-flight item
left its SubagentResult, transcript included, in the process-wide
registry for every SDK SubagentRuntime start/stop cycle.
A repository error while polling took the same exit without even
requesting cancellation, so the item was failed or requeued while its
subagent kept running unsupervised, and a retry could run it twice.
On either exit, cancel a run that is not terminal yet and remove its
entry from the persistent subagent loop once it stops. Cancellation is
cooperative, so the wait is bounded by the item's lease, after which
another worker owns the item and the entry is removed regardless.
* docs(changelog): reference #6785
* fix(subagents): log a scheduled batch release that fails
A concurrent future never reports an exception nobody retrieved, so a
release that raised on the subagent loop leaked its registry entry with
no log trace. Log it at error level with the execution id, as the task
tool's deferred cleanup already does.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(subagents): leave full results out of the polled batch item list
The item list endpoint returns only previews, but list_items selected
whole rows, so every page loaded and decoded each item's full report,
evidence JSON and prompt before discarding them. The workspace polls
this list every 2 seconds while a batch runs; with the default
max_result_chars a 100-item page held about 34 MB and blocked the event
loop for about 70 ms (170 MB and 300 ms at limit=500).
Defer those columns unless the caller asks for them, with raiseload so
an accidental read fails loudly instead of lazy-loading. A 100-item page
now peaks under 1 MB; responses are unchanged.
* test(subagents): pin the item list route to compact columns
Drive list_batch_items against a real SQLite repository so a route that
starts requesting full results fails, not only a repository change.
* docs(changelog): reference #6781
* test(subagents): scope the route test's seed update to its batch
* fix(agents): skip content blocks with unhashable type values
has_visible_content and ParentContextSnapshot.from_state classified a
content block's type with set membership (`in {"text", "output_text"}`
plus the _MEDIA_BLOCK_TYPES frozenset). `type: []` / `type: {}` are
legal JSON from a provider, and hashing an unhashable value raises
TypeError instead of skipping the block:
- has_visible_content runs on every model response through the
llm-error-handling (_raise_for_empty_response), terminal-response and
model-length middlewares, so one malformed block turned a completed
response into a failed run instead of a skipped block;
- from_state builds the history every background subagent dispatch is
seeded from, so the same block failed the dispatch.
Probe the type by equality with tuples so a non-string type simply fails
the probe, mirroring #6614's fix for extract_response_text and the
isinstance guard clarification already uses for form field types. The
media constant becomes a tuple too, so the second probe in the same loop
cannot move the crash one line down.
Tests pin the RED-to-GREEN cycle at both call sites: unit-level
has_visible_content, middleware-level empty-response classification,
and the dispatch snapshot across human/ai/tool messages.
Fixes#6663
Signed-off-by: yetuge <2219677952@qq.com>
* fix(agents): warn when a snapshot skips a non-string-typed block
Review follow-up on #6664: with the malformed block now skipped
instead of raising, a provider that systematically emits non-string
types would degrade the child's history with nothing in the logs to
explain it. The elif chain cannot log on fall-through —
reasoning/signature and tool-use exclusions are deliberate and must
stay quiet — so the signal keys on the type not being a str, matching
the guard the review suggested.
has_visible_content keeps its silence on purpose: the empty-response
retry already surfaces the symptom there.
Fixes#6663
Signed-off-by: yetuge <2219677952@qq.com>
---------
Signed-off-by: yetuge <2219677952@qq.com>