202 Commits

Author SHA1 Message Date
Hyeonsang Cho
c540399a5e
fix(gateway): scope per-run reads to the run's data identity (#6282)
* fix(gateway): scope per-run message and event reads to the run's data identity

start_run stamps run-event rows with the raw trusted owner id, but
GET /runs/{rid}/messages and /runs/{rid}/events called the event store with
user_id=AUTO, which resolves to the internal caller's make_safe_user_id form.
On the database event store, owners whose id contains characters outside
[A-Za-z0-9_-] (IM channel ids such as "feishu:owner-777") therefore got an
empty list. The regenerate source-run scan had the same mismatch and fell
back or returned 409.

Pass _run_scope_user_id() explicitly, as the thread message routes already
do (#5448), and add user_id to RunEventStore.list_messages_by_run() so every
backend accepts it like list_messages()/list_events(). Browser and API
sessions keep their per-user filter.

* fix(gateway): scope workspace-changes and archive reads to the run's data identity

The same raw-owner mismatch reached two more run-scoped routes. On the SQL
stores, GET /runs/{rid}/workspace-changes reported no changes because its
event lookup used user_id=AUTO, and GET /runs/{rid}/artifacts/archive
answered 404 because both its run-row lookup and its run.delivery event
lookup did.

Pass _run_scope_user_id() to both, and let get_workspace_changes_response()
forward an explicit user_id to the event store.

* docs(changelog): reference #6282 in the run-scope read fix entries

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 10:23:55 +08:00
Hyeonsang Cho
56aa71c583
fix(runtime): keep renewing the lease while a staged terminal run finalizes (#6263)
* fix(runtime): keep renewing the lease while a staged terminal run finalizes

With an event store, run_agent stages the terminal status in memory and
commits it to the run store only after the journal flush, delivery receipt,
workspace scan and duration checkpoint. _renew_leases selected runs by local
status, so it stopped renewing as soon as the status was staged. A
finalization longer than lease_seconds + grace_seconds let a peer, or this
worker's own reconciler, claim the still-active row and record the
successful run as an orphan error.

The worker now marks RunRecord.terminal_commit_pending until the deferred
commit is attempted, and lease renewal treats it like the existing
scheduled-goal barrier: keep renewing, release on a confirmed own commit,
and fence when a peer claims the row. A fenced run stops renewing.

* docs(changelog): reference #6263 for the staged terminal lease renewal fix

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 07:07:04 +08:00
subhash7476
20bd989fc1
feat(runtime): add per-LLM-call telemetry and summarization no-op event (#6236)
* feat(runtime): add P0 per-LLM-call telemetry and summarization no-op counter

Measurement only; no runtime behavior changes.

- RunJournal enriches llm.ai.response / llm.error metadata with
  langchain_run_id, caller_category, provider, model, stop_reason, status,
  token counts and rendered request size (chars). Unavailable values are None.
- Summarization middleware emits a middleware:summarize event per summarizer
  call and counts outputs identical to the prior summary_text. The call is
  never skipped.
- All telemetry fails soft.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(runtime): register summarize middleware tag, document telemetry contract

- Add MIDDLEWARE_SUMMARIZE_TAG to the middleware event catalog and the
  contract's known_tags; emit the middleware:summarize event with it.
- Document the optional per-call telemetry metadata on llm.ai.response and
  llm.error in contracts/run_event_stream_contract.json.
- Measure request size by walking string leaves instead of json.dumps on the
  event-loop thread.
- Initialise summarization call/no-op counters in __init__.
- Move RunJournal telemetry docs to the runtime guide.
- Add tests: llm.error without a start, replayed on_llm_end, and a DB-store
  metadata round trip.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(runtime): refresh token counters on usage replay

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-04 14:52:04 +08:00
Totoro
35023bb908
feat(scheduler): create conversation schedules and support agent self-stop (#6229)
* feat(scheduler): create conversation schedules and support agent self-stop

* fix(scheduler): validate goals and trial requests before dispatch

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
2026-10-04 06:46:13 +08:00
Wenchao An
beaae22088
feat(extensions): add host-bound Agent run control (#6190)
* feat(extensions): add host-bound Agent run control

* fix(extensions): preserve internal launches and scope run control

* docs(extensions): use a plugin-local run operation key
2026-10-03 21:16:01 +08:00
Inference_
a8dc1da205
fix(runtime): honor both cursors when reading thread messages (#6136)
* fix(runtime): honor both cursors when reading thread messages

* docs(runtime): clarify run-scoped combined cursor bounds
2026-10-01 20:11:10 +08:00
Wenchao An
4c05e88c52
feat: add inline @ references with multiple skill activation (#6063)
* feat(frontend): unify composer references with an @ picker

* feat: edit references inline and activate multiple skills

* docs: describe inline references and multiple skill selection

* fix: prevent sent drafts from returning and repair CI checks

* fix: reconcile inline references and address review feedback

* fix: preserve references through discovery errors and polish
2026-09-30 23:09:05 +08:00
hataa
1cd3eb742b
feat(authz): enforce skill authorization at assembly and slash-activation (#4063 Phase 3) (#4541)
* feat(authz): enforce skill authorization at assembly and slash-activation (#4063 Phase 3)

Phase 3 / Skills — the second of three resource-type PRs (Models, Skills,
Sandbox). The RBAC provider already maps "skill" → config key "skills"
(rbac.py _RESOURCE_POLICY_KEYS), so no schema change is needed.

Layer 1 (assembly-time, mirrors Phase 1B):
- filter_available_skills_by_authorization() in skill_filter.py filters the
  skill-name allowlist (set[str] | None) by the provider's "skill" policy.
  In lead agent, runs after _available_skill_names() so the catalog
  (describe_skill) and SkillActivationMiddleware share one filtered set.
  In subagent executor _load_skills(), filters config.skills before disk load.
- filter_resources_by_authorization() in enforcement.py — generic version of
  filter_tools_by_authorization for any resource with a name attribute.

Layer 2 (slash-activation):
- No new middleware code needed. SkillActivationMiddleware._resolve_activation
  already checks "reference.name not in self._available_skills". Since Layer 1
  filters the allowlist, denied skills cannot be slash-activated.

authorization.enabled: false is a complete no-op. available_skills=None (no
agent allowlist) + enabled: resolves all skill names from config and filters.
12 new tests + 243 existing authz/skills tests pass.

* test(subagents): wait on the chained future instead of polling done() in caller-loop teardown test

* feat(authz): enforce action-scoped skill:activate at runtime and reuse the catalog loader for filter candidates

* feat(authz): arm skill:activate on the subagent chain, unify the client skill-surface user, and record the Layer 2 reversal in the design notes

* chore(authz): align the subagent filter's fallback user with the chain's bucket convention

* feat(authz): gate autonomous skill loading on skill:activate, close the bootstrap assembly leak, and reuse one provider per subagent assembly

Review round 4 (willem-bd, CHANGES_REQUESTED):

- [P1] Autonomous load path: describe_skill filters its match set on the
  action-scoped skill:activate decision (all-denied is indistinguishable
  from "no match"), and ToolErrorHandlingMiddleware stamps skill_context
  entries only when the action allows it — a denied SKILL.md read gets a
  skill_context_denied marker instead, so durable context, allowed-tools
  policy, and autonomous secret bindings never activate a denied skill.
  File visibility itself stays Layer 1 (sandbox-projection semantics).
- [P1] Bootstrap assembly: build_middlewares/apply_prompt_template now
  receive the authorization-filtered set instead of the raw
  _BOOTSTRAP_SKILL_NAMES, and an empty skill_names index is preserved
  when deferred discovery is on rather than round-tripping through None
  into the legacy full-metadata prompt that re-advertises a denied
  skill (deferred-off keeps the legacy rendering; the filtered empty
  allowlist blanks it on denial).
- [P2] SubagentExecutor resolves ResolvedSkillAuthorization once per
  assembly (_resolve_skill_authorization, memoized per executor) and
  shares it across the Layer 1 filter (authorization=), the activation
  middleware, and both new gates — uncached provider factories can no
  longer hand the layers different instances/policy snapshots.

All gates share one helper (skill_filter.skill_activation_allowed) so
slash, describe, and skill-file-load cannot drift; provider errors keep
the configured fail-closed/fail-open policy. skill_authorization is
threaded through _build_runtime_middlewares into both runtime builders.

Tests: describe gating (deny/fail-closed/fail-open/disabled), read
stamping (denied marker vs entry) + extract_skills skip without warning,
bootstrap three-way param (allow/deny x deferred on/off), lead + subagent
chain wiring for the stamp gate, and the distinct-instance provider
factory repro. Each fix is mutation-verified (reverting it fails its
test). Full A/B suite run: no new failures vs HEAD.

* feat(authz): re-authorize persisted skill_context before reuse and route async paths through aauthorize

Review round 5 (willem-bd):

- [P1] Persisted entries: skill_context survives across runs, so both
  consumers now re-check the action-scoped skill:activate decision before
  reuse. SkillToolPolicyMiddleware._active_skills_for_paths skips a
  denied skill (same skip as a disabled/unallowlisted one; all-denied
  keeps the existing fail-closed-to-builtins semantics) and
  _in_context_secret_sources binds no secrets for a denied entry.
  skill_authorization is threaded into SkillToolPolicyMiddleware on both
  chains.
- [P2] Async provider API: skill_activation_allowed_async mirrors the sync
  helper through await provider.aauthorize() (same pattern as
  authorize_sandbox_execution_async). Async hooks precompute an
  activation_decisions map on the event loop (slash target + persisted
  entry names, one batch of awaits) and hand it to the worker-thread
  handler, which consults the map instead of the sync API:
  SkillActivationMiddleware.awrap_model_call (activation + secret
  binding), ToolErrorHandlingMiddleware.awrap_tool_call (_amaybe_stamp
  awaits the decision before stamping), and SkillToolPolicyMiddleware's
  awrap_model_call/awrap_tool_call (entry names + slash source name).
  describe_skill becomes a StructuredTool with both a sync function
  (invoke -> authorize) and a coroutine (ainvoke -> aauthorize), so async
  agent execution never routes a loop-affine provider through the sync
  API while sync callers keep their semantics.

Tests: allow-read-then-deny-next-run regressions for secret binding and
allowed-tools (partial deny does not poison, all-denied fails closed to
builtins), and an _AsyncOnlyProvider (sync authorize always fails,
aauthorize works) driving four async-path assertions (activation, stamp
allow/deny, describe, tool policy). All six fixes are mutation-verified;
full A/B suite run against HEAD shows no new failures.

* fix(authz): authorize skill activation by declared Skill.name, not directory basename

ShenAC-SAC's P2: bundled skills may declare a name that differs from their
directory (skills/public/vercel-deploy-claimable declares name: vercel-deploy),
but the runtime gates derived authorization targets from the container path.

- New deerflow/skills/container_registry.py: build_container_path_registry()
  + canonical_skill_name() — the one path-to-Skill.name resolution shared by
  the activation, tool-policy, and read-stamping middlewares.
- ToolErrorHandlingMiddleware resolves the read path through the (user-scoped,
  via new user_id kwarg threaded from both runtime builders) registry before
  the skill:activate decision, sync and async (registry lookup in a worker
  thread, aauthorize() on the loop). RBAC allow by the declared name now
  activates; allow by the directory name does not.
- SkillToolPolicyMiddleware async hooks canonicalize the policy paths first
  (three-phase: thread registry -> loop aauthorize -> thread filter reusing
  the registry; the cached per-model-step decision still short-circuits),
  so the decision map is keyed by the names _active_skills_for_paths checks —
  no more map miss falling back to the sync provider API from the thread.
  _entry_names removed: persisted entry["name"] is path-derived by
  construction; entries are resolved by path instead.
- SkillActivationMiddleware collects slash names (already canonical) plus
  persisted entry *paths*, canonicalized off-loop via
  _canonical_names_for_paths().

Regressions (test_skills_authorization.py 45->50): sync stamp target, RBAC
allow declared-name/deny directory-name both directions, async stamp via
_AsyncOnlyProvider, composed async slash -> tool-policy keeps the skill's
tools with zero sync fallback, persisted-entry secret binding survives the
name mismatch. All three mutations (stamp target, policy map key, entry
canonicalization) fail their tests.

* fix(authz): close the sync-fallback class on skill activation paths

willem-bd's two P2s on the canonical-name fix, plus a fourth instance of
the same class found while sweeping the membership:

- SkillToolPolicyMiddleware: a transient prepass registry failure is now
  preserved (_REGISTRY_LOAD_FAILED marker) — the worker-side filter applies
  its own fail-closed treatment instead of silently retrying storage,
  which recovered skills absent from the decision map and fell back to
  the synchronous provider API from the thread (fail-open retaining a
  denied skill's allowed-tools).
- SkillActivationMiddleware: an empty entry-path list no longer pays a
  full skill-tree scan on ordinary authorization-enabled async model
  steps (previously a passive path with zero registry I/O).
- 4th instance (reproduced with the PR fixtures, then fixed): the entry
  prepass failure on the secret-binding path let the consumer-side
  _load_skill_registry_by_path silently recover and resolve entry names
  outside the decision map — the same sync fallback, fail-open binding
  the aauthorize-denied skill's declared secret. The fix goes one step
  further than failure propagation: the prepass hands down the registry
  *snapshot* the decision map was keyed by, and entry sources resolve
  against that snapshot, so a map miss is impossible by construction
  (this also closes the narrower window where both loads succeed but
  disagree because storage changed mid-step). The slash source is
  deliberately unaffected: it does not consult the decision map
  (validated once at activation) and keeps its fresh per-call load.
- Class membership swept: skill_activation_allowed() sync call sites are
  exactly four (stamping sync path and describe's sync implementation are
  legitimately sync); the only two async-reachable fallback points (the
  _activation_allowed helpers) are both construction-guarded now.
  Structural hardening (typed async-batch decision object where a map
  miss is fail-closed by construction) recorded in the plan notes as a
  follow-up, anchored at those two helpers.
- _RegistryArg type alias replaces the degenerate dict|object|None union
  in the policy middleware signatures.

Tests (test_skills_authorization.py 50->54): transient prepass failure ->
builtins-only with exactly one storage attempt and zero provider calls;
ordinary async step -> zero registry scans; two-call scenario -> entry
binding zeroed while the slash binding survives; entries resolve against
the prepass snapshot under a mid-step rename. Mutations verified red:
marker reverted to None, over-broad slash kill, snapshot bypass.

* fix(authz): correct secret-binding reference points and gate the rendered skill reminder

Review round 9 (willem-bd) plus construction-audit follow-ups on the same
class — every async divergence between two loads of the same data now has
an explicit, pinned reference point:

- Slash secret source resolves from a FRESH post-activation registry: the
  prepass snapshot is taken before the aauthorize awaits, so binding the
  slash source from it would activate NEW_KEY content while injecting
  OLD_KEY when a skill's declared secrets change in that window. The
  'one scan per step' bonus that motivated sharing the snapshot is
  withdrawn — consistency wins. The two secret sources resolve
  independently: a transient fresh-load failure zeroes only the slash
  binding; entries keep binding from their own snapshot.
- Same-skill dominance: when a skill is both the slash source (fresh) and
  a persisted entry (snapshot), only the activation-era declaration binds
  — unioning both eras injected a secret the skill no longer declares.
  Aligns with the tool-policy rule (slash activation dominates for the
  run); different-skill entries still stack additively.
- The rendered 'Active skills' reminder (DurableContextMiddleware) no
  longer advertises skills the provider denies — surface consistency with
  describe_skill. Publication/consumption split: the activation
  middleware publishes per-step path-keyed skill:activate decisions into
  the run context (async: reused from the prepass snapshot, zero extra
  I/O; sync: one scan + authorize() only when entries exist — passive
  steps stay scan-free), the renderer consumes them and filters its
  render copy (state untouched; unpublished = permissive, matching the
  other run-context carriers). New carrier key listed in
  REDACTED_CONTEXT_KEYS with a redaction regression.
- Wiring regressions on both chains for everything added above (user_id
  to the stamp gate, skill_authorization + token pairing + activation-
  before-durable ordering to the durable renderer): each mutation
  verified red (M4-M13 across this change set).
- Tests 54->65: slash post-activation lookup, source independence,
  dominance, sync-chain sync-API path, render filter (async/sync/
  disabled with load-count assertions), chain wiring x2, redaction
  listing. Plan notes record rounds 9-2..6 including the reference-point
  post-mortem and the audit axes exhausted.

* fix(authz): anchor same-skill dominance on the slash identity, not binding results

Review round 10 (willem-bd) plus a self-audit extension the new checklist
items caught in the first fix itself:

- R10 (P2): the same-skill exclusion was derived from successfully BOUND
  slash sources. When the prepass snapshots required-secrets [OLD_KEY]
  and the operator strips the declaration during the await window, the
  slash source binds nothing (required_secrets gate) — the exclusion set
  stayed empty and the stale entry view kept injecting OLD_KEY for a
  skill that now declares no secrets. The exclusion now derives from the
  AUTHENTICATED slash activation identity (run-context source path,
  owner-token checked): the identity's declared name resolves from the
  fresh registry first, then the entry snapshot, and joins the exclusion
  set regardless of binding success.
- Self-audit (value->gate->downstream on the fix itself): the declared
  name can also change mid-run (same path, foo->bar). A name-anchored
  exclusion then holds only the new name while the snapshot entry
  resolves the same path to the old one — both eras bound again. The
  PATH is the stable identity shared by the activation and every earlier
  read: entries at the slash identity's path are now excluded before
  registry resolution (exclude_paths), independent of what either
  version calls the skill; the name-based set remains for cross-path
  same-name shadowing.

Tests 65->67: no-secrets-era dominance (ACTIVE secrets None under the
OLD_KEY-to-empty interleaving), path-anchored dominance across a mid-run
rename (only the activation-era key binds). Mutations verified red:
identity-derived exclusion disabled (M14), path exclusion disabled
(M16).

* test(authz): pass explicit config to lead middleware wiring test

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-30 19:45:07 +08:00
Totoro
295437692a
fix(agent,goal): stop coaching scripts when no bash tool is bound, and show the goal evaluator Human Input Card answers (#6082)
* fix(agent): drop script guidance from the lead prompt when no bash tool is bound

The workspace section told the agent how to write scripts and commands
("prefer relative paths", "avoid hardcoding /mnt/user-data/... inside
generated scripts") even when no tool could run them. With the default
LocalSandboxProvider host bash is off and the bash tool is removed, so
agents wrote helper scripts nothing could execute, then stalled or
computed by hand.

apply_prompt_template() takes bash_available (default True, so other
callers keep today's text). The lead agent (both assembly branches) and
the embedded client pass whether their authorized tools, bound or
deferred, include bash. Without it:
- the two script bullets become one line: no bash tool is bound, work
  out results directly and write them with write_file;
- the subagent section no longer shows direct bash examples, and marks
  the bash subagent unavailable, since it inherits the lead's tool groups;
- the ACP hint no longer suggests `bash cp`.

* fix(goal): show the goal evaluator the user's answer to a Human Input Card

The web UI sends a Human Input Card answer as a hidden human message
(hide_from_ui with a human_input_response) and shows it on the card.
format_visible_conversation() skipped every hidden message, so the
evaluator saw the question but never the answer. Following the user's
answer then read as the assistant guessing, and the goal stood down with
needs_user_input although the user had answered.

A hidden human message whose human_input_response passes
read_human_input_response() now adds a line
"User (Human Input Card answer): <value>" from the structured value; the
question is already on the card's own line. Other hidden messages, such
as goal continuations, stay out. Over the 12,000-char cap the latest
answer the cut drops is kept at the top next to the latest request, and
is never taken for the request itself. Without card answers the cut is
unchanged.

* fix(agent): keep the subagent examples and bash hint in line with the bound tools

Review follow-up for the lead prompt without bash.

- The last delegation example follows the tools. With a bash subagent on
  offer it is unchanged. A lead with bash and no bash subagent keeps
  "Run a routine test, build, or git command directly." without the Bash
  subagent sentence. A lead without bash gets a file-work example instead.
- The unavailable-bash description no longer points at the sandbox
  provider. Since #2253 it renders only when the bash subagent is
  registered and the lead binds no bash, where the sandbox is not the cause.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
2026-09-30 17:54:42 +08:00
Totoro
f75d89f39d
fix(goal): record a failed evaluator call as a stand-down (#6073)
* fix(goal): record a failed evaluator call as a stand-down

When the goal evaluator call raised (a provider error, or an answer that
was not valid JSON), _prepare_goal_continuation_input() logged a warning
and returned without writing anything. The goal stayed active with no
last_evaluation, or with an earlier run's verdict and run id, unlike the
other paths that stop the goal loop.

Record it like the no_durable_end_of_turn stand-down: blocker run_failed,
stand_down_reason evaluator_failed, the current run id, and a reason that
names only the exception type. Nothing is recorded if the run was aborted
meanwhile. A failure before the evaluator is called keeps its behavior and
now logs "Could not prepare goal evaluation".

* docs(gateway): fit the evaluator-failure note in the AGENTS.md budget

Main grew backend/app/gateway/AGENTS.md to 49,006 bytes, and the longer
note pushed the merged file past the 49,152-byte hard limit that
tests/test_agent_guidance_check.py enforces. Say it in one clause.

* docs(gateway): keep main's AGENTS.md line after the merge

The merge from main kept this branch's version of the goal paragraph,
which restored the Human Input Card wording #6068 replaced and put the
file at 49,194 bytes, over the 49,152-byte limit the agent-guidance
check enforces. Take main's line. It already says every non-satisfied
evaluation is persisted with last_evaluation, which this fix makes true
for a failed evaluator call.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-30 11:35:19 +08:00
Totoro
d73844b997
fix(goal): show the goal evaluator the tool calls and results (#6068)
* fix(goal): show the goal evaluator the tool calls and results

The evaluator read only the text of user and assistant messages and was told
to fail closed without explicit evidence, so it stood down with
missing_evidence on most completed file tasks: the tool calls and tool
results that the web UI shows never reached it.

Include the assistant's tool calls and the tools' results inside the
existing message window, shortened (200 characters per argument value, 600
per step, line breaks kept), and tell the evaluator to treat tool results as
data, that a successful tool call alone does not meet a goal, and that a
guessed input means needs_user_input. The no-progress and final-clear
signatures are unchanged.

* fix(goal): tighten the evaluator's tool evidence after review

- Pair each tool result with the latest call of its id before it. Some
  providers reuse ids such as call_0 across turns, and the window-wide
  map labelled an earlier read with a later write's name.
- Keep model-supplied tool names on one line in calls and result labels,
  and escape U+0085, U+2028 and U+2029 in arguments and results, which
  json.dumps(ensure_ascii=False) leaves raw, so no tool step can start a
  new evidence line.
- Leave out the results of a hidden assistant message's calls. The web UI
  renders a result only under its call, so neither shows there either.
  A clarification prompt stays: the UI shows it as its own card.
- Over the 12,000-char cap, keep whole lines from the end instead of
  slicing mid-line. Keep the latest user message that the cut drops at
  the top, whole when it and the lines after it fit, else shortened to
  2,000 chars. If the next assistant message back does not fit whole,
  keep its end in the room left. A marker counts the lines left out.
- Docs: the Gateway AGENTS.md and a worker docstring no longer say the
  evaluator never sees tool messages.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
2026-09-30 10:36:09 +08:00
Hyeonsang Cho
777961f84e
fix(events): read a non-filename run ID as an unknown run on JSONL (#6070)
* fix(events): read a non-filename run ID as an unknown run on JSONL

The run-scoped routes (events, messages, workspace-changes) pass the URL's
run_id straight to the event store. With run_events.backend: jsonl, an ID
that cannot be a filename (run.1, a%20b, %2e%2e) raised ValueError and
surfaced as a 500, while the memory and database stores return an empty
result.

JSONL reads and deletes now treat such an ID as an unknown run through one
helper, _existing_run_file(), which returns before building any path.
Writes still reject it.

* docs(changelog): reference #6070 for the JSONL run ID fix

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-30 10:07:58 +08:00
Hyeonsang Cho
b37f769b67
fix(persistence): reject SQLite file: URIs instead of opening them as filenames (#6069)
* fix(persistence): reject SQLite file: URIs instead of opening them as filenames

resolve_sqlite_conn_str passed a file: connection string through
unchanged, but LangGraph's SQLite checkpointer and Store factories
(SqliteSaver, SqliteStore, AsyncSqliteSaver, AsyncSqliteStore) connect
without uri=True, so SQLite treated the whole URI as a literal filename:
file:checkpoints.db?mode=rwc created a file with that exact name in the
working directory, file::memory:?cache=shared persisted to disk, and a
file:/// URI failed to open. The readiness probe did open URIs with
uri=True, so it checked a different file than the runtime used and
reported in-memory URIs as not_configured.

Reject file: URIs in resolve_sqlite_conn_str, the one helper all four
factories go through, with an error that names
checkpointer.connection_string. The probe drops its URI parsing and
reports a rejected connection string as unreachable.

* docs(changelog): reference #6069

* test(persistence): pin the case-sensitive file: URI check
2026-09-30 09:40:02 +08:00
Totoro
e6a3f83f2f
fix(goal): preserve active goals when artifact delivery fails (#5980)
* fix(goal): preserve active goals when artifact delivery fails

* chore(goal): explain completion guards and log skipped clears

* docs(goal): clarify required artifact delivery

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
2026-09-28 14:44:45 +08:00
NanPan
67ffbfdbb5
fix(runtime): drain subagent event flush on cancellation (#5950)
* fix(runtime): drain subagent event flush on cancellation

* test(runtime): pin subagent event flush cancellation ownership

* docs(subagents): pin event flush cancellation ownership

* docs: trim subagent guidance to fit inherited budget

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-27 21:04:39 +08:00
xbzz1018
e2c5027b0c
fix(tracing): tag graph runs with the effective model (#5902) 2026-09-27 08:06:03 +08:00
Yeager0204
b433e3c6a0
fix(runtime): record textless human input only once per run (#5903)
Co-authored-by: kwsYegar <331533331+kwsYegar@users.noreply.github.com>
2026-09-27 07:41:51 +08:00
NanPan
79a5af071a
fix(events): retain JSONL snapshot ownership across cancellation (#5795)
* fix(events): retain JSONL snapshot lock through cancellation

* test(events): cover cancelled JSONL snapshot ownership

* docs(events): record JSONL snapshot ownership invariant

* test(events): pin snapshot cancellation failure semantics

* docs(runtime): name drained JSONL snapshot tasks

* test(events): pin snapshot writer exclusion on disk

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-26 17:59:31 +08:00
RongJie G
842f561054
fix(runtime): drain agent stream close across cancellation (#5806)
* fix(runtime): drain agent stream close across cancellation

* docs(runtime): condense stream teardown guidance

* fix(runtime): surface stream close failures on cancellation

---------

Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-25 12:43:04 +08:00
dong
3d5cc1efae
fix(runtime): persist consumed middleware tool results (#5671)
* fix(runtime): persist consumed middleware tool results

* docs(agents): shorten journal reconciliation guidance

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-25 10:06:31 +08:00
Ryker_Feng
6d961e00e3
feat(skills): show loaded skill snapshots in answer details (#5758)
* feat(skills): show loaded skill snapshots in answer details

* docs: bring agent guidance under size budgets

* fix: align skill usage snapshots with visible tool output

* fix: validate skill usage producers and clarification run boundaries
2026-09-24 17:49:52 +08:00
yetuge
6b3d64cccb
fix(runtime): drain postgres schema-setup connection exit across cancellation (#5812)
_ensure_postgres_schema_with_pool runs inside the drained pool context but
acquired its connection with a raw 'async with pool.connection()'. Caller
cancellation arriving while the connection's __aexit__ is blocked interrupts
that exit, so the connection is never returned to the pool and the pool's own
drained teardown closes with a checked-out connection — the backend-ownership
invariant documented in runtime/AGENTS.md (same class as the provider-context
fix that introduced drained_async_context).

Use drained_async_context for the schema-setup connection so the connection
returns to the pool before the pool context unwinds.
2026-09-24 17:34:06 +08:00
NanPan
f6269fb6b1
fix(runtime): drain DB provider context exits across cancellation (#5757)
* fix(runtime): drain DB provider context exits

* test(runtime): pin drained async context semantics

* style(runtime): format drained context tests

* fix(runtime): preserve drained context exit failures

* style(runtime): format exit failure regression
2026-09-24 08:11:47 +08:00
Zeren Wang
31bfd40f38
chore(deps): bump langgraph-checkpoint to 4.2.0 / postgres 3.1.2 and delete the saver patch (#5734)
* chore(checkpoint): bump to checkpoint 4.2.0 and drop the stale saver patch

langgraph-checkpoint 4.2.0 fixes the write dropped after a full -> delta
migration (upstream langchain-ai/langgraph#8526) and keeps its own
InMemorySaver override, so the compatibility patch in checkpoint_patches.py
could neither detect the fix nor stand down: it read langgraph's version
while InMemorySaver ships in langgraph-checkpoint, and it would have kept
shadowing the upstream implementation on 4.2.0. The postgres release 3.1.2
locates plain-value delta seeds (upstream #8535).

- floor langgraph-checkpoint>=4.2.0,<5.0 in the harness and raise the postgres
  extra to >=3.1.2,<3.2; langgraph's own >=4.1.0 would otherwise let a
  resolver reintroduce the bug now that the patch is gone
- delete the patch, its version guard, and the three implementation-assertion
  tests; test_full_to_delta_migration_replays_on_same_thread stays as the gate
  and fails on 4.1.1 without the patch (verified by a one-off unpatch run)
- langgraph stays 1.2.9 and langgraph-checkpoint-sqlite stays 3.1.1

Verification: make lint; delta checkpointer/cache/worker-resume suites green
with the migration gate passing on memory and sqlite (postgres params skip
without TEST_POSTGRES_URI). The full make test run before the packaging-pin
fix below was 13 failed / 18523 passed; 12 of those failures reproduce on main
with the same environment (git stash baseline, identical failure sets), 7 of
them because the local config.yaml runs checkpoint_channel_mode: delta. The
remaining one, the postgres-extra string pin in tests/test_checkpointer.py, is
updated here.

Upstream status for the surrounding gaps: cache parity across off/cold/warm
states, postgres pagination (langchain-ai/langgraph#8448) and parallel-superstep
replay order (#8382) are still open upstream, as is the abandoned-branch fork
fix (#8548) that the delta resume linearization in runtime/runs/worker.py works
around, so that path stays.

* docs(changelog): keep checkpoint upgrade in Unreleased

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-23 09:18:26 +08:00
RongJie G
e01314442c
fix(mcp): scope sessions and task access by thread incarnation (#5556)
* fix(mcp): scope sessions and task access by thread incarnation

* fix(mcp): preserve thread incarnation in delegated subagents

* fix(mcp): preserve incarnation in durable batches

* fix(mcp): bind standalone graph lifecycle context

* fix(studio): preserve implicit thread creation metadata

---------

Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-23 08:33:20 +08:00
lau0708
ef3c1c2aee
fix(runtime): share one change position across an atomic thread operation (#5663)
* fix(runtime): share one change position across an atomic thread operation

MemoryRunStore.create_thread_operation_atomic marked each row it touched
separately, so one interrupt-and-replace consumed two positions in the
(change_seq, run_id) cursor that list_changed consumers page with. The SQL
store allocates a single position for the whole set, and runtime/AGENTS.md
documents that as the contract for an atomic thread operation.

The memory store is the backend an install runs whenever no durable database
is configured, so a cursor reader could observe the interrupted row and its
replacement under two positions instead of one, ordered by run_id.

Allocate one position after the raise-only scan -- so a rejected operation
still consumes nothing -- and reuse it for the claimed rows and the new row.

* test(runtime): pin that a rejected thread operation consumes no position

create_thread_operation_atomic allocates its change position only after the
raise-only candidate scan, so a ConflictError consumes nothing -- the memory
store's counterpart of the SQL store rolling back and leaving its clock
untouched. Nothing asserted that ordering.

A consumed-but-unused position leaves no trace in the rows themselves, so a
test comparing list_changed output passes even with the allocation hoisted
above the scan. Assert instead on the position the next accepted operation
lands on: that surfaces a hoist as a gap.
2026-09-22 11:38:43 +08:00
NanPan
e88599bb29
fix(runtime): drain sync checkpoint mutations across cancellation (#5664)
* fix(runtime): drain sync goal checkpoint commits across cancellation

* fix(runtime): drain sync rollback checkpointer mutations

* test(runtime): cover sync rollback mutation cancellation

* docs(runtime): generalize sync checkpoint mutation contract
2026-09-22 11:02:16 +08:00
NanPan
656db1223d
fix(runtime): drain provider close across cancellation (#5622)
* fix(runtime): drain provider close across cancellation

* test(runtime): cover redis provider teardown cancellation
2026-09-21 22:06:41 +08:00
spud
906c3d4554
fix(mcp): make durable task claims cancellation-safe (#4966)
* feat(mcp): re-scope to MCP task claim lifecycle only

Keep PR #4966 a small, closed MCP lease/cancellation state-machine change and
move RunJournal and Run lifecycle work into dedicated follow-ups. This branch
contains only the MCP task claim lifecycle:

- mcp task release/snapshot fencing by owner + per-claim lease token
- phase-level single-flight poll/cancel/notification owners with retained handoff
- routine cancellation no longer persisted as a task failure diagnostic
- bounded ordinary release ownership retention past the drain deadline
- 0018_mcp_task_lease_tokens migration + migration/bootstrap head assertions
- wait_for_task_until helper (MCP uses it); worker-specific capture helper moved
  to the run-finalization follow-up

RunJournal (journal.py + test_run_journal.py) and run lifecycle
(manager/worker/store/run sql + run tests) are preserved on
backup/cancellation-safety-full and will be raised as separate follow-ups.

* fix(mcp): unblock claims after ambiguous handoff resolves

A phase-level single-flight owner only guards an ambiguous claim outcome. Once
the claim resolves, the phase owner is released immediately; the handoff may
continue releasing returned rows as bounded, service-owned background work
(transferred to _compensation_tasks on timeout). Per-claim token fencing rejects
a late release against a newer claim generation, so a stuck release no longer
locks the whole phase until process restart.

- README: drop the stale progress-snapshot sentence from the bounded ordinary
  release description.
- service: pop the identity-checked phase owner as soon as the claim outcome is
  known, then release returned rows with the bounded path; carry the release in
  _compensation_tasks if it exceeds the drain deadline.
- mcp/AGENTS.md: document that only an unresolved claim outcome (not the handoff)
  blocks later phase scans, and that returned-row releases may continue in the
  background once the owner is released.
- tests: pin that the phase owner is released before a stuck release finishes
  while the release stays service strong-owned.

* refactor(mcp): remove unused single-record claim wrappers

_poll_one, _cancel_one, and _notify_one are unreachable in production: the
worker always processes claimed records through _run_claimed_batch, so these
wrappers preserved a second, dead single-record lifecycle (state is None)
whose only observable behavior was a wrapper-specific cancellation release.

Remove the three wrappers and migrate the regressions that guarded their
cancel/release invariants to exercise the production _run_claimed_batch path
(operation=_*_one_claimed, release=_release_*_after_cancellation). The single
wrapper-only "state is None" contract (test_poll_release_hang_without_batch)
is deleted; all 11 remaining invariants (CancelledError preservation, repeated
cancellation, poll-only token-fenced lease release, notification claimed vs
dispatched phase release, hung compensation -> service ownership, and
background compensation exactly-once observation) are now covered through the
real batch lifecycle.

* fix(mcp): fence claim-owned mutations against stale generations

The per-claim token check in the ORM release/apply paths was only in the
SELECT; the final write went out by primary key. On SQLite (where
with_for_update() is a no-op) a mutation from an older claim generation
could therefore clear a claim that a newer generation had reclaimed after lease
expiry — the exact distributed lease-fencing failure the per-claim token was
meant to prevent.

Make every claim-owned mutation a single atomic conditional UPDATE with the
owner and per-claim token in the WHERE clause (rowcount 0 => stale, return
False, no mutation):

- release_claim: atomic fence; record the poll-failure event after the fence
  wins (same transaction, holding the write lock).
- apply_snapshot / apply_cancel_snapshot: atomic fence; record the event after.
- finish_notification_run: atomic fence; use a CASE on event_version >>
  dispatch_version to keep a newer event pending for redelivery instead of
  swallowing it as delivered.

Add one regression per path: a stale generation's release/apply/finish after a
same-worker reclaim is rejected and never clears the newer claim.

* test(mcp): pin the migration chain head to the lease-token revision

0026_mcp_task_lease_tokens becomes the alembic head, so the chain-head pin in the 0025 repair test had to move on. Follow the 0023 precedent there (single head plus expected predecessor) instead of pinning a literal head, and give the new revision its own migration test, which owns the pin and covers the nullable claim-token columns on upgrade and their removal on downgrade.

* refactor(mcp): close cancellation cleanup leftovers

* fix(mcp): retain cancelled release diagnostics

* test(mcp): remove obsolete settled compensation case

* test(mcp): cover interleaved lease reclaim races

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 19:11:42 +08:00
spud
3776f6f5ec
fix(threads): clean persisted records safely on thread deletion (#5535)
* fix(events): serialize DB deletion with thread writers

* fix(runs): delete thread history without dropping reservations

* fix(feedback): support owner-scoped thread cleanup

* fix(threads): clean persisted records on deletion

* fix(threads): correct the feedback cleanup rationale

* test(runs): drop the wall-clock probe from the in-flight delete test

* docs: record the thread-delete and event-store fence contracts

* fix(threads): preserve legacy event-store delete compatibility
2026-09-18 18:32:42 +08:00
zhangwei-way
b6503e9a35
feat(knowledge): add per-message RAGFlow retrieval scope (#5238)
* feat(knowledge): integrate RAGFlow retrieval and management

* test(knowledge): cover merged listing tool

* feat(knowledge): add per-message retrieval scope

* chore(docs): remove unrelated document

* docs(knowledge): add interaction screenshots

* feat(knowledge): simplify scope selector trigger

* docs(knowledge): refresh selector screenshot

* feat(knowledge): defer standalone management

* docs(knowledge): show chat-only scope UI

* fix(knowledge): honor scope on clarification replies

* fix(knowledge): harden scoped replay validation

* docs(knowledge): clarify replay scope precedence

* fix(knowledge): keep provider settings on tools

* fix(config): preserve tools-only knowledge settings

* fix(knowledge): submit custom assistant identity

* refactor(knowledge): trim PR scope changes

* fix(knowledge): sanitize document scope display

* feat(knowledge): enable scope selection in main chat

* fix(knowledge): emphasize active scope icon without button frame

* fix(knowledge): close context scrubbing and refresh e2e checks

* fix(knowledge): preserve idempotent canonical retries

* fix(knowledge): accept promptless conversation runs

* style(knowledge): format backend regression tests

* chore(knowledge): trim PR scope and fix frontend format

* fix(knowledge): remove shared-scope notice

* fix(knowledge): remove scope persistence notice

* docs(knowledge): include main chat in catalog scope

* fix(knowledge): preserve scope recovery and upgrades

* fix(config): preserve LightRAG knowledge upgrades

---------

Co-authored-by: foreleven <for-eleven@hotmail.com>
2026-09-18 16:59:31 +08:00
RongJie G
94110e5dce
fix(subagents): close stream before releasing resources (#5221)
Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com>
2026-09-18 14:23:06 +08:00
xiaodu55
16f154f32b
fix(gateway): preserve owner isolation when thread metadata is missing (#5484)
* fix(gateway): preserve owner isolation when thread metadata is missing

Follow-up to the #5448 review P1 (post-merge finding): owner_check=True
also authorizes threads whose meta row is missing (legacy compatibility)
or NULL-owner (shared/pre-auth data). _run_scope_user_id returned None
for every trusted internal caller, which dropped the only remaining
per-user filter on those threads and let an internal caller acting for
owner A read owner B's persisted runs.

_run_scope_user_id now takes the thread_id and consults the thread meta
store: when an existing meta row establishes ownership, the authorized
thread's runs are still read unfiltered (merged #5448 semantics,
including owner-header-less internal callers); when the meta row is
missing or NULL-owner, the filter falls back to the acting owner's raw
stamp (the exact value start_run writes) — or the synthetic "default"
identity without an owner header — so cross-user runs stay hidden.

Isolation coverage uses the real MemoryThreadMetaStore with no metadata
row (and a NULL-owner row) plus another user's persisted run: /runs and
/runs/page must be empty and /runs/{run_id} must 404 for internal
callers, while an established-ownership thread keeps the unfiltered
read.

* fix(gateway): gate run-scoped sub-resource reads for internal callers

Review follow-up on #5484: the P1 owner-isolation class remained
reachable through run-scoped sibling reads that apply no per-user filter
at all — /runs/{run_id}/messages, /events, /join, /stream and
/workspace-changes query by (thread_id, run_id) directly, so on
missing/NULL-owner threads an internal caller acting for owner A could
still read owner B's run content by id (verified 200 at the previous
head).

- Extract _thread_ownership_established (shared meta-row check) and add
  _require_run_visible_to_scope: for internal callers on threads without
  established ownership, the run's own user_id stamp must match the
  acting owner's raw value (or the legacy "default" stamp) or the read
  404s. Established-ownership threads and every non-internal caller keep
  their existing thread-scoped semantics.
- Wire the gate into join, stream, messages, events and
  workspace-changes; reword the now-stale messages comment to track the
  new scoping semantics.

Regression tests: sub-resource reads 404 for a mismatched internal
owner while the matching owner reads them normally, and the owner-less
fallback branch (synthetic "default" filter on missing-meta threads) is
pinned. Red confirmed against the pre-gate head.

* fix(gateway): gate cancel and artifact archive for internal callers

Review follow-up on #5484 round 2: POST /cancel resolved runs unscoped
(require_existing=True only closes the missing-meta case — NULL-owner
meta rows still pass), so an internal caller acting for a different
owner could interrupt another owner's active run on a shared thread
while /join and /stream were already gated. The archive manifest and
download pair likewise leaked the other owner's delivered-file count
and a 200-vs-409 delivery oracle on NULL-owner threads (missing-meta
threads were already denied by require_existing=True).

All three routes now call _require_run_visible_to_scope; its docstring
records the extended coverage. NULL-owner-thread regression tests pin:
a mismatched internal owner gets 404 from cancel, manifest and archive
download, while the acting owner reaches the real conflict path (409 on
a terminal run) and reads the manifest (file_count 2).

* fix(gateway): tolerate state-less request stand-ins in the scope helpers

The new owner-isolation gate and _run_scope_user_id read request.state
directly, which crashed the FakeRequest-based unit suites for the run
events, workspace-changes and scope endpoints (backend-unit-tests shards
1/2/4 on #5484). Read the state object defensively first: a request
without state is simply not an internal caller, so those paths keep
their pre-gate semantics.

* fix(gateway): scope the thread token-usage aggregate by owner

Review follow-up on #5484 round 4: GET /{thread_id}/token-usage called
aggregate_tokens_by_thread(thread_id) with no user filter at all, so on
missing/NULL-owner threads an internal caller acting for owner A read
owner B's spend, model names, run count and (with include_active=true)
live activity; the NULL-owner variant reached browser sessions too.
build_context_usage's latest-model lookup was unfiltered as well.

aggregate_tokens_by_thread gains an optional user_id (mirroring
list_by_thread: explicit None = unfiltered, AUTO resolves the contextvar)
in the memory store, the SQL repository and the store base;
build_context_usage/_resolve_thread_model_name thread the scope through
the latest-run lookup; the token-usage endpoint passes
_run_scope_user_id's value. Established-ownership threads aggregate
unfiltered as before; shared/missing-meta threads narrow to the acting
identity. Stale helper-test comment reworded after the #5482 merge
adaptation.

* test(gateway): pin the unfiltered aggregate on established-ownership threads

Review follow-up on #5484 round 5: the established-ownership branch of
the token-usage scoping (store receives user_id=None) was the only
unpinned half of the contract — the round-4 call-assertions never set
app.state.thread_store, so their None came from the user-less stand-in
path. test_token_usage_unfiltered_on_established_ownership_for_
internal_callers seeds an established meta row plus runs stamped by two
different identities and asserts the totals fold (166 = 111 + 55);
together with the isolation tests it now catches both failure modes
(always-stamp narrowing and always-None leak).
2026-09-18 09:39:14 +08:00
NanPan
78117354b2
fix(events): preserve DB write-lock generation across deletion (#5462) 2026-09-17 22:06:03 +08:00
Fengmin Li
cda56aa282
fix(runtime): keep store-less run history through cleanup (#5453)
* fix(runtime): keep store-less run history through cleanup

run_agent schedules cleanup for every terminal run, but cleanup evicted
the in-memory record unconditionally. With a durable RunStore that is
safe: the store fallback in get()/list_by_thread() still serves the run.
Without one there is no fallback, so an embedded consumer that builds
RunManager() with its default store=None lost completed runs from
history entirely.

Gate eviction on a backing store, matching the retain-forever behaviour
documented for memory-only mode, and cover it: the two index/cleanup
tests now use a store, plus a regression asserting a store-less cleanup
keeps the record readable.

* test(runtime): terminalize the store-backed cleanup record first

Mirror the production sequence — run_agent only schedules cleanup once
the run is terminal and its store row is finalized — and pin that a
terminal store row hydrates with its status intact.
2026-09-17 07:38:38 +08:00
alanhuangyoo
0f2195e994
fix(goal): wait for the user when a turn ends on an unanswered question (#5467)
* fix(goal): wait for the user when a turn ends on an unanswered question

ask_clarification and the sandbox network prompt put their question in a
ToolMessage and end the graph. The goal evaluator only reads human and AI text,
so it never saw the question, judged the goal not met, and the worker queued a
hidden continuation telling the agent to keep going while the card was still
open. The agent could then act on a guess before the user answered.

Stand the goal down with blocker needs_user_input, without calling the
evaluator, when the trailing tool results include a human input request.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(goal): cover resuming after answered clarification

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-16 21:30:57 +08:00
Xuehao Xu
f0cb67b223
feat(extensions): expose incremental run evidence reader (#5405)
* feat(extensions): expose incremental run evidence reader

* fix(extensions): address run evidence review feedback

* docs(extensions): clarify run deletion reconciliation

* docs(migrations): align current head documentation

* fix(extensions): isolate run evidence event reads

* test: avoid pinning run change migration to latest head

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-16 21:18:05 +08:00
Zeren Wang
a58ab484a6
feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash (#5443)
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash

Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.

Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
  admission pinning (both server-owned sets + worker hoist); latest-only
  request-scoped <project> block via DynamicContextMiddleware
  wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
  prefix + marker + provenance, never persisted); journal audit
  fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
  locked check-and-set; hash-qualified immutable shelf storage with
  Paths helpers; upload/list/content/delete-to-trash routes; project
  delete trashes the shelf in-transaction; request-scoped bounded
  <documents> index with honest count/shown + actionable overflow note;
  list_project_documents/read_project_document tools registered only on
  pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
  parity); POST from-thread with provenance; attach-to-thread with
  lock-staged copy (archived source allowed); read-only thread-files
  view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
  file moves), purge (continuous row lock across unlink/delete/commit,
  retryable on FS errors), retention sweep (lazy + startup, 24h orphan
  guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
  archived banner, content-missing rows), /workspace/trash route,
  sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
  + specs.

Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
  artifact + content responses; unified unsandboxed-iframe PDF preview
  (fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
  claims + os.link commit with suffix retry; same-name re-upload now
  unique-names instead of replacing); hidden staging only, no visible
  placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
  revalidation; drain locked filesystem work on cancellation; preserve
  bytes when an insert's commit state is uncertain (including trashed
  rows)
- original-integrity checks before serving text or cached conversions;
  content_missing surfaced in list responses (UI reads the flag, no
  409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
  declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
  rejects fractional/infinite values; composer counts staged
  attachments; pending attachments persist until submission or removal;
  in-flight instruction/rename edits survive save refetches; shelf and
  trash pagination; conversation-file and thread-files pages stay
  subscribed to refetches

Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.

Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
  bounded: the indexed expiry purge still runs on every trigger
  (GET /api/trash/documents, POST /api/trash/purge) while the
  O(all rows + all files) reconciliation is throttled to one run per
  user per 15 minutes (process-local, per-user window). The startup
  sweep now runs as a background task instead of blocking gateway
  readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
  like the render path, so a pasted, fenced <project>/<documents> snippet
  survives markdown export while real injected blocks (never fenced) are
  still removed. Fence regexes moved to a dependency-free leaf module to
  avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
  attribute (the upstream e2e contract locates it via :not([title])), and
  the upstream artifact-preview spec now pins the new contract: PDFs
  render unsandboxed, images keep sandbox="".

* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title

- Shutdown cancelled only the shield around the background startup sweep,
  so an all-users reconciliation that outlived the 5s budget kept walking
  rows and files while the document repo and DB engine were disposed
  underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
  which cancels the task and drains it before worker exit: the shield
  keeps the wait bounded, the cancel makes it final (CancelledError lands
  at the sweep's next await, and `_run_startup_trash_sweep` only catches
  `Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
  previous fix round, leaving the PDF frame without an accessible name
  while its siblings keep theirs. Restore it (WCAG frame titles), assert
  it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
  instead of `iframe:not([title])`.

* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel

`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.

* fix(projects): round-4 review — make Empty trash delete what it confirms

`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.

Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.

Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
2026-09-16 18:46:18 +08:00
NanPan
fc4e0c32ba
fix(events): keep JSONL lock generation stable across deletion (#5455)
* fix(events): preserve JSONL lock generation across deletion

* test(events): cover JSONL lock generation after delete

* chore(events): clarify JSONL delete lock lifecycle

* style(events): restore JSONL trailing newline
2026-09-16 15:18:29 +08:00
PeaceMaker-best
14c9d44440
feat(runtime): persist tool-progress phase transitions (#5214)
* feat(runtime): persist tool-progress phase transitions

Record bounded warn, block, and recover decisions for lead and task subagent runs while preserving event-loop isolation, fail-open behavior, and concurrent transition order.

* fix(runtime): trust server-owned tool progress attribution

* fix(runtime): centralize trusted audit attribution

* fix(runtime): preserve complete tool progress audit state

* docs: trim tool progress guidance to pass size check

* fix(runtime): fence subagent audit recorder loop

* docs(readme): sync tool-progress event coverage

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>

---------

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: 嗜鵼 <hy2010hy2010@qq.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-15 08:45:08 +08:00
Undermoon1412
6c697f3067
fix(events): drain JSONL mutations before propagating cancellation (#5439)
* fix(events): drain JSONL mutations before propagating cancellation

Signed-off-by: Undermoon1412 <Undermoon1412@users.noreply.github.com>

* fix(events): cover cross-thread cancellation and name drain tasks

Signed-off-by: Undermoon1412 <Undermoon1412@users.noreply.github.com>

---------

Signed-off-by: Undermoon1412 <Undermoon1412@users.noreply.github.com>
Co-authored-by: Undermoon1412 <Undermoon1412@users.noreply.github.com>
2026-09-15 07:28:24 +08:00
Undermoon1412
f770ecc0b8
fix(events): preserve Unicode separators in JSONL event records (#5429)
Signed-off-by: Undermoon1412 <Undermoon1412@users.noreply.github.com>
Co-authored-by: Undermoon1412 <Undermoon1412@users.noreply.github.com>
2026-09-14 18:24:51 +08:00
Beautyl0ve
80f13935c2
feat(agents): allow custom agents to disable memory (#5167)
* feat(agents): allow custom agents to disable memory

Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com>

* fix(agents): honor memory opt-out during compaction

Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com>

* fix(runtime): preserve agent binding across state rewrites

* fix(client): apply named-agent memory policy

* fix(agents): address memory policy review feedback

Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com>

* fix(agents): address remaining memory opt-out reviews

Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com>

---------

Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com>
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-14 18:17:54 +08:00
alanhuangyoo
0a0d768107
fix(goal): stand the goal down once the run has hit its token budget (#5424)
* fix(goal): stand the goal down once the run has hit its token budget

Since #5410 goal continuations share the run's token budget, so a
continuation queued after the budget's hard stop only spends one more
model call before its tool calls are stripped. Pass the run's stop_reason
into the goal loop and stand the goal down with "token_capped" in that
case. The evaluator still runs first, so a goal the capped run satisfied
is cleared as before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(goal): list the token-budget stop among goal-loop preconditions

Also pin that a satisfied goal is cleared, not stood down, when the run
hit its token budget.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 15:46:12 +08:00
Totoro
533e30e7f2
[feat] add opt-in conversation reads for Gateway runs (#5399)
* feat: add scoped conversation reads to gateway runs

* refactor: share conversation tool path and reuse parsed text

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
2026-09-14 10:51:01 +08:00
alanhuangyoo
5a3725d1b8
fix(worker): don't fail the parent run on a subagent error carried by task_running (#5407)
* fix(worker): don't fail the parent run on a subagent error carried by task_running

The worker scans root frames for the LLM error-fallback marker to decide the
parent run's status, and custom frames are root frames. task_running events
carry each delegated subagent message, including a subagent's marked error
fallback, so a subagent that failed after retries marked the whole parent run
as error (and an edit-and-rerun rolled back) even though the executor already
reported task_failed and the lead answered. Skip custom frames in both the
single-mode and multi-mode stream paths.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(worker): cover the single-mode stream loop for task_running fallbacks

Run the task_running regression test with stream_modes=["custom"] too,
which takes the worker's single-mode loop. Without the single-mode
guard only that case fails.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 07:17:51 +08:00
Hyeonsang Cho
6f81daefff
fix(runtime): resolve omitted run owners before idempotent reuse (#5401)
* fix(runtime): resolve omitted run owners before idempotent reuse

HTTP run admissions do not pass a user_id. The SQL run store stamps the
request user from the contextvar onto the row, but RunManager kept None
on its process-local RunRecord, so the two disagreed about who owns the
run. A keyed retry that reached a peer worker, or the owning worker after
cleanup() released its local record, hydrated the stamped row, compared
its owner with None, and raised "Run idempotency key resolved to a
different thread or user", which start_run surfaced as a 500.
MemoryRunStore stored None on both sides and never hit the check.

The same mismatch hid HTTP runs from owner-scoped history reads, which
filter local records by the current user, and skipped the worker's MCP
background_tasks projection, which only runs for records with an owner.

create() and _admit_thread_operation() now resolve an omitted owner from
the current user before building the record, and keep None when no user
is in context instead of falling back to the default bucket, so the
local record and every store agree on the owner. HTTP runs now receive
the background_tasks projection, so the replay golden's values frames
gain that key.

* docs(changelog): reference #5401 in the keyed retry owner fix entry

* test(runtime): close the SQL engine when peer-reuse test setup fails

Review follow-up on #5401: the sql case of the two-worker start_run test
initialized the engine above the try whose finally calls close_engine().
init_engine() assigns the module-global engine and session factory before
bootstrapping the schema, so a failure there skipped the teardown and left
a stale engine for later tests in the same process.

Store setup and the RunManager workers now live inside the try, so the
teardown runs whether setup or the test body fails.
2026-09-13 20:28:32 +08:00
Hyeonsang Cho
28a81452ce
fix(runtime): stop idempotent reuse from blocking the thread on a peer worker (#5393)
* fix(runtime): stop idempotent reuse from blocking the thread on a peer worker

When an idempotent run admission lands on a worker that does not own the
run, RunManager hydrates the stored row and returns it as the reused
record. It also registered that row in the worker's local run map, but
only the owning worker's task lifecycle finalizes and cleans up local
records, so the copy kept its admission-time pending/running status for
the life of the process. On that worker every later reject-strategy
admission for the thread returned 409 until a restart, run reads kept
reporting the stale status, orphan reconciliation skipped the run as
locally live if the owner crashed, and a cancel took the local-owner
path and marked the owner's still-running row interrupted.

Return the hydrated row as a detached store-only handle instead of
registering it. A local record for the key is already returned before
the store insert, so the removed lookup of an existing local record was
unreachable. get(), cancel() and reconciliation now read the durable row
on the peer, matching the documented non-owner contract.

* test(runtime): pin keyed retries of a terminal reused run on the SQL store

Review follow-up on #5393: the post-cleanup release relied on a keyed
retry resolving through the terminal row's idempotency conflict, but
only MemoryRunStore pinned that path, and no test retried on the owner
after its local record was cleaned up.

The SQL repository test now retries the key on the peer and on the
owner once the run is terminal and cleaned up, asserting both get the
same run back as a store-only reused handle before the keyless
follow-up is admitted.
2026-09-13 18:16:22 +08:00
Wenchao An
1b76ab9060
feat: add opt-in task notes and compacted history recall (#5382)
* feat: add opt-in task notes and compacted history recall

* fix: validate task continuity state and preserve user answers

Honor explicit opt-out, preserve clarification replies and capture failure statuses, validate notebook writes, and clear branch archive references. Update the config version and audit optional LLM credentials, with regression and integration evidence.

* fix: align Helm config version with task continuity schema

* fix: preserve mixed task history and declare continuity policies

* fix: recover malformed history and evict archives atomically
2026-09-12 21:01:46 +08:00
Shxiao
572744975d
fix(tools): run tool assembly off-loop at async entry points (#5224)
* fix(tools): run tool assembly off-loop at async entry points

get_available_tools() may block on MCP cache initialization while it is
called on async agent-assembly paths (task_tool, durable batch execution),
stalling the calling event loop for the full discovery duration.

Dispatch the (unchanged, synchronous) assembly call to a worker thread via
asyncio.to_thread at the two async entry points so the loop keeps processing
requests, SSE frames, cancellations, and timers.

Fixes #5172

* fix(tools): offload lead-agent assembly off-loop and pin with blocking-io anchors

Review follow-up for #5224:

- run_agent now dispatches agent_factory(...) through asyncio.to_thread, so
  lead-agent assembly (including both get_available_tools call sites in
  _assemble_lead_agent) runs off the event loop — the Gateway headline
  scenario from issue #5172.
- _ensure_sync_invocable_tool takes a double-checked threading.Lock, making
  the in-place tool.func wrap on the shared tool singletons explicitly
  single-shot now that assembly can run concurrently on worker threads.
- Add backend/tests/blocking_io/test_tool_assembly_offloop.py: blocking-probe
  anchors for task_tool and SubagentBatchService._execute_item under the
  strict Blockbuster gate, plus a meta-check proving the gate trips on the
  exact syscall class (ExtensionsConfig.from_file on the loop). Verified the
  anchor goes red when the offload is flattened back to a plain call.

* fix(gateway): build checkpoint state accessor off-loop; anchor run_agent offload

Review follow-up for #5224:

- Add abuild_checkpoint_state_accessor (asyncio.to_thread around the
  unchanged sync builder) and switch every async call site to it: the
  stateless_wait route, thread_runs, both threads call sites, and the
  build_thread_checkpoint_state_accessor boundary. The agent-factory
  assembly re-enters get_available_tools() and may block on MCP cache
  initialization; repeat calls hit _state_accessor_graph_cache and only
  pay the thread hop.
- Add a third blocking-io anchor driving the real run_agent with minimal
  RunManager/bridge stubs; the factory performs a real production blocking
  read (ExtensionsConfig.from_file()) and the test asserts assembly never
  runs on the main thread. Verified the anchor goes red when the run_agent
  offload is flattened back to a plain call.
- Adapt the test_threads_router checkpoint-builder patch sites to the new
  async name.

* refactor(tools): carry assembly offloads on a dedicated bounded pool

Review follow-up for #5224:

- Add utils/assembly_io.py: a dedicated ThreadPoolExecutor (default 8
  workers, DEER_FLOW_ASSEMBLY_WORKERS-overridable, mirroring
  utils/file_io.py and tools/sync.py) with run_assembly(), which copies
  contextvars explicitly. A hung stdio MCP server parks its worker for
  the full MCP timeout; carrying assembly hops on the loop's default
  executor would let a few parked assemblies queue every other
  to_thread/run_in_executor(None, ...) caller behind them.
- Switch all four offloads (run_agent, task_tool, batch _execute_item,
  abuild_checkpoint_state_accessor) to run_assembly().
- State the cold-path behavior in the accessor docstring: the graph
  cache validates factory identity, so non-identity-stable factories may
  duplicate lead-agent assembly across concurrent readers (MCP discovery
  stays process-wide single-flight); the pool bounds the duplicates.
- Add a fourth blocking-io anchor driving build_thread_checkpoint_state_
  accessor with a per-resolution fresh factory (always a cache miss) and
  the real production blocking read; enumerate all four offloads in the
  gate's module docstring. Verified the anchor goes red when
  abuild_checkpoint_state_accessor is flattened back to a plain call.

* fix(subagents): revalidate batch item before launch; make assembly pool observable

Review follow-up for #5224:

- _execute_item() revalidates the durable state right after assembly and
  before executor.execute_async(): renew_item_lease() returns valid=False
  when cancel_batch() terminalized the item or the lease was lost while
  assembly was parked, and the launch is skipped (the canceller already
  finalized the item). Previously the launch was unconditional and the
  poll loop's cancellation checks only started after execution began.
- Regression test driving the real SQLite repository: a blocking assembly
  probe parks _execute_item, cancel_batch() lands, and the launch is
  skipped with the item staying cancelled. Verified the test goes red
  when the revalidation is removed.
- run_assembly() tracks pending assemblies and logs a throttled WARNING
  once the pending count exceeds the worker count, so assembly starvation
  (workers parked on a hung MCP server) is distinguishable from idle.
- The run_agent blocking-io anchor now binds a sentinel extension
  snapshot via ctx.extensions and asserts the factory observed it through
  get_agent_build_extensions(), pinning run_assembly()'s ContextVar
  propagation. Verified red when ctx.run is dropped.
- Document the assembly pool in backend/AGENTS.md.

* fix(utils): decrement the assembly pending count on the pool thread

The pending-assembly counter behind the starvation warning decremented
from the asyncio future's done callback, which never fires once the
submitting loop is closed while its worker is still running: the count
ratcheted up permanently and eventually fired the starvation warning
with no starvation behind it (reproduced at 97dc9bec by review).

Decrement instead from the dispatched work item: run_assembly() wraps
func so a finally drops the count under the pending lock on the pool
thread, and the done callback is gone.

Pin the counter with tests/test_assembly_io.py: a healthy call returns
the count to zero, and an abandoned loop (stopped while the worker is
parked) does not wedge it — the abandoned case goes red against the old
done-callback decrement.

* docs(utils): fix the pending-counter comment after the decrement move

The comment still described the removed done-callback decrement,
contradicting _work()'s own comment; state the actual mechanism
(increment on the loop before dispatch, decrement from the dispatched
work item's finally on a pool thread).

* test(gateway): retarget checkpoint-accessor stubs to the services seam

thread_runs and runs now call abuild_checkpoint_state_accessor, so the
upstream wait-reader, regenerate-prepare, and idempotency tests must stub
the sync builder where abuild resolves it (app.gateway.services); stubbing
the removed router re-exports fails with AttributeError at setup. The
async seam semantics are unchanged: run_assembly invokes the stubbed
sync builder off-loop and propagates its return values and exceptions.

Move the agent/tool assembly off-load note from backend/AGENTS.md to
deerflow/utils/AGENTS.md (next to assembly_io.py) so the effective
instruction chain for agents/middlewares no longer grows past the AG002
hard limit.

* fix(runtime): serialize same-key accessor assembly and release queued-cancel slots

Address the three review follow-ups on the assembly off-load:

- assembly_io: a job cancelled while still queued never runs its work
  item, so the dispatched finally never fired and _pending_assemblies
  stayed elevated until a false starvation warning. Exactly-once cleanup
  now rides the concurrent future's cancelled() state — cancel() only
  succeeds before the executor starts the item, so cancelled() is true
  precisely when the finally will never run — plus a submit-failure
  release; the one-worker queued-cancellation case is pinned red/green.
- services: overlapping cold readers sharing one cache key could both
  run full agent assembly. _state_accessor_graph now serializes per key
  through a thread-side KeyedLockTable (pool threads, no running loop)
  and re-validates factory/app-config identity under the lock, so the
  factory runs exactly once while identity changes still rebuild. Cache
  dict access is lock-guarded now that construction runs off-loop.
- guidance inventory: register deerflow/utils/AGENTS.md in
  EXPECTED_GUIDANCE_PATHS so test_repository_has_the_approved_scoped_
  guidance_shape matches the relocated assembly note (CI shard 4).

* test(keyed-lock): pin KeyedLockTable reclamation and waiter bypass directly

Thread-side counterparts of the async table's own tests: overlapping
hold() calls serialize (a late arrival joins the live entry instead of
creating a second lock that bypasses a queued waiter), the last check-in
pops the entry, and many unique keys leave the registry empty. Both
regressions verified red — popping unconditionally trips the late-arrival
test, never reclaiming trips the many-keys test.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-12 07:41:24 +08:00