74 Commits

Author SHA1 Message Date
Hyeonsang Cho
ec3f3d4d22
fix(threads): strip newlines from message ids in regenerate scan warnings (#8350)
The two scan-limit warnings on the regenerate and edit-replay paths
logged the assistant and user message ids raw. Those ids come from
checkpoint state, and run input keeps a client's message id verbatim,
so a thread owner could start a forged line in the Gateway's text log.
Pass them through sanitize_log_param like the other warnings in the
same functions.
2026-11-18 14:30:44 +08:00
哈基米
f95d4f6353
fix(gateway): map checkpoint-mode errors in run-prepare routes (#8296)
* fix(gateway): map checkpoint-mode errors in run-prepare routes

The regenerate and edit-regenerate prepare routes read the same thread
checkpoints as the threads router but built the accessor and read the
latest checkpoint under a generic handler, so a delta/full mismatch
surfaced as a 500 instead of the 409 that names the mode flip. Move the
mapper to a shared module, reuse it for the accessor, latest-checkpoint
and history-scan failures, and update the troubleshooting page now that
the routes behave like the other state surfaces.

* fix(gateway): preserve route-level HTTP errors in run-prepare

The checkpoint-read fallback now sits around the accessor build, so a
config-loading HTTPException from the accessor was being rewritten as a
generic 500. Re-raise HTTPException before the generic handler and cover
the 503 passthrough with a test.

* docs: confirm the mode mismatch before advising a restart

A null token usage or a /runs/wait error body can also come from a
configuration or token-counting failure, so the troubleshooting entry
should point at the 409 from the thread/state endpoints first.

* fix(gateway): remove duplicate checkpoint imports after merge

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-11-17 12:10:08 +08:00
EddieMurphy
ae871431a2
fix(sandbox): sanitize config-failure logs and map archive errors to 503 (#8185) 2026-11-13 20:41:27 +08:00
Wenchao An
3d826c6d95
feat(approvals): add Ask, Auto and Full tool permission modes (#8033)
* feat(hitl): add human-in-the-loop approval for high-risk tool calls

* fix(hitl): address approval review findings

* docs(hitl): keep agent guidance under size limit

* fix(hitl): guard approval pause recovery

* docs(hitl): shrink inherited guidance

* fix(hitl): harden approval admission and recovery

* fix(approvals): address review safety and reconciliation gaps

* ci: retry approval checks after runner timeout

---------

Co-authored-by: wangzeren <1004695029@qq.com>
2026-11-11 20:59:05 +08:00
Totoro
3f9a78e02e
fix(goal,artifacts): verify completion and artifact handoff (#7961)
* fix(goal,artifacts): verify completion and artifact handoff

* fix(goal,artifacts): harden shared artifact handoff

* docs(agents): shrink inherited guidance to fit chain budget

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
2026-11-07 18:21:49 +08:00
Xiao Hou
830c8ff6f1
fix(runs): trim event_types filter entries before querying (#7758)
* fix(runs): trim event_types filter entries before querying

* style(runs): compute the trimmed event type once per entry

Addresses the review nit on #7758: use the walrus pattern already present
in this file (lines 657 and 1658) instead of stripping each entry twice.

No behavior change.

---------

Co-authored-by: gouyu-hou <gouyu-hou@users.noreply.github.com>
Co-authored-by: gouyu-hou <213296172+gouyu-hou@users.noreply.github.com>
2026-11-01 23:27:00 +08:00
Liu Hao
762b7188ab
fix(conversation): tolerate malformed event metadata (#7721)
* fix(conversation): tolerate malformed event metadata

* fix(gateway): guard malformed event metadata

* fix(gateway): guard malformed message kwargs

* fix(gateway): log malformed transcript metadata

* fix(gateway): ignore malformed message roles
2026-10-31 18:25:31 +08:00
Totoro
44e372dd7f
feat(runtime): support Queue and Steer during active Web runs (#7729)
* feat(runtime): add Queue and Steer for active Web runs

* style(frontend): format Queue and Steer controls

* fix(runtime): retain user Steer controls and order active inputs

* docs: add Chinese changelog entry for Queue and Steer

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-31 11:35:36 +08:00
Totoro
c48447a4ae
fix(runtime): present outputs before a loop wrap-up and explain step-limit stops (#7588)
* fix(runtime): present outputs before a loop wrap-up and explain step-limit stops

Loop-detection warnings now ask the agent to present the files it already
created in /mnt/user-data/outputs before its final answer, when
present_files is available.

A run that exhausts recursion_limit now ends with stop_reason
recursion_capped and a plain message that lists outputs it created but
did not present. An active goal stands down without an evaluator call.
The /wait error envelope keeps the GraphRecursionError name, and IM
channels reply with the step-limit message instead of a generic failure.

* docs(changelog): note run-end delivery and step-limit stops (#7588)

* test(runtime): stop pinning LangGraph's step count in the step-limit test

The produced paths and the message already follow the observed model
calls, so the exact count only pinned LangGraph's per-step accounting.
Also note on the step-limit goal branch that its caller passes no
evaluator factory, so it has to stay ahead of the evaluator section.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-27 10:34:34 +08:00
NanPan
7cfd3c57d2
fix(gateway): bound run event type filters (#7370)
* fix(gateway): bound run event type filters

* test(gateway): cover run event filter cardinality

* style(gateway): group run event filter limit with module constants
2026-10-24 16:11:57 +08:00
Nan Gao
085924a3d0
fix(channels): run each IM message once across instances, restarts and late redeliveries (#7104)
* fix(channels): run each IM message once across instances, restarts and late redeliveries

Inbound IM messages were deduplicated only by the 10-minute inbound dedupe
store (#7103). Discord direct messages had no key at all, a Discord thread
reply got an instance-dependent key, a Discord thread was created before
dedupe ran, a redelivery past the window started a second run, and two
instances racing on a conversation's first messages bound it to two
threads.

- Discord messages are keyed (discord, guild_id or "dm", posting channel,
  message_id); the posting channel is the same on every instance.
- MessageBus.claim_inbound lets an adapter record the manager's dedupe key
  before a side effect; Discord claims it on the Gateway loop before
  _create_thread, marks the published message dedupe_claimed, and gives the
  claim back when the message is never committed.
- runs.wait / runs.stream / runs.create send an Idempotency-Key derived from
  the dedupe key, and keyed human message ids are uuid5 of it (a uuid4 per
  attempt made every keyed retry a 409). A reused run's status-only /wait or
  stream gap is resolved to this message's own turn via threads.get_state;
  409s about the message's own keyed run keep the key instead of reading as
  a busy thread; an ended reused GitHub run gets no follow-up watcher.
- A conversation's first thread binding is claimed insert-if-absent
  (claim_thread_id on channel_conversations and channel_thread_bindings);
  the loser adopts the winner and deletes the empty thread it minted.
  /new and /agent use still rebind.

Docs: IM_CHANNEL_CONNECTIONS.md, CONFIGURATION.md, channels AGENTS.md,
CHANGELOG (en/zh).

* fix(channels): give back Discord dedupe claims for messages never processed

Review round 1 on #7104. A Discord claim records the manager's dedupe key
before the message is queued, so every path that drops the message unrun
must release it, or a peer refuses the redelivery for the rest of the TTL:

- The hand-off task releases the claim on cancellation too (stop() drains it
  while it resolves the connection identity), via await_drained so the
  cancellation still propagates.
- ChannelManager._discard_pending_inbound releases the key of a claimed
  message that shutdown drops from the queue, after the grace period and on
  a cancelled shutdown.
- The adapter tracks claims until they are committed. A release refused
  because stop() already closed cross-loop intake, or a hand-off cancelled
  before it ran, leaves the claim tracked, and stop() releases it last.

* fix(gateway): keep IM run idempotency stable when /new rebinds the conversation

Review round 2 on #7104. The Gateway scoped every Idempotency-Key by owner
and thread. /new rebinds an IM conversation to a fresh thread, so a late
redelivery of a message admitted on the old thread was requested on the new
one, got a different scoped key and started a second run.

- Keys from trusted internal callers (the IM channel manager) are scoped by
  owner only (`internal-run:` prefix); browser and API keys keep the
  owner-and-thread scope (`http-run:`).
- A key that resolves to a run of another thread or user raises
  RunIdempotencyKeyMismatch, a RuntimeError subclass so in-process launchers
  see the same type, and start_run answers it with 409 "Idempotency-Key
  already used with a different request" instead of a 500. The channel
  manager already treats that 409 as the message's admitted run.

Docs: API.md, IM_CHANNEL_CONNECTIONS.md, gateway AGENTS.md (reworded to stay
under its budget), CHANGELOG (en/zh).

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-19 22:57:19 +08:00
Totoro
0fe6e63a68
feat(goal): show why a chat goal stopped and when it was met, and redact the goal evaluator and continuation input (#6556)
* feat(goal): record the met goal and keep goal state on the history head

- Add a goal_outcome LastValue channel (GoalOutcomeState). Only the
  guarded clear of a satisfied ordinary goal writes it, in the same
  checkpoint as the clear; the reply anchor is the latest visible AI
  message with text, the reply the visible signature keys on. A goal
  stored without created_at still clears and gets a record.
- write_thread_goal removes the record on every other goal write and
  lists it in channel_versions, new_versions and metadata writes when
  it changes. Threads that never had a record keep their checkpoint
  shape. The sync-only SqliteSaver and CachedHistorySaver, which take
  the #6448 sync fallback, store and drop it the same way.
- goal_outcome is server-owned: POST /state and non-internal run input
  that carry it get 400. A POST /state goal write, or a goal in
  external run input, clears it.
- A branch head written onto its replay base resets the channels the
  head lacks, so a goal or record that write_thread_goal removed does
  not come back. Branching from the met turn keeps the record without
  the met goal.
- The POST /history head now carries an active goal and an achieved
  goal_outcome, also before any message exists, so the post-run
  history refetch keeps them. Older entries stay title/thread_data.
- contracts/thread_goal_contract.json pins the chat stand-down and
  check-failure codes, the record shape and status, the head keys and
  the host-written reasons.

* feat(frontend): show why a goal stopped and when it was met

- The goal bar derives its state from the active goal, the goal_outcome
  record and the messages: set, in progress, continuing, stopped (limit,
  auto-continue off, no progress, token budget, missing evidence, failed
  run, unknown code), waiting for you, waiting, couldn't check the goal,
  paused and goal met. An 8/8 stand-down now reads "Stopped" with the
  next step instead of "Continuing 8/8", and a count-0 stand-down no
  longer looks like a fresh goal.
- "Goal met" shows while its anchor reply is the latest visible message
  (a visible human message, or a visible AI message with text, moves the
  chat on), so a reload agrees with the live view. An active goal wins
  over the record, and this tab's /goal set or clear hides it.
- useActiveGoal drops the optimistic goal when the record of that same
  goal arrives, which fixes the bar that stayed after a goal met on the
  first run (the clear frame has no goal key), also for a goal stored
  without created_at. Server goal values pass through isGoalState; a
  /goal status read uses the backend's own active check, so the bar, the
  toast and the edit lock agree with the server.
- Row 2 gives the reason and what to do next; Details holds the full
  goal, the checker's own note (never host-written text, a check failure
  or a paused goal's older check), the assumption note and an unknown
  code, and starts collapsed in each new state. The bar is a labelled
  section with a live region that announces state changes only (another
  goal's same state counts) and re-reads its text after a language switch.
- The edit pencil on the latest turn is shown disabled with "Editing is
  off while a goal is set" instead of hidden; the message toolbar shows
  on focus-within, so keyboard users can reach it.
- A /goal set or clear rejected with 409 keeps the draft and says a run
  is still going; other failures keep the server's message and the draft.
  A /goal request replaced by a newer one leaves the composer alone, so
  a double submit no longer empties it before the second one fails.
- Copy lives in inputBox.goalBar (en-US, zh-CN); a contract test reads
  contracts/thread_goal_contract.json, and the raw-identifier guard now
  catches stand-down codes and goal field names.

* test(frontend): cover the goal bar on both chat pages

- tests/e2e/goal-status.spec.ts runs every case on the chat page and the
  agent chat page, since each wires the goal bar and the edit lock itself:
  a goal stopped at 8/8, one waiting for details and one waiting on an
  open question card show their reason and next step, also after a
  reload; the locked edit pencil gets keyboard focus, shows its toolbar
  and tooltip, does not open the editor, and hides while a file uploads;
  Goal met after one automatic follow-up (In progress, Continuing 1/8,
  then "Auto-continued once") survives a reload and hides after the next
  visible message; a goal met on the first run of a new chat replaces the
  set bar and frees the pencil, and a new /goal replaces Goal met while
  its own run goes on; a /goal set or clear refused with 409 keeps the
  draft for the retry and shows its own busy toast.
- The mock POST /history head carries only the contract's
  history_head_keys, with goal only while active and goal_outcome only
  when achieved; thread state and run values omit an unset goal instead
  of sending goal: null. PUT and DELETE /goal drop the met record,
  setGoalWritesBusy returns the Gateway's 409, and a repeat create of a
  requested thread id returns the thread as is, like the Gateway.
- The raw-identifier guard also catches checker blockers and the met
  record's field names.
- frontend/src/AGENTS.md: the GoalStatus sentence now names goal,
  goal_outcome and useActiveGoal, reworded in place.

* test(frontend): keep the draft through a failed upload and a busy /goal

#6412 fires onSent only once a send is dispatched, so a failed upload
keeps the text, files and stored draft. A /goal the Gateway refuses with
409 never reaches sendMessage, and the composer rejects it so PromptInput
keeps the text. Drive both through the real sendMessage in one composer:
a failed upload keeps "summarize it" stored, a busy /goal keeps its text
and stored draft with no run, and the retry saves the goal and starts
one run with the objective, not the "/goal" text. After that run the
composer is empty and no stored draft comes back.

* fix(frontend): drop the stop condition's closing period before the pause clause

A stored stop condition often ends with a period, so the schedule card
read "…remain)., pauses itself" (and "…。,满足后自动暂停" in Chinese).
Trim the condition's closing sentence punctuation before the clause that
follows it. The event line, where the condition ends the sentence, is
unchanged.

* fix(frontend): point "evidence missing" at the agent, not a user reply

"Evidence missing" asked the user to reply with what's left, which is
the push a goal is meant to save. It now asks the agent to show the
result or explain what's missing, and offers /goal clear only for a goal that
no longer applies: clearing drops the met record, so work that is right
would never show "Goal met".

zh-CN follows. Copy only: no key, state or mapping changes, and the
scheduled page's reason labels are untouched.

tests/unit/components/workspace/goal-status-next-steps.test.ts checks
the row in en-US and zh-CN, and that no stop row offers /goal clear
without that condition. goal-status.dom.test.tsx and
goal-stream.dom.test.tsx now expect the new copy.

* fix(frontend): say "Couldn't check the goal" once in the goal bar

After a goal check that could not run, the bar's second line borrowed
the scheduled page's whole reason, so it repeated the chip: "Couldn't
check the goal · Couldn't check the goal; the run itself may be fine.
It's checked again after your next message." Chinese had the same
repeat.

The line now gives only why the check failed ("The run itself may be
fine", "No final reply was saved" or "The chat changed during the
check"), then when it is checked again. The screen-reader announcement
follows. The chip and the scheduled page's reason labels are unchanged.

tests/unit/components/workspace/goal-status-unchecked.test.ts checks
every check-failure code in en-US and zh-CN: the label appears once,
and the line still says why the check failed.

* feat(goal): redact the evaluator input when pii_redaction is enabled

The goal evaluator calls its model directly, outside
PiiRedactionMiddleware, so it sent raw user lines, tool arguments, tool
results and Human Input Card answers even with pii_redaction enabled.

With pii_redaction enabled:
- the messages in the evidence window are redacted whole before the
  evidence caps can cut an identifier in two, as TitleMiddleware does.
  This reuses redact_queued_messages, which already covers content,
  call arguments and card answers. Earlier messages are left alone: the
  evaluator never sees them, and the pass is synchronous work on the
  event loop (0.84 s for an 800-message, 4 MB thread, 0.06 s for its
  window);
- the assembled input, the goal objective included, is redacted again.

A redaction error fails the check with evaluator_failed; raw text is
never sent instead. When redaction is off, the input is byte-identical.
Thread state keeps the raw text. README.md, config.example.yaml and the
middleware docstring list the evaluator with the other redacted inputs.

tests/test_goal_pii.py puts an address where the evidence caps cut a
tool argument, a tool result and a card answer, and checks the
objective, the unchanged input when redaction is off, that only the
evidence window reaches the redactor and gives the same evidence as
redacting the whole thread, that a redaction error stands down with
evaluator_failed before the model is called, and the worker path.

* fix(goal): redact the hidden continuation when pii_redaction is enabled

The hidden /goal continuation repeats the goal objective and the
evaluator's reason and evidence summary. PiiRedactionMiddleware skips
it, since it is a framework message and not a user message, so with
pii_redaction enabled each continuation turn sent them raw to the
agent's model, and the message stayed in the thread for later model
calls. The durable-context block and the evaluator input already redact
the objective.

With pii_redaction enabled, make_goal_continuation_message now redacts
the objective, the reason and the evidence summary with redact_text
before the message is built. The thread keeps the redacted message,
which the UI hides; the goal state keeps the raw objective.

The worker builds the message before it counts the continuation. A
redaction error fails the check the way an evaluator redaction error
does: run_failed with stand_down_reason evaluator_failed, the goal stays
active, nothing is sent and the continuation budget is not spent. Only
the exception type is stored. With redaction off, the message is
byte-identical.

README.md, config.example.yaml and the middleware docstring list the
continuation with the other redacted inputs.

tests/test_goal_continuation_pii.py checks the redacted objective,
reason and evidence summary against the tokens redact_text gives, the
unchanged message with redaction off or no config, the worker path
under the run config, and a redaction error.

* docs(changelog): link the goal outcome entries to #6556

Fill the four ([#PR]) placeholders in CHANGELOG.md and CHANGELOG_zh.md with this PR's number and add the matching link references.

* fix(frontend): keep an active goal stored without timestamps visible

POST /threads/{id}/state and run input can store an active goal without
created_at or updated_at. The backend still evaluates it and rejects an
edit with 409 (is_active_goal checks only status "active"), but
isGoalState required both timestamps, so history and live values
dropped it: both chat pages hid the goal bar and unlocked the last-turn
pencil, and only a /goal status read, which used a looser predicate,
brought them back.

isGoalState now accepts any record with a string objective and status
"active", and normalizeGoalStatusRead uses it, so thread values, live
updates and status reads share one rule. Malformed or inactive values
are still dropped.

Tests cover a sparse active goal from the history head and from a live
update before the post-run refetch (bar shown, hasGoal true), and a
malformed goal that stays hidden.

* test(goal): check the outcome write metadata the same way on every saver

PostgresSaver stores checkpoint metadata through
get_serializable_checkpoint_metadata, which drops "writes", so the
[postgres] case of test_goal_outcome_round_trips_through_the_sync_fallback
failed with KeyError: 'writes' when CI ran it against a live database.

The test now records the metadata write_thread_goal passes to put and
asserts the clear's and the next goal's "writes" there, for every saver.
The stored channel values are checked as before, and on sqlite and
cached-sqlite the stored "writes" must still equal what was passed.

* docs(agents): keep goal_outcome out of the agents guidance chain

The middlewares AGENTS.md chain is already over the 98,304-byte AG002
hard limit on main, and the PR-mode check fails any chain a PR touches,
so adding goal_outcome to the ThreadState field list in
deerflow/agents/AGENTS.md failed the agent-guidance job. goal_outcome
stays documented in runtime/AGENTS.md and backend/app/gateway/AGENTS.md,
which are outside that chain.

* refactor(goal): read the evaluator's evidence and redaction from one window

format_visible_conversation recomputed the window start that the redaction
pass takes from _evidence_window. Both now use _evidence_window, so a later
change to the cap or to what counts as visible moves both.

The window test now fills the window with tool-only replies and tool results.
Before, it caught a cap change in one copy but not a change to what counts
as visible.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-10 00:20:45 +08:00
zeng-bohan
c6ad5ed93c
fix(gateway): classify duration-only checkpoints without the Postgres-dropped marker (#5988)
* fix(gateway): classify duration-only checkpoints without the Postgres-dropped marker

On Postgres the savers funnel metadata through langgraph's
get_serializable_checkpoint_metadata, which pops writes before the row
lands, so persist_run_history_metadata's runtime_run_duration marker never
comes back and every replay/history read path treats metadata-only duration
leaves as addressable conversation state.

is_duration_only_checkpoint now classifies in two tiers:
- writes present (memory/SQLite) is authoritative — the per-writer key is
  exact, so a title leaf carrying runtime_interrupt_title with inherited
  run_durations is not a duration checkpoint;
- writes absent (Postgres) falls back to the surviving stamps
  (source == run_durations / run_message_ids keys).

Stamps alone are not exact: _ensure_interrupted_title and
_rollback_to_pre_run_checkpoint copy the head's metadata and inherit them.
Read paths tolerate that because every stamp-bearing leaf either copies its
parent's messages verbatim or restores an ancestor's exact message state, so
a skipped leaf only advances a message-presence scan onto a state-equivalent
checkpoint. Destructive callers opt out via stamp_fallback=False:
enforce_thread_retention keeps widening the class only through the
parent-shape check in _mark_duration_leaves_without_the_marker.

* fix(gateway): confirm the duration stamp fallback by parent shape

Review on #5988 surfaced a real false positive: write_thread_goal copies
the head's metadata and thereby inherits the run_durations/run_message_ids
stamps while changing the goal channel, and on Postgres its own writes.goal
marker is popped too — so the stamp fallback classified the goal leaf as
duration-only and the branch scan walked past it, copying channel values
from before the goal write.

The stamp fallback now also requires the duration writer's shape: it clones
the parent's channel_versions verbatim, while every state-changing leaf
(title, goal) bumps at least one version. Callers resolve the parent from
their own context — the lineage walk fetches the grandparent only for
stamp candidates, the history-window scans (branch, latest-turn,
regenerate filter, chronological fallback) index the window they already
hold — and stamps without a resolvable parent are refused: scanning one
state-equivalent copy is free, skipping a goal leaf is not.

Also corrects the mechanism claims from the first pass: langgraph's
update_state paths (rollback, compaction, manual updates) persist metadata
built from scratch and cannot inherit the stamps at all; the inheriting
writers are title and goal.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(gateway): share the stamp gate and reuse the walked grandparent

Review round 2 on #5988 — all three items, no functional changes:

- extract _has_postgres_stamps as the single gate shared by the classifier
  fallback tier and the walk probe, so widening one widens the other;
- carry the grandparent fetched for a stamp candidate into the next
  iteration (it is exactly the next parent), turning a chain of N Postgres
  duration leaves from ~2N+1 ancestor reads into N+1;
- pin the unresolvable-grandparent promise at walk level: a stamp
  candidate whose own parent link was pruned stays addressable and remains
  the replay base.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(gateway): resolve persisted channel versions for the duration stamp fallback

Materialized accessor snapshots expose no checkpoint payload, so the verbatim-copy shape check always failed open and Postgres duration leaves stayed addressable. Carry the version maps alongside the snapshots (raw snapshots expose them directly, graph-backed ones re-read the raw tuple through the accessor) and use them at every replay-base scan site.

* perf(gateway): resolve stamp-fallback versions lazily and once per tuple

The Postgres stamp fallback needs the raw channel versions for its shape check, and both window scans resolved them eagerly for every entry: a branch request paid up to 2N serial roundtrips for a window of N, and the regenerate scan resolved the same tuples a third time through resolve_history_versions.

Resolve them only for stamp candidates - the marker tier and the stamp gate decide without versions - and memoize by config identity, so a marker-decided window pays nothing and each distinct tuple pays at most one raw read.

The lineage walk had the same shape: a stamp candidate the shape check refuses re-read its grandparent as the next iteration's parent, because the prefetched slot was only filled when the candidate was skipped. Ancestor reads are now memoized by config identity, so no ancestor is fetched twice.

Dropped the dead checkpointer parameter of find_checkpoint_before_message_chronologically: no caller passed it, and wiring one in from an async handler would put a blocking get_tuple on the event loop.

---------

Co-authored-by: zeng-bohan <310902929+zeng-bohan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-09 20:30:19 +08:00
RongJie G
8b8fdf4387
fix(gateway): make resume retries idempotent (#6499)
* fix(gateway): make resume idempotency command-aware

Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>

* docs(gateway): document keyed resume rollout restrictions

---------

Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-09 17:11:00 +08:00
Daoyuan Li
734f9e0e92
fix(runtime): omit leading reasoning from run summaries (#6419) 2026-10-07 17:20:11 +08:00
Totoro
35cdcab1c5
feat(scheduled-tasks): signals, honest notices and fair queueing for server-started runs (RFC #6340, 2/2) (#6393)
* feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks

- Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal
  streak boundary) and stop_condition (the user's stop rule, own column).
- Stop rule is normalized (stop_rule.py) and appended only at launch; the
  stored prompt never contains it. Wording adapts when scheduler.tool_enabled
  is off.
- Willem 1: page-created tasks get goal evaluation, <standing_notes>, the
  previous-run reference and own-schedule stop; request_stop and the
  scheduled capability no longer require an origin conversation.
- Repository: single reactivation choke point raising
  ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop
  condition/note changes (not on resume); goal-check failures neither count
  nor reset the streak; pause of a finished task returns "finished"; trials
  on a paused task keep the host pause marker.
- New repository reads: automatic_runs_used_for, active_run_status_for,
  list_manageable_from_thread, thread_relation/thread_run on thread tasks,
  task_ids_for_thread, run_number, and run_number/total_tokens/summary on
  run history rows.
- REST: goal/max_runs/end_at/stop_condition on create and PATCH (null
  clears, naive end_at = task timezone), resume recomputes from now with an
  optional renewal body, once_time_passed, task_finished, limits_exhausted,
  scheduler_not_running on create, trigger returns outcome/existing/thread_id,
  task responses add automatic_runs_used and active_run_status.
- Coded errors {code, message, params} for routes, shared validation and the
  chat capability (scheduled_task_errors.py), pinned by
  contracts/scheduled_task_errors_contract.json; goal notes contract v2 with
  check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py).
- /api/features adds scheduled_tasks {available, running, tool_enabled,
  min_interval_seconds}; ScheduledTaskService.is_running.
- Docs: six README scheduler sections, API.md scheduled tasks section
  (breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs.
- Tests: migration 0031, stop rule, error contract, run listing, lifecycle
  over HTTP, repository, service and capability cases; updated tests for the
  new error shape, finished-task pause and head revision.

* feat(scheduler): conversation update/resume, run-conversation authority and readable launches

- schedule_task: update and resume actions, stop_condition and clear_fields
  parameters, plain parameter types (value rules come back as coded results),
  rewritten docstrings for schedule_task and stop_scheduled_task.
- Capability: update/resume share the REST validation and error codes;
  interactive turns in a task's run conversation may manage that task;
  scheduled runs keep only own-stop; grants require a running scheduler.
- Results: compact JSON-safe task views with local times, display card/text,
  last run in list, timezone_source/browser_timezone, now_local; no ISO, user,
  lease or origin fields; stop_instructions removed.
- Timezone: explicit > browser (context.client_timezone, validated in
  start_run and never forwarded to the run config) > not needed for intervals
  and offset run_at > timezone_required.
- Trial gate accepts natural direct requests after acknowledgements and
  particles; trial results report thread_id and existing.
- Scheduled launches: stable message id scheduled-<task_run_id>, server-owned
  deerflow_scheduled_origin metadata with the user-language parts, run thread
  title for fresh-thread tasks.
- Worker records goal continuations in the verdict; lead prompt language rule
  covers stored task text and scheduled runs.
- Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs.

* feat(frontend): scheduled-task core types, formatting, errors and glossary

- throwGatewayApiError now throws GatewayApiError (status, code, params,
  rawMessage); message stays the string detail or detail.message
- scheduled-task types gain stop_condition, automatic_runs_used,
  active_run_status, origin/notes, run_number/total_tokens/summary, thread
  relation, trigger result, renewal and scheduled origin; isTaskBusy and
  hasActiveRun helpers
- api: fetchScheduledTask, resume with optional renewal body, typed trigger
  result and update payload with null clears
- hooks: polling at 3 s while a run is active (list, thread list, run
  history), useScheduledTask with live gating and final 404, useInView,
  localized error toasts; run history refreshes tasks when a run finishes
- new pure modules: errors.ts (contract code to localized copy), format.ts
  (relative task times, viewer time, display zone), status.ts, actions.ts,
  run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked
  goal checks, describeTaskOutcome and parseStopRunId
- features: scheduled_tasks block with defaults when missing
- buildRunContext sends client_timezone
- titleOfThread takes a localized untitled label; breadcrumb names scheduled
  tasks and agents; reference chips use the localized untitled label
- schedule input: shared browserTimeZone, labelled selects, aria-pressed
  type toggles, interval minimum from the hint template
- i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes,
  recipe instructions moved into i18n
- e2e mocks follow the new task, trigger, resume, thread-task and features
  shapes; expectNoRawIdentifiers helpers for e2e and DOM tests
- tests for errors, format, cron, status, actions, goal outcome, run errors,
  hooks, run context, thread titles and features; scheduled e2e copy updated

* feat(frontend): list-first scheduled tasks page with outcomes and editable goals

- page.tsx is now a thin composition of components/workspace/scheduled-tasks:
  status tabs (All/Active/Paused/Finished) and search over the task list,
  ?task_id= deep link with router.replace on selection, ?thread_id= chip
  with Show all, empty state, scheduler-off notice and unavailable state
- task list rows show the schedule in words and one status line (next run,
  running now, paused by agent, auto-paused, finished by its cap)
- task detail: status badge, actions from availableActions with reasons,
  More actions menu (Duplicate, Copy task ID, Delete), outcome notice,
  Runs / Stops when / Each run's goal / Does / Notes from chat / History,
  origin chat link; success toasts for every action
- run rows: run number or Trial run, summary, goal and assumption badges,
  continuations, tokens, Open chat link; raw errors and the evaluator
  reason only behind Details
- create/edit/duplicate dialog with goal, stop condition, safety caps and
  advanced agent/context options; PATCH sends only changed fields, null
  clears a cap
- renew dialog raises or removes caps and resumes in one request; a
  limits_exhausted Resume opens it
- removed i18n keys the old page used; added feedback.resumedNoTime
- recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for
  the new page; DOM tests for the detail and the form dialog
- CHANGELOG entries and the frontend AGENTS.md scheduled-task note

* feat(frontend): live schedule card, run conversation provenance and header entry

- schedule_task results (create, update, pause, resume, trial, delete) become
  an assistant:scheduled-task message group rendered as a live card: schedule,
  next run, stop condition with the safety cap, goal, where results go, Run
  once now, Pause/Resume and Open task; the card polls the task only while it
  is on screen and shows deleted, scheduler-off and limits-exhausted states
- one card per task per assistant turn; the card group is not a turn boundary
  for usage, branching, editing, outline or skill attribution
- a human message with deerflow_scheduled_origin stays a human group with
  scheduledOrigin and renders as a "Scheduled run / Trial run" block with the
  task instructions, stop condition and notes collapsed; the launched text is
  never shown and the message is not editable
- readable step labels for schedule_task actions and stop_scheduled_task
- ThreadScheduledTasksButton replaces the header link: hidden without tasks,
  shows the count, links to the task (or the chat filter), and a run's chat
  links to its task
- recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread
  e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card,
  run block, header button, step labels and reconnect hydration
- i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and
  frontend AGENTS.md notes

* fix(scheduler): address verification findings

Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check.

* fix(scheduler): address review findings

* fix(gateway): no-transform on SSE responses

All four text/event-stream responses (thread run create stream, join,
existing-run GET/POST stream, stateless create stream) now take their
headers from app/gateway/sse_headers.py and send
Cache-Control: no-cache, no-transform, so compressing proxies such as
Next's rewrite proxy stop buffering streams. Content-Location and
X-Accel-Buffering are unchanged; scheduled and MCP-notification runs keep
stream_mode=None.

Tests pin the headers on every SSE route and fail on a literal no-cache
header left in routers. Docs: API.md SSE headers note, CHANGELOGs.

* feat(scheduler): per-owner run cap and fair queue selection

New scheduler.max_concurrent_runs_per_user (default 2, 0 = off, effective
min(cap, max_concurrent_runs)) caps launching/running scheduled runs per task
owner. claim_queued_run checks it right after the global count under the same
budget lock, so it is atomic across workers and Pods. list_queued_runs ranks
same-thread FIFO heads per owner with ROW_NUMBER() OVER (PARTITION BY owner)
and orders the batch by that rank, so the drain is owner-fair even with the
cap off; owners at their cap drop out before the batch limit. Orphan rows
(task deleted while queued) are never held back by the cap.

Gateway lifespan passes the startup value; the service constructor default
stays 0. Docs: config.example.yaml (config_version 54), CONFIGURATION.md,
the six READMEs, backend AGENTS.md, CHANGELOGs (behavior change flagged).
The live-task quota is untouched.

* feat(scheduler): durable lifecycle events for the originating chat

When a chat-created schedule is paused by the agent, auto-paused or finishes,
finalization now reports task_stopped / task_paused / task_finished, only on a
live -> paused/finished transition. The new finish_task_at_end_condition
replaces the three inline "mark completed" blocks (admission rejected as ended,
claim-time skip, complete_if_ended), so every site emits exactly once.

The scheduler's finalization observer is always installed. It writes
scheduled_task_events rows for the originating chat in the same transaction as
the state change, deduplicated on (task_id, anchor, event), with a title
snapshot, stop condition, run number, last run outcome and the chat's newest
run as placement anchor. Lookups fall back on bad stored data; only database
errors propagate. Manual trials drop only run_* events. The IM step keeps
PR1's selection until B4.

New GET /api/threads/{thread_id}/scheduled-task-events (owner check, caller's
rows, oldest first); rows survive task deletion and go with the chat.
Migration 0032_activity_and_task_events adds the events table plus the
run-origin column/index and read-state tables used later, each object guarded
on its own. Contract scheduled_goal_notes v3 pins the lifecycle vocabulary.
Docs: API.md, migrations chain, backend AGENTS.md, six READMEs, CHANGELOGs.

* feat(scheduler): honest, localized and self-contained IM notices

Every scheduled-task IM notice is now enqueued by the finalization observer
in the transaction that records the outcome, one message per occurrence
(task_stopped > task_paused > task_finished > run_failed > run_unmet >
run_completed; a once task sends only its run event). The merged auto-pause
replaces the separate unmet + pause messages, agent stops and finishes now
notify, and recovery-finalized runs notify exactly once. Idle finishes use a
deterministic idle: key derived from the lifecycle anchor. The old
non-transactional enqueue in handle_run_completion is removed.

New app/channels/capabilities.py holds CHANNEL_CAPABILITIES with a
proactive_notifications flag (adds the missing qq entry); only push-capable
providers (WeCom) get rows, /api/channels/providers returns the flag, and the
worker ends legacy rows for other providers as terminal failed.

New app/scheduler/notification_text.py renders en-US/zh-CN text: title, what
happened (plus the last run's outcome), the tasks page's one-line run summary
(redacted) when the agent replied, and a closing line. No IDs, no links, no
public_base_url. Locale is the owner's new `locale` preference, else the new
channel_connections.notification_locale (default en-US).

Docs: config.example.yaml (config_version 55), CONFIGURATION.md, API.md,
channels AGENTS.md, six READMEs, CHANGELOGs (behavior change flagged).

* feat(gateway): thread origin, activity feed and read markers for server-created threads

Runs started by the server for a user (scheduler, IM channels, GitHub
agents, extension handles, MCP notifications) now carry a server-owned
metadata.deerflow_origin {kind, provider?, namespace?} (new harness module
deerflow.runtime.run_origin, pinned by contracts/thread_origin_contract.json).
It is stamped only from request.state.run_origin or by the internal channel
caller; run admission drops client copies from both metadata and
config.metadata, and thread create/patch strip it. RunRepository derives
runs.origin_kind on put and on create_thread_operation_atomic (agent runs
only), and gains latest_change().

New ThreadReadRepository keeps a per-user, never-decreasing read position
(thread_read_markers) and a per-user read clock (thread_read_versions).
POST /api/threads/{id}/read marks a thread read, thread search items carry
unread, and thread delete removes the markers. GET /api/thread-activity pages
the caller's run changes over the run-change clock, returns only threads
changed by server-originated runs plus read_version, and 503s without SQL;
/api/features reports thread_activity.available.

Docs: API.md, gateway AGENTS.md (condensed to stay under its budget),
CHANGELOGs.

* fix(scheduler): adapt PR2 to PR1 follow-ups

PR1 moved the scheduled-task lifecycle rules out of backend/AGENTS.md into
the persistence guide and grew the sandbox AGENTS.md chain to within 267
bytes of its hard limit. The per-owner queue-fairness detail now lives in
the persistence guide next to the lifecycle rules (backend/AGENTS.md keeps
the cap and a link), so check_agent_guidance passes again.

PR1's goal-outcome unit test pinned scheduled_goal_notes_contract version 2;
PR2 bumps it to 3 (lifecycle vocabulary only, the v2 keys are unchanged).

* fix(scheduled-tasks): polish from live acceptance

- zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒,
  and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2
  days) instead of 每 1 分钟 or 每 24 小时; several units keep the number
- zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between
  the interpolated time and the Chinese after it; a copy test guards every
  scheduled-task zh string against a placeholder glued to Chinese
- run summary: when the first line of the final reply only leads in to a list
  (ends with ":" or ":"), the list items follow it as plain text (bullets,
  numbering, checkboxes, bold, code and links stripped), joined with ";" for
  CJK text and "; " otherwise, within the 160-character cap
- a schedule_task result (create, update, pause, resume, trial, delete) that
  arrives in a chat invalidates that chat's task list, the tasks list and the
  named task, so the header button and cards update without waiting for the
  idle poll; results already in the loaded history do not refetch
- e2e and unit fixtures are now the live acceptance recordings (DeepSeek
  Flash, demo data, fictional names): live-weekday.json (create, trial, edit,
  pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1
  and run 2, paused by the agent), fetched with GET from the demo Gateway and
  trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card,
  run-thread and tasks-page specs run in both locales over the recorded data

* docs(changelog): reference #6378 for the scheduled-tasks entries

* docs(agents): shorten the scheduled-task pointer to stay under the guidance limit

* test(scheduled-tasks): pin the live takes to their recorded time zone

The live fixtures were recorded in Asia/Shanghai, and an interval task
reads in the viewer's zone, so the 16:48 assertions failed on UTC CI
runners. Pin the browser zone in the two live-take e2e specs and the
zh live-take unit test.

* feat(frontend): thread origin, activity polling, read markers and task events core

Frontend core for server-created threads and schedule lifecycle events
(PR2 F1); the sidebar markers (F2) and the chat event line (F3) build on it.

- core/threads/origin.ts: threadOriginOf reads metadata.deerflow_origin,
  then channel_source (github -> GitHub origin), then a legacy
  scheduled_task_id; key and kinds pinned to
  contracts/thread_origin_contract.json. labelOfThreadOrigin and the now
  exported labelOfChannelProvider(provider, t) use the new localized
  threads.origin.providers names (飞书, 企业微信, ...), with the English map
  (gaining GitHub) only as the fallback.
- core/threads/activity.ts: useThreadActivity polls /api/thread-activity
  every 15 s (focus refetch, never in the background, off unless
  features.thread_activity.available). The first response only seeds the
  cursor; server-origin threads, a truncated page or a higher read_version
  invalidate the thread lists, a scheduled thread also ["scheduled-tasks"];
  an invalid cursor re-seeds. useMarkThreadRead debounces POST /read by 1 s,
  skips a thread whose cached unread is false, patches the search and
  infinite caches optimistically and stores the returned read_version so a
  tab's own read does not refetch its lists. AgentThread gains unread.
- core/scheduled-tasks/events.ts: ScheduledTaskEvent, the events query
  (["scheduled-tasks","thread-events",id], one fetch per saved chat, not
  gated on the task list so a deleted task's lines stay, refetched when a
  task of the chat changes status or finishes a run), describeTaskEvent
  (en/zh copy, title and condition as segments, last-run suffix, See that
  run / Open task) and placeTaskEvents (end of the after_run_id turn, else
  the tail, ordered by created_at); lifecycle names pinned to contract v3.
- features: thread_activity (default unavailable); channels:
  proactive_notifications on providers.
- Locale sync: the I18nProvider reports the UI language and a signed-in
  preference session saves it as the new `locale` preference when the
  account's differs (and on every switch), so IM notices follow the UI
  language; nothing is written with auth disabled.
- i18n: threads.* (unread, origin labels, provider names),
  scheduledTasks.events.*, history.waitingForSlot (now shown for queued run
  rows), channels.scheduledUpdates.*.
- e2e mocks: thread activity feed and controller, POST /read, the events
  route, providers with proactive_notifications, search unread, features
  thread_activity (off by default). Preference e2e mocks return the UI
  locale so they keep asserting only their own edits.
- Docs: frontend AGENTS.md, CHANGELOGs.

* fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in

- The edit, duplicate and renew dialogs keep a stored end time's instant
  while its field still shows it. A minute-precision wall value cannot
  keep seconds or name the later 01:30 of a DST fall-back, so a
  title-only edit moved the cap an hour earlier, and raising the run
  limit re-sent the end time without its seconds.
- Creating or duplicating a task from a chat's task view drops the chat
  filter when the new task does not belong to that chat, instead of
  showing "This task isn't one of this chat's tasks".

* feat(frontend): origin markers and unread state in the thread lists

Server-created threads (PR2 F2) now say who created them and whether they
are new, in the sidebar and on the Chats page, and reach an open sidebar
without a reload.

- ThreadOriginIcon (thread-channel-source.tsx): clock for a scheduled run,
  the provider mark for IM threads, the GitHub mark (new in
  ChannelProviderIcon) for GitHub threads, a puzzle piece for extensions;
  the localized "Scheduled run" / "From Feishu" / "From GitHub" / "From an
  extension" label is its accessible name and tooltip. ThreadChannelIcon
  stays exported as a wrapper, and the provider chip tooltip is localized
  (no more "Feishu channel").
- Unread dot after the title (aria-hidden dot plus sr-only "Unread"); the
  row aria-label becomes "{title}, unread" / "{title},未读". Never shown for
  the open thread.
- WorkspaceSidebar mounts useThreadActivity once.
- useMarkOpenThreadRead (components/workspace/chats): the default and
  custom-agent chat pages mark the thread read once its metadata has
  loaded, when its cached unread flips to true while open, when the tab
  becomes visible, and from onFinish (a send or a joined run ended). Each
  trigger goes through useMarkThreadRead, so it is debounced and a no-op
  while the cached unread is false.
- Tests: thread-origin-icon and recent-chat-list DOM tests (en/zh, active
  row, no raw identifiers), use-mark-open-thread-read DOM tests,
  e2e thread-activity.spec.ts (new scheduled run appears unread within one
  fake-clock poll without a reload, opening it posts /read and clears the
  dot, a read on another device clears it, IM and GitHub markers, zh
  labels, the Chats page, no polling when unavailable);
  thread-history.spec.ts now finds the localized origin icon.
- Docs: frontend AGENTS.md, CHANGELOGs.

* feat(frontend): schedule event line in the originating chat and channel notice support

The chat that created a schedule now says when the schedule paused or
ended, and the channel cards say which apps receive scheduled task updates
(PR2 F3).

- ScheduledTaskEventLine (mockup m2, marker 4): a dashed, muted row with
  the describeTaskEvent text (title in <strong>, stop condition clamped to
  one line with the full text in its tooltip, last-run suffix), the time
  via formatTaskTime and a right-aligned "See that run" (run chat) or
  "Open task" link. PauseCircle for paused, CircleCheck for finished and
  once-done, CircleX for once-failed. role="note", aria-label "Scheduled
  task update" / "定时任务通知", data-task-id and data-event-id only.
- MessageList takes scheduledTaskEvents and renders placeTaskEvents output
  through a new VirtualMessageList.renderAfterGroup, so each line sits at
  the end of its after_run_id turn inside the group's measured row (also
  when the list virtualizes); lines with nowhere to go render at the tail.
- Both chat pages load useThreadScheduledTaskEvents (not gated on the task
  list, so the lines stay after a task is deleted). The custom-agent page
  passes its agent so "See that run" opens the run on that agent's route.
- ChannelScheduledUpdates: "Scheduled task updates: sent here" / "... not
  available for this app yet" (定时任务通知:会发送到这里 / 此应用暂不支持)
  on the Settings provider cards and under each sidebar channel row; nothing
  when an older backend does not report proactive_notifications.
- e2e mock controller: setScheduledTasks.
- Tests: scheduled-task-event-line and channel-scheduled-updates DOM tests
  (every kind in en and zh, hrefs, clamp, no raw identifiers, Chinese
  title), MessageList placement DOM tests; e2e
  scheduled-task-chat-events.spec.ts (the line appears with the card's
  "Paused by agent" without a reload, See that run opens the run chat,
  anchored vs tail placement, two events in time order, the line survives
  task deletion and Open task shows the task is gone; en-3 and zh-1) and
  channels.spec.ts (WeCom sent here, Feishu not yet; en and zh).
- Docs: frontend AGENTS.md, CHANGELOGs.

* fix(scheduled-tasks): tie pause notices to the pause they describe

- Paused by agent links and times only the run named in last_error. When
  that run is not loaded, the notice shows no link and the task's own time
  instead of an earlier run that also asked to stop.
- The outcome notice reads the latest history page whichever page is
  shown, so browsing older runs no longer swaps the auto-pause reason and
  "Open latest run" to an earlier miss. It shares that page's request.

* fix(scheduler): address PR2 review findings

- Event lines: hold an event whose anchor run is on an unloaded history
  page instead of placing it at the bottom (hasMoreHistory); tail fallback
  only for fully loaded (branched/pruned) history or no anchor.
- Idle-finish anchor follows its reason: end:<iso> only for an end_at
  finish, seq:<n> otherwise, so a later end-time finish is not deduplicated
  away by an earlier max_runs finish.
- Observer takes FOR SHARE on the originating threads_meta row, so a racing
  thread delete cannot leave an orphan event line.
- One taskPagePath (core), re-exported by the card.
- Notification locales derived from NotificationLocale; /preferences uses it.
- IM notices drop generic reasons (goal_not_met_yet, unknown) instead of
  repeating the sentence.
- Localized provider names on Settings cards, sidebar rows and setup dialog.
- zh event lines: no space after a Chinese title (Latin-aware gap).
- Unread row label keeps the origin marker; chats page keeps the time as a
  description.
- Sidebar shows the scheduled-updates line only for connected apps;
  Settings says "available after you connect" for unconnected push apps.
- Event line keeps a real space before the time.

* fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes

- The edit form compares a cron in the form its schedule input emits, so an
  untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only
  edit no longer re-arms or refuses a finished task.
- Chat resume asks for a timezone, as update does, when a naive end_at would
  land in the UTC placeholder of a zone-free task.
- Re-arming a task clears a host pause marker, as resume does, so a later
  manual pause no longer reads as "Paused by agent".

* test(scheduler): consolidate PR2 backend test setup

- Fold the repeated task, origin-chat and observer setup of the
  lifecycle-event repository tests into one lifecycle_task builder.
- Build the per-owner-cap services through one _capped_service builder
  that keeps every cap explicit at the call site.
- Table the five SSE route header checks into one parametrized test with
  the same routes, bodies and Content-Location expectations.
- Reuse bind_wecom and hoist repeated local imports in the event tests.

Every case and assertion is kept; collected cases are unchanged.

* test(frontend): consolidate PR2 thread activity and origin test setup

Extract shared builders and helpers in the frontend tests PR2 adds, with
every case, input and assertion kept:

- activity.test.ts: seededClient/apply helpers and one LIST_KEYS constant
  replace the per-test seed, spy and apply boilerplate.
- mark-read.dom.test.tsx: newClient() and advance() replace the repeated
  QueryClient wrapper and fake-timer act blocks.
- origin.test.ts: the English and Chinese label checks become tables; the
  no-English-name check now runs on every Chinese row.
- thread-activity.spec.ts: a scheduledRun() builder replaces four inline
  copies of the scheduled run thread.
- channels.spec.ts: the PR2 block uses setLocaleCookie and one card helper.

Collected cases are unchanged (unit 211, e2e 60); no test was removed.

* fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages

useLatestScheduledTaskRuns kept React Query's default refetch on mount,
focus and reconnect, so going offline and back while an older history page
was shown fetched the latest page again. Refreshing stays with the history,
which refreshes the latest page only while it is shown.

* fix(scheduler): two gaps found in live acceptance

- A fresh scheduled run thread is named when it launches. The worker
  copied the title to the thread list only when the run ended, so the
  sidebar, which now shows the new run thread at once, listed it as
  untitled while it ran.
- A task capped at one run finishes with "its one run is done" /
  "唯一一次运行已完成" in the chat event line and the IM notice, instead
  of "all 1 runs are done".

* test(scheduled-tasks): let the viewer's time carry its own day label

Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days,
so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the
fixed "01:00 · 09:00 your time" assertion failed depending on the hour.

* test(scheduler): expect the one-run finish copy in service notices

* fix(gateway): create a scheduled run thread with its title

Pre-setting the name after start_run raced the thread row, which the run
worker creates asynchronously: live, one of two run threads still reached
the sidebar untitled. Run admission now creates a new thread with the title
its run input carries, which the worker would copy only at run end.

* docs(changelog): reference #6393 for the scheduled-task signal entries

* fix(scheduled-tasks): route event-line run links by the agent that ran them

The originating chat's agent was used for "See that run", but a task's
agent can change after the chat created it, so the link could open the run
chat on another agent's route. Lifecycle events now record run_agent_name,
the task's agent when the run finished (edits are refused while a run is
active), and the line routes by it.

* fix(scheduled-tasks): load up to the route's maximum of chat event lines

The chat loads its lifecycle events once, and the route's default of 50
would silently drop the oldest lines of a chat with many schedules. Ask
for the maximum, 200.

* fix(threads): keep activity reads and the first cursor in step with the lists

- Accepting the first activity cursor refreshes the thread lists once. The
  lists load on their own, possibly before a run the seed already counts,
  and later polls start after it, so such a thread never appeared.
- A local read whose returned read clock jumps past that one read also
  covers reads on other devices; the lists are refreshed instead of the
  clock silently skipping those threads' stale unread dots.
- An invalid cursor re-seeds through the same path, so it refreshes once.

* docs(agents): bring the middlewares guidance chain back under the hard limit

#6378 and #6363 each fit the budget, but together the middlewares AGENTS.md
chain on main is 98,425 bytes, 121 over the 98,304 hard limit, so Lint
Check fails on main. Move the scheduled-task recovery, launch-update and
queue-timeout rules from backend/AGENTS.md into the persistence guide's
lifecycle section, which backend/AGENTS.md already points to. The chain is
now 98,067 bytes.

* fix(helm): bump the chart's config_version to 55

config.example.yaml moved to 55 in this PR, so the Helm chart's
validate-chart drift check failed. The chart's embedded config needs no
field changes: the new scheduler and channel keys have defaults.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-07 10:22:15 +08:00
Hyeonsang Cho
c540399a5e
fix(gateway): scope per-run reads to the run's data identity (#6282)
* fix(gateway): scope per-run message and event reads to the run's data identity

start_run stamps run-event rows with the raw trusted owner id, but
GET /runs/{rid}/messages and /runs/{rid}/events called the event store with
user_id=AUTO, which resolves to the internal caller's make_safe_user_id form.
On the database event store, owners whose id contains characters outside
[A-Za-z0-9_-] (IM channel ids such as "feishu:owner-777") therefore got an
empty list. The regenerate source-run scan had the same mismatch and fell
back or returned 409.

Pass _run_scope_user_id() explicitly, as the thread message routes already
do (#5448), and add user_id to RunEventStore.list_messages_by_run() so every
backend accepts it like list_messages()/list_events(). Browser and API
sessions keep their per-user filter.

* fix(gateway): scope workspace-changes and archive reads to the run's data identity

The same raw-owner mismatch reached two more run-scoped routes. On the SQL
stores, GET /runs/{rid}/workspace-changes reported no changes because its
event lookup used user_id=AUTO, and GET /runs/{rid}/artifacts/archive
answered 404 because both its run-row lookup and its run.delivery event
lookup did.

Pass _run_scope_user_id() to both, and let get_workspace_changes_response()
forward an explicit user_id to the event store.

* docs(changelog): reference #6282 in the run-scope read fix entries

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 10:23:55 +08:00
quifox
06e9bfd269
fix: propagate current run failures from wait endpoints (#6217)
* fix: propagate current run failures from wait endpoints

* docs: clarify checkpoint wait errors and SDK behavior
2026-10-04 07:46:59 +08:00
ZJPex
f33b4fb4bf
fix(gateway): preserve clarification answers on regenerate (#5544) 2026-09-19 10:18:13 +08:00
xiaodu55
16f154f32b
fix(gateway): preserve owner isolation when thread metadata is missing (#5484)
* fix(gateway): preserve owner isolation when thread metadata is missing

Follow-up to the #5448 review P1 (post-merge finding): owner_check=True
also authorizes threads whose meta row is missing (legacy compatibility)
or NULL-owner (shared/pre-auth data). _run_scope_user_id returned None
for every trusted internal caller, which dropped the only remaining
per-user filter on those threads and let an internal caller acting for
owner A read owner B's persisted runs.

_run_scope_user_id now takes the thread_id and consults the thread meta
store: when an existing meta row establishes ownership, the authorized
thread's runs are still read unfiltered (merged #5448 semantics,
including owner-header-less internal callers); when the meta row is
missing or NULL-owner, the filter falls back to the acting owner's raw
stamp (the exact value start_run writes) — or the synthetic "default"
identity without an owner header — so cross-user runs stay hidden.

Isolation coverage uses the real MemoryThreadMetaStore with no metadata
row (and a NULL-owner row) plus another user's persisted run: /runs and
/runs/page must be empty and /runs/{run_id} must 404 for internal
callers, while an established-ownership thread keeps the unfiltered
read.

* fix(gateway): gate run-scoped sub-resource reads for internal callers

Review follow-up on #5484: the P1 owner-isolation class remained
reachable through run-scoped sibling reads that apply no per-user filter
at all — /runs/{run_id}/messages, /events, /join, /stream and
/workspace-changes query by (thread_id, run_id) directly, so on
missing/NULL-owner threads an internal caller acting for owner A could
still read owner B's run content by id (verified 200 at the previous
head).

- Extract _thread_ownership_established (shared meta-row check) and add
  _require_run_visible_to_scope: for internal callers on threads without
  established ownership, the run's own user_id stamp must match the
  acting owner's raw value (or the legacy "default" stamp) or the read
  404s. Established-ownership threads and every non-internal caller keep
  their existing thread-scoped semantics.
- Wire the gate into join, stream, messages, events and
  workspace-changes; reword the now-stale messages comment to track the
  new scoping semantics.

Regression tests: sub-resource reads 404 for a mismatched internal
owner while the matching owner reads them normally, and the owner-less
fallback branch (synthetic "default" filter on missing-meta threads) is
pinned. Red confirmed against the pre-gate head.

* fix(gateway): gate cancel and artifact archive for internal callers

Review follow-up on #5484 round 2: POST /cancel resolved runs unscoped
(require_existing=True only closes the missing-meta case — NULL-owner
meta rows still pass), so an internal caller acting for a different
owner could interrupt another owner's active run on a shared thread
while /join and /stream were already gated. The archive manifest and
download pair likewise leaked the other owner's delivered-file count
and a 200-vs-409 delivery oracle on NULL-owner threads (missing-meta
threads were already denied by require_existing=True).

All three routes now call _require_run_visible_to_scope; its docstring
records the extended coverage. NULL-owner-thread regression tests pin:
a mismatched internal owner gets 404 from cancel, manifest and archive
download, while the acting owner reaches the real conflict path (409 on
a terminal run) and reads the manifest (file_count 2).

* fix(gateway): tolerate state-less request stand-ins in the scope helpers

The new owner-isolation gate and _run_scope_user_id read request.state
directly, which crashed the FakeRequest-based unit suites for the run
events, workspace-changes and scope endpoints (backend-unit-tests shards
1/2/4 on #5484). Read the state object defensively first: a request
without state is simply not an internal caller, so those paths keep
their pre-gate semantics.

* fix(gateway): scope the thread token-usage aggregate by owner

Review follow-up on #5484 round 4: GET /{thread_id}/token-usage called
aggregate_tokens_by_thread(thread_id) with no user filter at all, so on
missing/NULL-owner threads an internal caller acting for owner A read
owner B's spend, model names, run count and (with include_active=true)
live activity; the NULL-owner variant reached browser sessions too.
build_context_usage's latest-model lookup was unfiltered as well.

aggregate_tokens_by_thread gains an optional user_id (mirroring
list_by_thread: explicit None = unfiltered, AUTO resolves the contextvar)
in the memory store, the SQL repository and the store base;
build_context_usage/_resolve_thread_model_name thread the scope through
the latest-run lookup; the token-usage endpoint passes
_run_scope_user_id's value. Established-ownership threads aggregate
unfiltered as before; shared/missing-meta threads narrow to the acting
identity. Stale helper-test comment reworded after the #5482 merge
adaptation.

* test(gateway): pin the unfiltered aggregate on established-ownership threads

Review follow-up on #5484 round 5: the established-ownership branch of
the token-usage scoping (store receives user_id=None) was the only
unpinned half of the contract — the round-4 call-assertions never set
app.state.thread_store, so their None came from the user-less stand-in
path. test_token_usage_unfiltered_on_established_ownership_for_
internal_callers seeds an established meta row plus runs stamped by two
different identities and asserts the totals fold (166 = 111 + 55);
together with the isolation tests it now catches both failure modes
(always-stamp narrowing and always-None leak).
2026-09-18 09:39:14 +08:00
xiaodu55
e493390aea
fix(gateway): scope edit/regenerate helper fallbacks by data identity (#5483)
The message edit/regenerate prepare chain resolves source runs through
_resolve_run_id_for_message, _require_successful_source_run and
_find_interrupted_target_run_id, which filtered by the authorization
identity (get_current_user) — the same conflation fixed for the runs and
messages read endpoints in #5448, left out of scope there. For trusted
internal callers the authorization identity never matches the raw
owner-stamped run rows, so regenerate/edit-regenerate prepare fails with
409 on threads the caller is authorized on (#5482).

The three helpers now resolve their filter id through _run_scope_user_id
as well; both prepare endpoints keep their owner_check=True
authorization and browser/API sessions keep the per-user filter.

Regression tests extend test_thread_runs_internal_scope.py to the helper
fallback paths: internal callers resolve raw-owner-stamped runs
(including the interrupted-run and status-409 paths), browser sessions
keep the per-user filter and 409 on cross-user runs.
2026-09-16 23:32:57 +08:00
xiaodu55
0745fb268f
fix(gateway): scope runs read endpoints by data identity, not authorization identity (#5448)
* fix(gateway): scope runs read endpoints by data identity, not authorization identity

Trusted internal callers are authorized as a synthetic internal user
(id "default", or the make_safe_user_id-normalized owner with an owner
header), while start_run stamps run rows with the raw trusted-owner value.
list_runs / get_run / list_runs_page filtered by the authorization identity,
so the store-side user filter never matched and internal callers always saw
an empty runs list (or 404) for threads they are authorized to read.

The three read endpoints now resolve their filter id through
_run_scope_user_id: internal-role callers skip the per-user filter (thread
visibility is already authorized by owner_check=True), and browser/API
sessions keep the existing per-user filter unchanged.

Regression tests cover both identities across the three endpoints (list,
keyset page, single get) with a MemoryRunStore seeded with mixed-owner rows;
without the fix the four internal-caller cases fail while the
browser-session isolation case passes.

* fix(gateway): route the message read endpoints through the same data-identity scoping

Review follow-up on #5448: list_thread_messages and list_thread_messages_page
resolved get_current_user and passed it as the data filter to the event-store
scan, hidden-run lookups, turn-duration injection and the feedback queries —
the same authorization-vs-data identity conflation fixed for the runs
endpoints, leaving the #5437 empty-read symptom in place for lossy owner
values. Both endpoints now resolve their filter id through
_run_scope_user_id as well.

Regression tests extend to the two message endpoints, asserting the resolved
filter identity at the runs-store and feedback-repo boundaries (None for
internal callers, the session user id for browser sessions).

* fix(feedback): deterministic per-run collapse for unfiltered feedback reads

Review follow-up on #5448: with _run_scope_user_id returning None for
internal callers, the feedback lookups now receive an explicit-None user id,
which skips the user_id WHERE in FeedbackRepository. On shared/NULL-owner
threads several browser users can hold feedback on the same run, and
list_by_thread_grouped / list_by_run_ids collapsed rows per run_id via a
dict comprehension over unordered results — the feedback attached to the
last AI message would be an arbitrary user's row.

Both methods now order by created_at ASC with feedback_id as the tie-break,
so the collapse deterministically keeps the most recently created feedback.
_run_scope_user_id's docstring now documents that the resolved id also
scopes feedback and event-store reads, not just run rows.

Regression test seeds multi-user feedback on one run and asserts the
collapse outcome is stable across repeated unfiltered reads.

* docs(feedback): the collapse keeps the most recently written feedback

created_at is refreshed on upsert, so the surviving row per run is the
most recently written (created or updated), not the most recently
created — align both docstrings with the ordering key's actual semantics.
2026-09-16 15:56:25 +08:00
Totoro
533e30e7f2
[feat] add opt-in conversation reads for Gateway runs (#5399)
* feat: add scoped conversation reads to gateway runs

* refactor: share conversation tool path and reuse parsed text

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
2026-09-14 10:51:01 +08:00
Shxiao
572744975d
fix(tools): run tool assembly off-loop at async entry points (#5224)
* fix(tools): run tool assembly off-loop at async entry points

get_available_tools() may block on MCP cache initialization while it is
called on async agent-assembly paths (task_tool, durable batch execution),
stalling the calling event loop for the full discovery duration.

Dispatch the (unchanged, synchronous) assembly call to a worker thread via
asyncio.to_thread at the two async entry points so the loop keeps processing
requests, SSE frames, cancellations, and timers.

Fixes #5172

* fix(tools): offload lead-agent assembly off-loop and pin with blocking-io anchors

Review follow-up for #5224:

- run_agent now dispatches agent_factory(...) through asyncio.to_thread, so
  lead-agent assembly (including both get_available_tools call sites in
  _assemble_lead_agent) runs off the event loop — the Gateway headline
  scenario from issue #5172.
- _ensure_sync_invocable_tool takes a double-checked threading.Lock, making
  the in-place tool.func wrap on the shared tool singletons explicitly
  single-shot now that assembly can run concurrently on worker threads.
- Add backend/tests/blocking_io/test_tool_assembly_offloop.py: blocking-probe
  anchors for task_tool and SubagentBatchService._execute_item under the
  strict Blockbuster gate, plus a meta-check proving the gate trips on the
  exact syscall class (ExtensionsConfig.from_file on the loop). Verified the
  anchor goes red when the offload is flattened back to a plain call.

* fix(gateway): build checkpoint state accessor off-loop; anchor run_agent offload

Review follow-up for #5224:

- Add abuild_checkpoint_state_accessor (asyncio.to_thread around the
  unchanged sync builder) and switch every async call site to it: the
  stateless_wait route, thread_runs, both threads call sites, and the
  build_thread_checkpoint_state_accessor boundary. The agent-factory
  assembly re-enters get_available_tools() and may block on MCP cache
  initialization; repeat calls hit _state_accessor_graph_cache and only
  pay the thread hop.
- Add a third blocking-io anchor driving the real run_agent with minimal
  RunManager/bridge stubs; the factory performs a real production blocking
  read (ExtensionsConfig.from_file()) and the test asserts assembly never
  runs on the main thread. Verified the anchor goes red when the run_agent
  offload is flattened back to a plain call.
- Adapt the test_threads_router checkpoint-builder patch sites to the new
  async name.

* refactor(tools): carry assembly offloads on a dedicated bounded pool

Review follow-up for #5224:

- Add utils/assembly_io.py: a dedicated ThreadPoolExecutor (default 8
  workers, DEER_FLOW_ASSEMBLY_WORKERS-overridable, mirroring
  utils/file_io.py and tools/sync.py) with run_assembly(), which copies
  contextvars explicitly. A hung stdio MCP server parks its worker for
  the full MCP timeout; carrying assembly hops on the loop's default
  executor would let a few parked assemblies queue every other
  to_thread/run_in_executor(None, ...) caller behind them.
- Switch all four offloads (run_agent, task_tool, batch _execute_item,
  abuild_checkpoint_state_accessor) to run_assembly().
- State the cold-path behavior in the accessor docstring: the graph
  cache validates factory identity, so non-identity-stable factories may
  duplicate lead-agent assembly across concurrent readers (MCP discovery
  stays process-wide single-flight); the pool bounds the duplicates.
- Add a fourth blocking-io anchor driving build_thread_checkpoint_state_
  accessor with a per-resolution fresh factory (always a cache miss) and
  the real production blocking read; enumerate all four offloads in the
  gate's module docstring. Verified the anchor goes red when
  abuild_checkpoint_state_accessor is flattened back to a plain call.

* fix(subagents): revalidate batch item before launch; make assembly pool observable

Review follow-up for #5224:

- _execute_item() revalidates the durable state right after assembly and
  before executor.execute_async(): renew_item_lease() returns valid=False
  when cancel_batch() terminalized the item or the lease was lost while
  assembly was parked, and the launch is skipped (the canceller already
  finalized the item). Previously the launch was unconditional and the
  poll loop's cancellation checks only started after execution began.
- Regression test driving the real SQLite repository: a blocking assembly
  probe parks _execute_item, cancel_batch() lands, and the launch is
  skipped with the item staying cancelled. Verified the test goes red
  when the revalidation is removed.
- run_assembly() tracks pending assemblies and logs a throttled WARNING
  once the pending count exceeds the worker count, so assembly starvation
  (workers parked on a hung MCP server) is distinguishable from idle.
- The run_agent blocking-io anchor now binds a sentinel extension
  snapshot via ctx.extensions and asserts the factory observed it through
  get_agent_build_extensions(), pinning run_assembly()'s ContextVar
  propagation. Verified red when ctx.run is dropped.
- Document the assembly pool in backend/AGENTS.md.

* fix(utils): decrement the assembly pending count on the pool thread

The pending-assembly counter behind the starvation warning decremented
from the asyncio future's done callback, which never fires once the
submitting loop is closed while its worker is still running: the count
ratcheted up permanently and eventually fired the starvation warning
with no starvation behind it (reproduced at 97dc9bec by review).

Decrement instead from the dispatched work item: run_assembly() wraps
func so a finally drops the count under the pending lock on the pool
thread, and the done callback is gone.

Pin the counter with tests/test_assembly_io.py: a healthy call returns
the count to zero, and an abandoned loop (stopped while the worker is
parked) does not wedge it — the abandoned case goes red against the old
done-callback decrement.

* docs(utils): fix the pending-counter comment after the decrement move

The comment still described the removed done-callback decrement,
contradicting _work()'s own comment; state the actual mechanism
(increment on the loop before dispatch, decrement from the dispatched
work item's finally on a pool thread).

* test(gateway): retarget checkpoint-accessor stubs to the services seam

thread_runs and runs now call abuild_checkpoint_state_accessor, so the
upstream wait-reader, regenerate-prepare, and idempotency tests must stub
the sync builder where abuild resolves it (app.gateway.services); stubbing
the removed router re-exports fails with AttributeError at setup. The
async seam semantics are unchanged: run_assembly invokes the stubbed
sync builder off-loop and propagates its return values and exceptions.

Move the agent/tool assembly off-load note from backend/AGENTS.md to
deerflow/utils/AGENTS.md (next to assembly_io.py) so the effective
instruction chain for agents/middlewares no longer grows past the AG002
hard limit.

* fix(runtime): serialize same-key accessor assembly and release queued-cancel slots

Address the three review follow-ups on the assembly off-load:

- assembly_io: a job cancelled while still queued never runs its work
  item, so the dispatched finally never fired and _pending_assemblies
  stayed elevated until a false starvation warning. Exactly-once cleanup
  now rides the concurrent future's cancelled() state — cancel() only
  succeeds before the executor starts the item, so cancelled() is true
  precisely when the finally will never run — plus a submit-failure
  release; the one-worker queued-cancellation case is pinned red/green.
- services: overlapping cold readers sharing one cache key could both
  run full agent assembly. _state_accessor_graph now serializes per key
  through a thread-side KeyedLockTable (pool threads, no running loop)
  and re-validates factory/app-config identity under the lock, so the
  factory runs exactly once while identity changes still rebuild. Cache
  dict access is lock-guarded now that construction runs off-loop.
- guidance inventory: register deerflow/utils/AGENTS.md in
  EXPECTED_GUIDANCE_PATHS so test_repository_has_the_approved_scoped_
  guidance_shape matches the relocated assembly note (CI shard 4).

* test(keyed-lock): pin KeyedLockTable reclamation and waiter bypass directly

Thread-side counterparts of the async table's own tests: overlapping
hold() calls serialize (a late arrival joins the live entry instead of
creating a second lock that bypasses a queued waiter), the last check-in
pops the entry, and many unique keys leave the registry empty. Both
regressions verified red — popping unconditionally trips the late-arrival
test, never reclaiming trips the many-keys test.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-12 07:41:24 +08:00
wutongyuonce
3c7d3303d3
feat(gateway): paginate thread run history (#5283)
* feat(gateway): paginate thread run history (#5282)

GET /api/threads/{thread_id}/runs stays a bare array of the newest 100
runs so LangGraph SDK clients keep working. Add GET /runs/page with a
(created_at, run_id) keyset cursor so callers can walk older history.

* fix(gateway): reject one-sided run history cursors

RunManager.list_by_thread now raises if only one of before_created_at
or before_run_id is set, matching the HTTP 422. Document the per-page
sort cost on the SQL keyset query, and add the missing CHANGELOG
[#5282] link definition.

* fix(gateway): round-trip run page cursors through query strings

Emit next_before_created_at with a Z suffix so '+' is not decoded as a
space. Accept that space, and Z, when parsing. Treat blank cursor
fields as absent and reject a non-ISO before_created_at in RunManager
so a harness caller cannot silently restart at the newest page.

* style(gateway): ruff-format run page cursor files

Collapse the one-sided cursor ValueError and the two before_created_at
asserts so ruff format --check passes at line-length 240.
2026-09-09 14:56:31 +08:00
wutongyuonce
9ad79baf97
feat(gateway): support idempotent thread runs (#5258)
* feat(gateway): support idempotent thread runs

Accept Idempotency-Key on thread-scoped create, stream, and wait endpoints, scoped by owner and thread before durable admission.\n\nRefs #5257.

* fix(gateway): handle idempotent run reuse on wait and stream

Reused store-only records have no local task. /wait now waits on the
bridge when it can observe the stream, and otherwise returns durable
status instead of a stale checkpoint. A reused terminal stream that has
been evicted emits gap/reload_durable_state. Replay is bound to the
original input and assistant_id.

* fix(gateway): 409 reused in-flight streams on this worker

A store-only running record on a process-local bridge has no owner
stream. POST /runs/stream used to subscribe anyway, which created an
empty log and waited forever. Match join: 409 unless the run is already
terminal, so missing-stream retries can still emit gap.

* fix(gateway): keep observer joins off the idempotent stream-gap path

sse_consumer keyed missing-stream gap on the sticky
idempotency_reused flag, so a later join of a terminal run inherited
it. Gate that branch on apply_on_disconnect, which already separates
creating streams from joins. Document the retry outcomes clients have
to handle.

* fix(gateway): gate missing-stream gap on creating retry

Reuse apply_on_disconnect to pick gap vs end changed sse_consumer default path, so a missing stream started returning gap for default callers and for out-of-scope POST /api/runs/stream. Keep that branch behind emit_gap_on_missing_stream and pass it only from thread-scoped /runs/stream on this request reuse.

* fix(gateway): keep wait reuse off later checkpoints

Direct handler calls were crashing because FastAPI Header() leaked in as the Python default. Bind Idempotency-Key with Annotated so the default is None, and ignore non-str keys.

A reused completed /wait was still reading the latest thread checkpoint. After a later run on the same thread that is the later run's result. Return durable status instead of claiming the head as this run's output.

* fix(gateway): snapshot wait reuse and refresh store status

idempotency_reused lives on the shared cached record. Capture it before awaiting completion so an overlapping retry cannot suppress the original creating /wait checkpoint.

A store-only peer record still holds admission-time status after the owner publishes END. Refresh durable status/error before returning them.
2026-09-08 14:40:05 +08:00
Ryker_Feng
8d8ca506ba
feat(artifacts): download run files as zip (#5117)
* feat(artifacts): download run files as zip

* fix(artifacts): address archive review feedback

* fix(artifacts): gate unavailable archive downloads

* fix(artifacts): verify archive availability

* fix(artifacts): harden archive consistency

* fix(artifacts): reject archive path aliases
2026-09-01 22:01:21 +08:00
Sunshine
a956bbc030
fix(runs): reject cancel actions on GET stream joins (#5092)
* fix(runs): reject cancel actions on GET stream joins

stream_existing_run is registered for both GET and POST, and its
?action=interrupt|rollback branch cancels the run. The CSRF middleware
exempts GET, so a session-authenticated browser could be forced
cross-site (img/script/top-level navigation) into
GET /api/threads/{id}/runs/{run_id}/stream?action=interrupt|rollback —
a state-changing GET that bypasses the CSRF protection guarding the
POST variant. Introduced with the dual registration in #1403.

The handler's docstring already documents cancel-then-stream as
POST-only (the LangGraph SDK's joinStream/useStream stop button uses
POST); enforce it: GET with an action answers 405, action-less GET
joins and POST cancel-then-stream are unchanged.

Regression drives the real router: GET+action is 405 with the run left
running, plain GET join still streams, POST+action still cancels.

* fix(runs): scope the 405 detail to the action requirement

"GET is a read-only stream join" overstates the current main: on a
locally-owned run with the default on_disconnect=cancel, a GET join's
disconnect can still trigger cancellation. That observer-disconnect
vector is closed by #5041; the detail here should only claim what this
guard enforces.

* fix(runs): harden GET stream action rejection

* fix(runs): align stream schema with method contract

* test(runs): pin GET stream action 405 through the production stack

Review follow-up (defence-in-depth): the GET-action suite drove bare
FastAPI() apps, so nothing pinned that a session-authenticated
cross-site GET reaches the route gate at all once CSRF exempts the
safe method. test_pat_auth.py already assembles the production
middleware order (AuthMiddleware inner, CSRFMiddleware outer), so its
mirror app now registers the real _reject_get_stream_action
dependency on a GET join route.

The new case pins the end-to-end premise: an authenticated GET
?action=interrupt is answered 405 + Allow: POST by the production
route dependency, while the same unauthenticated GET dies at
AuthMiddleware's 401 before any route logic runs.

Validation: focused suites (test_pat_auth, test_stream_get_action,
test_csrf_middleware) — 62 passed; ruff check + format clean; the new
case errors on the pre-fix baseline (guard absent), confirming the
pin.
2026-09-01 15:51:37 +08:00
Sunshine
bf740ffa90
feat(auth): add personal access tokens for programmatic API access (#5041)
* feat(auth): add personal access tokens for programmatic API access (#4849)

Backend-first implementation of the PAT contract from #4849: show-once
dfp_ tokens bound to their owning user (AUTH_SOURCE_PAT,
is_internal=false), digest-only storage (migration 0017), strict
credential precedence (invalid Bearer is a 401, never cookie fallback),
CSRF double-submit skipped only for Bearer requests while
auth-endpoint origin checks still run, scopes intersecting the authz
route permissions, session-auth-only PAT management and password
changes, and throttled best-effort last_used_at stamps.

* fix(auth): harden PAT scope boundary and schema parity from adversarial review

Independent review of the initial draft found: (1) scopes only constrained
the threads/runs permission axis while admin routes treated a PAT as its
(possibly admin) owner — is_admin_user now rejects PAT callers outright
since no scope grants admin capability; (2) the model declared a column
UNIQUE constraint while migration 0017 created a named unique index, so
downgrade failed on create_all-bootstrapped DBs — both now use the named
unique index; (3) auth-disabled mode is an operator override and now stays
ahead of the Bearer check so a stray Authorization header cannot 401 an
E2E sandbox; plus wiring the previously-unused constants, bounding the
last_used_at stamp cache, and four new tests (middleware-level expiry,
expires_in_days, admin-capability rejection with session control, and the
auth-disabled precedence).

* docs(api): document personal access tokens for programmatic API access

* fix(auth): close PAT security boundaries from review (default-deny routes, extension admin suppression)

P1-1: scope intersection only constrains @require_permission routes, so
undecorated mutation routes (DELETE /api/memory, POST /api/agents, Lark
credential switching, channel config) accepted a PAT holding a single read
scope. AuthMiddleware now enforces a default-deny route policy in
auth/pat.py: PAT requests are admitted only to the thread/run lifecycle
routes the v1 scopes govern; everything else answers 403 regardless of
scopes. Session-cookie callers are unaffected.

P1-2: the extension principal resolver projected is_admin/roles from the
raw system_role, so an admin-owned PAT passed
deerflow_extension_api.require_admin on contributed routes despite the
documented no-admin guarantee. The projection is now PAT-aware and
suppresses every admin signal for PAT callers, mirroring
deps.is_admin_user.

Both fixes carry regression tests (route outside policy 403 + session
control; production resolver admin suppression), and API.md documents the
default-deny boundary.

* fix(auth): enforce PAT scopes on stateless run entry and harden decorator

Follow-up hardening from an independent audit of the P1 fixes:

- POST /api/runs/stream and /api/runs/wait were the only allowlisted run
  entrypoints without @require_permission, so a threads:read-only PAT
  could still start runs (same bug class as P1-1, now closed): both now
  carry @require_permission("runs", "create"). POST /api/threads and
  POST /api/threads/search gain threads:write / threads:read for the
  same reason. Authorization-disabled deployments see no change (the
  permission set resolves to all permissions).
- require_permission now binds the wrapped signature to locate a
  positionally-passed request before injecting the test stub, fixing
  'got multiple values for argument' on direct positional unit-test
  calls.
- API.md: the intro PAT example used GET /api/models, which the new
  default-deny policy 403s — replaced with GET /api/threads; the
  default-deny route list now spells out method sets.

Regression test: threads:read-only PAT is 403 on the decorated stateless
entry while a runs:create PAT passes.

* fix(auth): address review P2s (empty Authorization header, PAT name trimming, API example)

- CSRFMiddleware treats an explicitly empty Authorization header as
  present (is None), so an invalid credential always reaches
  AuthMiddleware's uniform 401 instead of a CSRF 403 that varies by
  method/CSRF state. Regression: empty-header request dies at auth.
- PATCreateRequest strips the name and rejects whitespace-only values
  before token generation; created names are stored trimmed.
- API.md intro PAT example now uses the implemented
  POST /api/threads/search endpoint (GET /api/threads does not exist).
- AGENTS.md trimmed back under the guidance soft budget after the
  upstream merge.

* fix(auth): tighten PAT route policy to implemented methods only

The allowlist admitted GET /api/threads, a method no router implements.
Pre-authorizing a dead method weakens the default-deny boundary: a
future GET collection route added without a permission decorator would
become PAT-reachable without an explicit policy change. Restrict the
rule to POST, fix the stale GET description in API.md's PAT
constraints, and document the default-deny boundary accurately in the
gateway AGENTS.md guidance (only the threads/runs allowlist is
PAT-reachable; every other authenticated route 403s PAT callers).

Audited every remaining rule against the mounted routers: all other
method+path entries map to real routes. Regression:
test_pat_policy_does_not_pre_authorize_unimplemented_methods.

* test(auth): guarantee the negative digest test mutates the token

token[:-1] + "X" is identical to the original whenever the generated
token already ends in X (1/62), making the negative digest assertion
fail intermittently. Choose the replacement character based on the
existing tail so the mutated token always differs.

* fix(auth): require runs:cancel for cancel-then-stream requests

stream_existing_run is gated at runs:read so action-less stream joins
work with read-only credentials, but its ?action=interrupt|rollback
branch cancels the run — a separate permission. A runs:read-only PAT
passed both the PAT route policy and the route decorator and could
interrupt or roll back an active run, bypassing the runs:cancel scope.

Decorators cannot express query-parameter-conditional permissions, so
the check lives in require_cancel_permission_when_action(), applied at
the top of the handler. Regression drives the real helper through the
production middleware: runs:read-only PAT + action is 403, the same
token joins action-less, runs:read+cancel passes, session control
unaffected.

* docs(changelog): add the PAT feature entry

* docs(readme): add personal access tokens section

Repo documentation-update policy requires user-facing features to
update README.md in the same changeset; the PAT feature previously
touched only backend/docs/API.md and the gateway AGENTS.md.

* fix(auth): require runs:cancel for mutating multitask strategies

All five run-creation entrypoints were gated only by runs:create, but
RunCreateRequest.multitask_strategy accepts interrupt/rollback and
start_run forwards it to create_or_reject, which terminates an
already-active run. A runs:create-only PAT could therefore kill an
existing run through a create request, bypassing runs:cancel.

Decorators cannot express body-parameter-conditional permissions, and
per-route checks leave the same hole for the next entrypoint, so the
gate lives in start_run itself — the single choke point every
run-creation path (HTTP routes and internal launchers) flows through.
Regenerate launches pass multitask_strategy="reject" and are
unaffected; requests without a stamped auth context (internal/test
compositions) skip the gate.

The check is the shared authz.require_cancel_permission_if primitive;
require_cancel_permission_when_action now delegates to it, so every
request dimension that carries cancel capability (query action, body
strategy) flows through one gate.

Regression drives the real middleware stack: runs:create-only PAT +
interrupt/rollback is 403 with the exact detail, reject (explicit and
default) stays available, runs:create+cancel passes, session control
unaffected; a source anchor pins the gate inside start_run.

* fix(runs): keep observer joins from applying creator cancel-on-disconnect

sse_consumer's finally block applied the record's on_disconnect=cancel
policy on ANY consumer's disconnect. The join surfaces (GET /join and
the action-less GET/POST stream join) feed it the existing RunRecord,
so anyone with thread read access — including a runs:read-only PAT —
could cancel a locally-owned running run simply by closing the SSE
connection, without runs:cancel. The policy expresses the creator's
intent for their own connection; an observer's disconnect must never
be read as that intent.

sse_consumer gains apply_on_disconnect (default True). The two join
surfaces pass False; the creating endpoints (thread-scoped and
stateless create-and-stream) keep the creator semantics unchanged.
wait_for_run_completion needs no change: its callers are creator-side
or post-explicit-cancel paths only.

Regression exercises a real generator close — the same machinery
Starlette drives on client disconnect — against the production
sse_consumer: creator stream disconnect cancels, observer join
disconnect does not; a wiring anchor pins both join call sites and the
creator defaults. API.md documents the cancel-capability constraint
(this fix plus the action/strategy gates) in PAT Constraints.

* test(auth): pin the multitask gate behaviorally; state wait invariant

Independent adversarial review of the round-5 fixes found the P1-a
regression only mirror-pinned: the source anchor could be satisfied by
a comment, and deleting the gate from start_run would not fail the
suite. This drives the production start_run directly — a create-only
auth context gets 403 with the exact detail for interrupt, and a
reject request with no cancel permission at all proceeds past the gate
(never a permission 403).

Also documents wait_for_run_completion's creator-side invariant
(every caller is the creating endpoint or post-explicit-cancel) so a
future observer wiring thinks twice before reusing it — the one-caller-
away variant of the observer-disconnect P1.

* docs(changelog): correct the PAT entry's digest and route-policy description

The entry said HMAC digests (the implementation stores SHA-256 digests,
as documented in API.md and pinned by the repository tests) and claimed
the route policy admits 'implemented stateless endpoints' (it admits
the thread/run lifecycle routes, narrowing further by scopes). Also
notes the cancel-capability gate now covering action and multitask
strategies.

* fix(auth): enumerate the PAT runs route policy per implemented subroute

The runs subtree rule was a GET|POST /runs(/.*)? wildcard — it
pre-authorized every current and future subroute under /runs, including
methods the router never implemented (e.g. GET /runs/stream), which is
the same latent default-deny weakening the threads collection rule was
tightened for: a future route added under /runs would become
PAT-reachable without an explicit policy change.

The wildcard is replaced with six segment-precise rules covering exactly
the 14 implemented method+path combinations; the {run_id} slot
necessarily matches any single segment, so the POST-only collection
names (stream, wait, regenerate, edit-regenerate) are excluded from the
GET run-id rule via negative lookahead — no dead method stays
pre-authorized. Behavior for implemented routes is unchanged.

test_pat_runs_policy_admits_exactly_the_mounted_routes derives the
expected set from the mounted thread_runs router instead of a
hand-maintained list: every implemented GET/POST route under /runs must
be admitted, routes in this router outside the subtree stay denied, and
representative unimplemented neighbors are denied — so adding a route
under /runs now fails CI until it is explicitly allowlisted, and a
removed route leaves a dead rule visible. API.md's PAT constraints list
the enumerated routes and drops a feedback mention that belonged to the
stateless /api/runs axis.

* docs(migration): add the 0017 renumbering coordination note to 0017

The PR's migration-coordination comment states each migration file
carries the note; the file did not. Adds it: numbering was generated
against main head 0016 alongside #5078 and #4843; whoever merges first
keeps the slot, the others renumber on rebase (revision/down_revision
plus the bootstrap head assertions).

* fix(auth): pad base62 tokens to a fixed 43-char width

int.from_bytes discards leading zero bytes, so the unpadded encoder
returned a variable-length body — empty for all-zero input, and shorter
than 40 characters for any draw below 62**39 (~1 in 14.5M), leaving
test_generate_pat_token_format probabilistically flaky and the token
body without stable width (review round 6, P3).

_base62 now left-pads with "0" to _base62_width(len(data)) — the exact
integer digit count (62^43 > 2^256 > 62^42, so 43 for 32 bytes). The
format test asserts the exact fixed width instead of a probabilistic
floor, and a new unit test pins the all-zero, leading-zero-byte, and
max-value edges deterministically.
2026-08-29 23:50:45 +08:00
Baldwinzc
baaf2bad47
fix(gateway): stamp turn_duration on last AI message only in /messages/page (#4755)
_enrich_thread_message_page inlined its own turn_duration loop that
stamped EVERY AI message of a run, re-introducing #4152 on the
/messages/page endpoint (the legacy /messages and /history endpoints
already route through stamp_turn_duration_on_last_ai after #4163, but
that fix missed the page path introduced earlier in #4065). A
multi-step turn thus rendered the same run lifetime beside every
intermediate AI message, reading as repeated thinking latency.

Replace the inline loop with the shared stamp_turn_duration_on_last_ai
helper so all three message endpoints agree: the run's wall-clock
duration lands on its final visible AI message only.
2026-08-11 22:00:57 +08:00
Vanzeren
095092418c
fix(gateway):unify thread id validation (#4589)
* fix(gateway): unify thread ID validation at the API boundary

Thread ID entry points accepted arbitrary strings while downstream
consumers (filesystem paths, Kubernetes Provisioner, JSONL event store)
each enforced different character restrictions, so invalid IDs were
persisted first and only failed later during sandbox/workspace init.

Centralize validation in deerflow.utils.thread_id (pattern
^[A-Za-z0-9_-]{1,64}$): validate at routers, RunCreateRequest,
scheduler dispatch, paths.py, JSONL store, embedded client, and align
the Provisioner pattern (pinned by a parity test). UUIDs are still
generated only when no ID is supplied; caller-supplied opaque IDs stay
supported.

Deliberate exceptions: DELETE /threads/{id} keeps str as the legacy
cleanup escape hatch (filesystem cleanup guarded), read-only
client.get_thread stays unvalidated, and scheduler rows with legacy
invalid IDs record a failed dispatch instead of raising out of the
poll loop.

* docs: document canonical thread ID contract

README: caller-supplied thread IDs need not be UUIDs; the canonical
pattern and per-endpoint behavior. AGENTS.md: the shared
deerflow.utils.thread_id contract, its enforcement boundaries, and the
legacy-ID escape hatches.

* fix(gateway): close thread ID validation gaps at remaining entry points

Follow-up to the canonical thread ID contract: a full audit found the
uniform-422 coverage only reached about half of the thread_id surfaces.

- routers: 18 routes still took a bare thread_id: str — 13 in
  thread_runs.py (including the five messages/events/workspace-changes
  reads that returned 500 on the JSONL event store vs 404/empty on the
  DB store), 4 read routes in threads.py, and the suggestions route
  flagged in review. DELETE /api/threads/{id} keeps str as the declared
  legacy-cleanup escape hatch.
- client: upload_files/delete_upload/list_uploads/get_artifact now
  validate up front, fulfilling the RFC's 'all mutating entry points'
  clause (get_thread stays unvalidated as the declared legacy read path).
- tui: the /resume literal-ref fallback validates against the canonical
  contract and reports a descriptive error instead of failing deep in
  the client.
- scripts/support_bundle.py: replace the drifted dot-allowing pattern
  with a byte-identical copy of THREAD_ID_PATTERN (kept local so the
  script still runs with a broken venv).

* test(gateway): guard the canonical thread ID contract against regressions

- test_thread_id_route_contract.py: static AST sweep asserting every
  route handler with a thread_id parameter annotates ThreadId
  (whitelist: the DELETE escape hatch), plus a runtime sweep hitting
  all 44 thread_id routes with a non-canonical ID and asserting a 422
  that names thread_id, plus a websocket upgrade-rejection case.
- test_thread_id_validation.py: client entry-point validation,
  support_bundle pattern parity, and TUI literal-ref fallback tests.
- Align two tests that encoded the old contract (dotted IDs).
2026-08-01 19:42:44 +08:00
Amorend
85c3909c2e
feat: show real-time context window usage (#3125) (#3183)
* feat: show real-time context window usage in chat UI (#3125)

Adds a `context_usage` block to `GET /api/threads/{id}/token-usage`
(token count from the live checkpoint, the thread model's
`context_window`, and a percentage), introduces a new
`ModelConfig.context_window` distinct from the per-call `max_tokens`
output cap, and surfaces the percentage in the chat header — inside
`TokenUsageIndicator` when token-usage tracking is on, or as a
standalone badge when it's off so context capacity stays visible
independent of cost tracking.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: per-category breakdown for context window usage

Replace the single-number context_usage payload with a Claude-Code-style
breakdown — messages, system prompt, skills, system/MCP tools (active +
deferred), custom agents, memory injection, autocompact buffer, and free
space — and surface it in the chat UI with a segmented progress bar and
per-row table.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(config): document context_window across model examples

Add `context_window` to every example model in config.example.yaml so the
new chat-UI "% context used" indicator works out of the box for whichever
example a user adopts. Each value is the published default at the time of
writing; users are pointed at the official model spec to verify. Bumps
config_version to 11 so `make config-upgrade` flags outdated user configs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* style: ruff format (line-length 240)

No behavior change — collapses two multi-line expressions that fit on
one line under the project's 240-char limit. Picked up by `make format`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* review: address Copilot bot comments on #3183

- token-usage-indicator: switch `{contextPercentage && (...)}` to an
  explicit `!= null` check. (The string `"0"` is actually truthy in JS so
  the original code wasn't buggy, but the explicit check is clearer.)
- context-usage-breakdown: drop the `useMemo` around segments/totals — the
  computation is O(n) over a handful of rows and the previous memo deps
  omitted `t.contextUsage.categories`, so the bar's tooltips/aria-labels
  could stay in the old language after a locale switch.
- context_usage._split_tools: snapshot MCP names from
  `get_cached_mcp_tools()` directly instead of re-reading
  `extensions_config.json` after `get_available_tools()` already loaded
  it. Removes redundant file I/O on every `/token-usage` poll.
  (`get_available_tools()` still emits its own INFO logs — silencing
  those is out of scope here.)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* style(frontend): prettier --write context-usage-breakdown

CI's `pnpm format` (prettier --check) caught two lines previously
formatted by hand. Collapses one comma to fit on one line; no behavior
change.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(gateway): correct context-usage breakdown + add exact token counting

The context-usage indicator shipped two bugs that silently zeroed whole
breakdown rows (both caught by try/except, so the feature looked alive but
produced wrong numbers):

1. _count_system_prompt passed app_config= to get_deferred_tools_prompt_section,
   which only accepts deferred_names -> TypeError swallowed -> system_prompt
   row always 0, and used_tokens/percentage undercounted by the full prompt.
   Also subtracted the deferred section twice (the rendered prompt already
   excluded it). Fix: derive deferred names deterministically and pass them to
   apply_prompt_template; drop the redundant subtraction.

2. _split_tools imported a non-existent get_deferred_registry -> ImportError
   swallowed -> all four tool-category rows always 0. Fix: classify via the
   public is_mcp_tool predicate + tool_search.enabled (mirrors
   build_deferred_tool_setup); the MCP tag is set by get_available_tools.

Added token_usage.counting (approximate|exact). 'exact' routes text/schema/
message counting through the model tokenizer (tiktoken cl100k_base) via the
existing memory-module machinery (lazy load + cache + cooldown + CJK-aware
fallback), so CJK-heavy threads stop being undercounted by chars//4.

Regression + e2e tests added; 6621 backend tests pass.

* fix(gateway): harden context usage accounting

* fix(gateway): count promoted MCP tools as active in context usage

Promoted tools (deferred MCP tools the thread has fetched via tool_search)
have their full schema bound on every subsequent turn by
DeferredToolFilterMiddleware, so they consume context like any active tool.
The breakdown previously left them in the reserved *_deferred rows, under-
counting the thread's used_tokens.

Classification now treats a tool as deferred only when tool_search is enabled,
it is MCP-sourced, AND it has not been promoted. The promoted set is read from
the checkpoint's channel_values and scoped by catalog hash — matching the
runtime middleware, so a stale promotion from MCP-config drift cannot inflate
the active count.

The static system prompt still lists all deferred tool names (promotions only
affect schema binding, not the prompt), so _count_system_prompt's deferred
rendering is intentionally left unchanged.

8 new tests cover classification, catalog-hash scoping (match / drift /
compute-failure / malformed), and checkpoint extraction.

* fix(context): address review feedback

* fix(context): count structured message payloads

* fix(context): harden usage accounting

* fix(config): bump schema for context usage fields

* refactor: narrow context usage to core indicator

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-31 21:57:22 +08:00
Xinmin Zeng
e56481d9e3
fix(runs): stamp elapsed duration once per run across message APIs (#4163)
* fix(runs): show run duration once per run and label it as elapsed work

turn_duration is the run's wall-clock lifetime (updated_at - created_at),
but both message endpoints stamped it onto every AI message of the run
and the frontend rendered each stamp as 'Thought for X seconds' — the
same number repeated per message, and tool-wait time presented as model
thinking latency (#4152).

- share one stamping helper between list_messages and list_run_messages:
  only the run's final non-middleware AI message carries turn_duration
  (list_run_messages already tried to do this but iterated reversed()
  without stopping at the first match)
- completed-state trigger copy becomes 'Worked for X seconds' since the
  number includes tool execution, not just thinking

Fixes #4152

* fix(threads): stamp turn_duration once per run in /history replay too

get_thread_history stamped turn_duration on every AI message of a run
instead of only the last one, so a run with several AI messages (e.g.
a tool call followed by a final answer) showed the same duration badge
more than once on reload.

Rebasing onto main picked up #4118, which replaced the /history
duration path with a checkpoint-metadata fast path plus an
event-store/run-manager fallback for legacy checkpoints - both of
which stamped every AI message per run_id, reintroducing this same
bug on both paths. This commit folds the once-per-run stamping
(stamp_turn_duration_on_last_ai) into both paths instead of only the
legacy one, and narrows the fast-path/fallback boundary to a per-run
completeness check (turn_run_ids - checkpoint_run_durations) instead
of a per-message one, so a run whose duration is already in checkpoint
metadata never re-triggers the event-store correlation fallback just
because its non-final AI messages correctly have no turn_duration.

* fix(frontend): stop caching a client-measured turn_duration per message

A run can produce several standalone content-only AI messages (e.g. a
subagent handoff), each briefly becoming the newest message and each
mounting its own Reasoning timer via the shared turnStartTime. Wiring
onTurnDurationChange let that per-message timer cache and display a
"Worked for X seconds" badge the moment a later message superseded it
- with a different, premature number, since the timer only measured
that message's own streaming window, not the run's total elapsed
time. That reintroduced the #4152 duplicate-badge bug through a path
the backend fix (once-per-run stamping) doesn't cover.

cachedDuration now only ever mirrors a backend-confirmed
turn_duration (already handled by the other effect here), so a
superseded message can no longer display a stale or wrong duration.

* fix(frontend): use 'Worked for' framing even without a persisted duration

The completed-state copy diverged depending on whether turn_duration
had landed yet: "Thought for a few seconds" (no duration) vs. "Worked
for X seconds" (duration present). Both describe the same completed
run, just with or without a persisted number, so the framing
shouldn't disagree on what happened.

* docs(runs): note why the middleware skip is inert on /history, add test

stamp_turn_duration_on_last_ai's middleware-caller skip only has an
effect on the event-store message shape (metadata.caller); checkpoint
messages replayed on /history never carry that field. Document why
that's not a gap - middleware writes go to thread metadata, not the
messages channel, so no middleware message ever reaches a
checkpoint's messages list.

Also lock the middleware-skip contract on list_thread_messages with
its own regression test, mirroring the existing one for
list_run_messages - the thread feed only gained this skip through the
same shared helper, and previously stamped every AI message,
middleware included.

* chore(frontend): retain current run duration UI

The frontend behavior was superseded by #4348; keep the latest main implementation while retaining this PR's backend fix.

* test(frontend): retain current reasoning coverage

Align the old PR test with the frontend implementation already merged in #4348.

---------

Co-authored-by: fancyboi999 <fancyboi999@users.noreply.github.com>
2026-07-29 12:47:30 +08:00
Aari
e47bf80122
fix(runtime): regenerate interrupted responses (#4524) 2026-07-28 22:15:09 +08:00
Aari
9a43d8276d
fix(gateway): replay edit and rerun from a settled checkpoint (#4534)
Editing the only turn of a thread reran the original prompt: the model
answered the question the edit was replacing while the UI showed the
edited text, and the edit vanished on reload.

The replay-base lookup decided whether a checkpoint predates the target
user message by message id alone. DynamicContextMiddleware re-keys the
first user turn to `{id}__user` mid-run, so every checkpoint written
before it holds the same prompt under an id the lookup cannot match. The
scan walked past those and anchored inside the run that produced the
turn — a checkpoint that still contains the original prompt and owns the
injection node's pending writes, which the replay then re-added after the
edited message.

Require the replay base to be a settled checkpoint (no pending tasks) in
both the lineage walk and the chronological fallback. That rule is
middleware agnostic: the first turn now anchors on the thread's empty
initial checkpoint and later turns on the previous run's tail, which also
drops the existing reliance on LangGraph discarding a stale `__start__`
write.

Edit replay additionally passes `head_checkpoint` so it resolves its base
lineage-first like regenerate does, and a replayed user message is
restored to its pre-swap id: replaying `{id}__user` into a state that has
no reminder yet makes the middleware treat the turn as already injected
and silently drops its date and memory block.

Frontend: a prepared replay masks the turn it supersedes, so the
optimistic-message baseline is taken from the post-mask human count. The
pre-mask count can never be exceeded when the replay puts exactly one
human message back, and on the first turn the runtime re-keys the
replacement message so identity comparison cannot stand in for the count.

Fixes #4531
2026-07-28 22:12:27 +08:00
MiaoRuidx
8a78c264b7
fix(runtime): cancel runs across live gateway workers (#4500)
* docs(runtime): design cross-worker cancellation

* fix(runtime): cancel runs across gateway workers

* fix(runtime): harden cross-worker cancellation races

让取消请求与 owner 终态写入通过持久化 CAS 决定先后,保证首次取消 action 在不同 worker 路由下保持一致。\n\n将 heartbeat 收敛为续租后仅发送本地中止信号,并补齐完成竞态与路由重试的回归用例。

* docs(runtime): drop implementation plan from PR

移除仅用于实现过程的跨 worker 取消设计记录,保留 README 和 backend/AGENTS.md 中面向最终行为的文档。

* fix(runtime): preserve local cancel fallback

* test(runtime): adapt worker run manager fakes

* docs(runtime): fix run cancel migration registry

---------

Co-authored-by: MiaoRuidx <12540796+MiaoRuidx@users.noreply.github.com>
2026-07-28 21:45:20 +08:00
ShitK
b1984cf4ab
fix(security): reject legacy MCP credentials in run metadata (#4448)
* docs: design run metadata secret admission

* docs: refine run metadata secret boundaries

* docs: plan run metadata secret fix

* fix(security): centralize legacy run metadata policy

* fix(security): reject secrets at run admission

* fix(security): hide legacy secrets from history APIs

* docs(security): migrate MCP credentials to secret context

* fix(security): redact legacy runnable config metadata

* fix(security): reject legacy config metadata credentials

* fix(security): hide legacy secrets from run kwargs

* docs(security): clarify config redaction boundary

* docs: keep issue 4416 planning local
2026-07-28 21:31:23 +08:00
Aari
919caf7c83
fix(gateway): keep a manual rename through edit and rerun (#4539)
Renaming a conversation and then editing one of its turns reverts the
title to whatever it was before that turn ran, so the user's own name for
the thread is silently replaced by an older automatically generated one.

Edit replay resumes from the checkpoint before the edited turn, and that
checkpoint predates the rename. Regenerate already guards against exactly
this rollback by replaying the current title as graph input; the edit
replay path was added later and did not carry the guard over.

Replay the title the same way, but only when the replay base already has
one. An untitled base belongs to a thread the title middleware has not
named yet — pinning the current title there would keep a name generated
from the prompt this edit just replaced, and stop the middleware from
naming the rewritten turn.
2026-07-28 18:06:23 +08:00
Ryker_Feng
fcbf0609b0
feat(chat): edit and rerun latest user turn (#4377)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-27 22:46:51 +08:00
Vanzeren
625c07b993
fix(runtime): resume original title when regenerating (#4480)
* fix(runtime): rusume original title when regenerating

* test(runtime): cover regenerated title sync
2026-07-27 11:32:10 +08:00
Nan Gao
58befaf248
fix(thread-history): keep completed subtask cards stable after reload (#4432)
* fix(thread-history): hide subagent AI responses

* refactor(thread-runs): remove unused _is_middleware_message_row helper

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-25 09:29:11 +08:00
ShitK
a4ede80deb
fix(runtime): reject unsupported run options and stream modes (#4430)
* fix(runtime): reject unsupported run options

* fix(runtime): align SDK run compatibility

* fix(frontend): avoid unsupported events stream mode

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-24 19:24:24 +08:00
Huixin615
fbc1463809
fix(gateway): preserve regenerate state in branched threads (#4358)
* fix(gateway): preserve regenerate state in branched threads

* test(gateway): isolate branch regenerate regression config

* fix(gateway): preserve branching for legacy histories

* fix(gateway): harden branch regenerate lineage

* docs(gateway): clarify branch checkpoint behavior

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-24 08:57:48 +08:00
Vanzeren
42baed8c8c
feat(checkpoint): dual-mode checkpoint storage with LangGraph DeltaChannel (#4292)
* feat(checkpoint): dual-mode checkpoint storage with LangGraph DeltaChannel

Add a restart-required database.checkpoint_channel_mode ("full" default,
"delta") that stores the messages channel via LangGraph 1.2 DeltaChannel,
cutting checkpoint storage from O(n^2) to O(n) for append-only history.
Existing full checkpoints seed delta state transparently; no data migration.

- config: mode schema + freeze-on-first-use with
  CheckpointModeReconfigurationError; mode marker persisted in checkpoint
  metadata; unsafe delta->full downgrade rejected fail-closed with
  CheckpointModeMismatchError (run-level error, failed state read)
- state: delta message state schema; CheckpointStateAccessor centralizes
  materialized reads for all consumers (threads API, branches,
  regeneration, compaction, state updates, memory, goal workers)
- runtime: raw writers (run durations, interrupted title, thread goal)
  parent their checkpoints to the checkpoint they derive from, preserving
  delta ancestry; rollback forks the pre-run lineage through a state
  mutation graph with Overwrite restores; InMemorySaver delta-history
  override delegates to the base walk (fixes dropped first write after
  migration, also present upstream)
- tests: conformance suite over {memory, sqlite, postgres} covering
  migration replay, stable message IDs, storage shape and writer
  preservation; conftest fixture isolates the frozen mode between tests;
  stale config fakes refreshed
- ci: backend unit tests gain a postgres service

* fix(checkpoint): close materialization gaps in goal flow, guard public factory

- Route goal-continuation message reads through CheckpointStateAccessor:
  raw channel_values reads see the delta sentinel in delta mode, which
  disabled goal continuation (stand_down=no_durable_end_of_turn) after
  durable assistant turns. Raw tuples remain for tuple-only metadata
  (checkpoint id, pending_writes).
- Reject checkpoint_channel_mode='delta' + checkpointer in
  create_deerflow_agent at construction: factory-built persisted graphs
  bypass mode-marker injection and the fail-closed gate, reproducing
  silent mixed-mode state loss. Delta without persistence stays allowed.
- Import the postgres saver lazily (pytest.importorskip in the fixture)
  so the documented default install collects the suite; add a CI job
  running pytest --collect-only on uv sync --group dev without extras.
- Fix test_checkpointer fallback test to patch get_app_config at its
  use site (provider module), making it deterministic when a local
  config.yaml selects a persistent backend.

* fix(gateway): preserve extension-owned channels in state mutations, bump config version

- build_state_mutation_graph / build_checkpoint_state_mutation_accessor
  accept an explicit state_schema; branch and POST /state now compile the
  mutation graph from the thread's effective schema
  (graph_state_schema on the assistant graph). The base-ThreadState
  fallback silently discarded channels contributed by custom
  AgentMiddleware.state_schema on branch (data loss) and returned a
  false-success 200 on POST /state.
- POST /state validates values keys against the mutation graph's
  channels and rejects unknown fields with 422 instead of ignoring
  them; reducer detection covers extension channels
  (BinaryOperatorAggregate or DeltaChannel) so Overwrite replace
  semantics work for middleware reducers in both modes.
- Endpoint regression: custom AgentMiddleware.state_schema value
  survives branch, updates through POST /state, and an unknown field
  receives 422.
- config_version 26 -> 27 for the new database.checkpoint_channel_mode
  (example, Helm chart values + README, support-bundle fixture), so
  existing installs get the outdated-config warning and
  make config-upgrade merges the field; covered by a test driving the
  real example file and the real config-upgrade script.

* fix(gateway): resolve assistant schema via one boundary, copy branch reducer values with Overwrite

GET /threads/{id}/state now resolves the thread's assistant_id through a
single reusable boundary (thread metadata -> assistant_id -> effective
graph), so channels contributed by a custom AgentMiddleware.state_schema
are materialized instead of dropped by the default lead schema. POST
/state uses the same boundary instead of resolving the schema ad hoc.

Branch writes wrap every copied reducer channel in Overwrite (derived
from the effective mutation graph: BinaryOperatorAggregate + DeltaChannel),
not just messages, so already-aggregated values are never re-merged.

Regression tests use a real AgentMiddleware.state_schema with a
non-identity reducer in both full and delta modes: GET /state returns the
extension value, POST /state replaces it, branch preserves it
byte-for-byte; the unknown-field 422 is a separate assertion.

* refactor(checkpoint): collapse read-path round-trips and ship dual-mode parity tests

Address review round 4 on PR #4292:

- Push ahistory/history limit through Pregel into checkpointer.alist
  (SQL LIMIT) instead of materializing all rows and breaking in Python
- Fold the read-side mode-compat gate onto the returned snapshot's
  metadata; only writes keep the pre-write tuple fetch (fail-closed)
- Cache factory-built accessor graphs per (assistant_id, mode) with
  factory-identity revalidation; state reads no longer build a lead
  agent per request
- get_thread: one snapshot fetch + one raw pending_writes fetch on the
  resolved checkpoint (post-checkpoint __error__ writes never surface
  in snapshot.tasks; verified empirically)
- DeerFlowClient.get_thread: single checkpointer.list walk collects
  pending_writes per checkpoint instead of N get_tuple calls
- InMemorySaver delta-history patch: stand-down when the upstream
  override disappears, try/except guard, validated-version warning,
  guard tests
- make_lead_agent mode precedence: first freeze is owned by app_config
  (client-supplied configurable key ignored); once frozen, injected
  key/app_config must match or fail closed
- Rollback: lock in non-message channel restoration via fork
  inheritance with a dedicated reducer-channel test
- Add tests/test_threads_checkpoint_mode.py and
  tests/test_gateway_checkpoint_mode.py referenced by AGENTS.md and
  the PR validation section: lifecycle parity (memory + sqlite),
  per-step blob-count storage guard, gateway endpoint parity

Counted-saver tests pin checkpoint round-trips for aget/ahistory so
these regressions cannot silently return.

* fix(checkpoint): precise mode-mismatch HTTP mapping, gate E2E, and accessor resilience

- threads router: map CheckpointModeMismatchError to 409 (with cause and
  thread id) and CheckpointModeReconfigurationError to 503 across all state
  endpoints instead of swallowing both into a generic 500
- gate coverage: seed a real delta checkpoint into AsyncSqliteSaver and
  assert aget/aupdate/ahistory fail closed; assert 409 at the HTTP boundary
  through the real route stack
- rollback: compile the restore mutation graph with the thread's effective
  state schema per the build_state_mutation_graph contract
- inheritance contract locks: rollback and manual compaction preserve
  middleware-contributed channels via checkpoint fork cloning
- services: revalidate the accessor-graph cache against app_config identity
  so config.yaml hot-reloads never serve a stale compiled graph
- services: degrade full-mode state reads to raw checkpointer reads when the
  agent factory is unavailable (delta gate still applies; delta mode has no
  fallback)
- deps: override websockets==16.0 (langgraph-sdk 0.4.2's <16 pin silently
  downgraded 16.0 -> 15.0.1; pin is not grounded in any API incompatibility)
  and bump the langchain lower bound to what the lockfile actually resolves

* fix(checkpoint): include anchor checkpoint in degraded history walk + cover get_thread

- _RawCheckpointReadAccessor.ahistory: alist(before=...) is exclusive while
  pregel's get_state_history treats config.checkpoint_id as the inclusive
  start; fetch the anchor explicitly so both read paths paginate identically
- extend the degraded-path gateway test: GET /thread returns raw values, and
  POST /history with before starts at the anchor checkpoint

* fix(gateway): preserve degraded checkpoint timestamps

* fix(gateway): harden degraded checkpoint access

* fix(gateway): resolve assistants for checkpoint reads

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-22 08:33:29 +08:00
Ryker_Feng
e89edb39b1
fix(gateway): reject non-positive read limits (#4284) 2026-07-18 22:12:57 +08:00
Vanzeren
1769b2de0d
fix: read run stop_reason from runtime context (#4188)
* fix: read run stop_reason from runtime context

* fix: address review feedback for #4188 stop_reason integration

   - migration 0005: use safe_add_column for consistency and drift detection
   - worker: clear runtime.context stop_reason at start of each _stream_once
     turn so a clean continuation doesn't inherit a prior cap reason
   - tests: replace circular unit test with real middleware integration
     tests that exercise LoopDetectionMiddleware._apply and
     TokenBudgetMiddleware._apply through the worker, proving the full
     middleware → runtime.context → persist pipeline

* fix(test): resume conftest

* fix: stamp stop_reason in all guard middlewares, fix clearing semantics
2026-07-16 08:19:52 +08:00
Vanzeren
13fd8e229a
fix(checkpoint): persist run duration in checkpoints for history reads (#4118)
* fix: persist run duration in checkpoints for history reads

* fix(checkpoint): harden run duration persistence

* fix(checkpoint): persist run durations in metadata

* fix(checkpoint): address review findings for run duration persistence

- Add valid_duration_entry() shared validation helper (worker.py)
- Rename _persist_run_durations -> persist_run_durations as public API
- Import public persist_run_durations and valid_duration_entry in threads.py
- Use BackgroundTasks for lazy backfill write to avoid blocking history reads
- Add TODO about O(runs) growth of run_durations in checkpoint metadata
- Document REGENERATE_HISTORY_RAW_SCAN_LIMIT doubling assumption

* fix(checkpoint): replace pruning TODO with justification

Accumulated run_durations overhead (~50 bytes/run_id) is negligible compared
to messages channel blobs; no pruning strategy is needed.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-14 21:37:47 +08:00
AnoobFeng
446fa03801
fix(context): resolve context compress bug (#4065)
* fix(runtime): persist original human input outside model sanitization

* refactor(history): load thread messages by global event sequence

* fix(frontend): make summarization rescue a transient history bridge

* fix(frontend): old message not append tail

1. add identity anchor
2. add bridgeOrder

* fix(frontend): lint error fix

* fix: address review feedback and harden pagination coverage

- defer transient history ref writes until after render commit
- cover large middleware-only history scans
- verify infinite-query refetch recalculates page cursors
- document AI event types and anchor-weaving differences

* fix: harden message pagination and enrichment

- append unmatched live tails after canonical history
- warn and stop when pagination has_more lacks a cursor
- deep-copy restored UI messages to isolate model-facing content
- log invalid event sequence and non-advancing cursor errors
- pass user_id explicitly through event-store history queries
- cover middleware-only AI runs across memory, JSONL, and DB stores

* fix: address pagination review feedback

* fix(frontend): checkpoint has unknow redener content, optimize the anchor policy

* fix(frontend): unit test issue missed previously, remove the TanStack cache trimming

* fix(gateway): harden message history queries and provenance

- reject externally forged original_user_content metadata
- validate provenance metadata in upload and sanitization middleware
- make run lookups fail closed by default
- batch feedback queries by run ID
- align memory message filtering with persistent stores
2026-07-14 10:43:13 +08:00
heart-scalpel
b53c1ae0e0
fix(runs): cancel degrades to lease takeover for multi-worker (#4064)
* fix(runs): cancel degrades to lease takeover for multi-worker

Work item 4 of the multi-worker ownership epic
(https://github.com/bytedance/deer-flow/issues/3948).

Problem: POST /runs/{run_id}/cancel landing on a non-owning worker
returns 409 — the cancel button silently fails under GATEWAY_WORKERS>1
with no sticky routing. cancel() required the current worker to hold
the in-memory task/abort_event, which any non-owner pod cannot satisfy.

Changes:
- RunManager.cancel() returns CancelOutcome enum (cancelled /
  taken_over / lease_valid_elsewhere / not_active_locally /
  not_cancellable / unknown) instead of bool, so the router can map
  each outcome to the right HTTP response.
- New store primitive claim_for_takeover(): a single atomic
  conditional UPDATE that marks a run as error only when
  status IN (pending, running) AND (lease IS NULL OR
  lease < now - grace). Closes the stale-read / concurrent-heartbeat
  race — if the owner renews between our read and write, the UPDATE
  matches 0 rows and we surface lease_valid_elsewhere.
- HTTP cancel + stream-join endpoints route on CancelOutcome:
  cancelled -> 202 (or 204 with wait=true); taken_over -> 202
  immediately (no SSE streaming — the run is terminal on another
  worker, streaming would hang); lease_valid_elsewhere -> 409 +
  Retry-After header computed from lease_expires_at + grace_seconds.
- RunManager.grace_seconds exposed as a public property; the router
  no longer reaches into _run_ownership_config.
- _is_lease_expired extracted to a module-level function, shared by
  RunManager.cancel() and MemoryRunStore.claim_for_takeover().
- GATEWAY_WORKERS=1 + heartbeat_enabled=false is zero-regression:
  the non-local path short-circuits to not_active_locally, preserving
  the original 409 behaviour the existing tests pin.

Tests: 12 new (5 store primitive + 4 cancel-takeover unit + 3 HTTP
including a regression guard verifying POST /stream?action=interrupt
on a dead-owner run returns 202 instead of hanging on SSE).
244 directly-related tests pass; 36/36 blocking-IO gate pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(runs): guard update_status and self-terminate on takeover

Two defenses close a split-brain window where the original owner
could overwrite a peer's takeover status:

- update_status (SQL + memory store) now guards on
  status IN ('pending','running'). When takeover already set
  the row to 'error', the owner's final status write matches
  0 rows and is dropped.

- _persist_status: when update_status returns False, check
  whether the row exists before attempting recovery via put().
  If the row exists (takeover by another worker), skip recovery
  instead of blindly upserting over the takeover.

- Heartbeat _renew_leases: when update_lease returns False
  (row no longer pending/running or owner changed), cancel the
  local task so wasted CPU is bounded to the next heartbeat
  tick (~10s) instead of the full task lifetime.

Also fix three reviewer feedback items:

- Re-fetch the store row when cancel() returns
  lease_valid_elsewhere, so Retry-After uses the owner's
  freshly-renewed lease instead of a stale value from
  request start.

- Fallback 'unknown' in takeover error message when
  owner_worker_id is NULL (pre-ownership data).

- Remove dead else-10 branch from grace_seconds property
  (unreachable — all callers are downstream of the
  heartbeat_enabled guard).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(runs): pin split-brain defences from update_status guard + heartbeat

Three tests lock down the takeover authoritativeness so a
late-running owner cannot overwrite a peer's claim:

- update_status must reject writes when the store row is already
  terminal (taken over by another worker).
- _persist_status must skip row-recovery via put() when the row
  exists but has been taken over.
- Heartbeat _renew_leases must cancel the local task when
  update_lease returns False (row claimed by another worker).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(runs): precise outcome + log when local cancel loses to peer takeover

Two reviewer precision nits on the split-brain defence:

- _persist_status: branch the skip-reason log on existing["status"].
  error → WARNING "peer takeover" (anomalous); interrupted/success →
  INFO "local cancel/completion race" (expected when user hits stop
  as the run finishes). Stops noisy false-positive takeover warnings
  in operator logs.

- cancel() local path: when _persist_status returns False, re-check
  the store. If a peer's claim_for_takeover flipped the row to error
  between our in-memory cancel and the guarded update_status, surface
  taken_over instead of cancelled so the client sees a status
  consistent with the store.

Test: test_cancel_returns_taken_over_when_peer_claims_during_local_cancel
pins the race outcome.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(runs): widen update_status guard, de-duplicate lease helpers, add coverage

Round 3 of reviewer feedback:

- Widen update_status guard to status IN ('pending','running','interrupted').
  The original guard blocked interrupted→error (the rollback finalize path),
  losing the "Rolled back by user" message. interrupted is now permitted
  while error/success stay locked — takeover protection unchanged.

- claim_for_takeover False now re-reads the store row to distinguish causes:
  owner renewed lease → lease_valid_elsewhere; row went terminal →
  not_cancellable; another worker already took it over → taken_over.

- Extract _raise_lease_valid_elsewhere() helper to de-duplicate the
  409+Retry-After block shared across cancel_run and stream_existing_run.

- Extract _lease_expired_or_null() in persistence/run/sql.py to
  de-duplicate the lease-expiry SQL WHERE clause shared by
  claim_for_takeover and list_inflight_with_expired_lease.

- 11 new tests: 5 SQL-layer claim_for_takeover (expired/valid/NULL/
  terminal/nonexistent), 3 _compute_retry_after unit (NULL/unparseable/
  normal), 2 claim re-read precision (terminal/takeover), 1 stream
  endpoint 409+Retry-After.

Not addressed (non-blocking, reviewer agreed):
- The 2–3 store.gets in the takeover cold path: optimizing the API to
  accept a pre-fetched record would couple the router to the manager
  more tightly than justified by the perf gain.
- The lease-expiry inline loop in MemoryRunStore.list_inflight_with_-
  expired_lease pre-computes cutoff once for all rows; switching to the
  shared _is_lease_expired helper would recompute datetime.now() per row
  with no real benefit.

260 related tests pass; 36/36 blocking-IO gate pass; ruff clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(runs): de-duplicate lease-expiry helper, restore defensive fallback

Address final round of review feedback:

- Extract is_lease_expired to deerflow.utils.time (no _ prefix, public
  utility). Manager and MemoryRunStore now import from the same place
  instead of the store reaching backward into the manager for a private
  function.

- Restore defensive else-10 fallback in grace_seconds property (removed
  in an earlier round). The guard is unreachable for current callers but
  protects future ones from AttributeError.

- Comment the transient in-memory interrupted vs store error state when
  a local cancel is superseded by a peer takeover.

- Comment the max(1, ...) floor in _compute_retry_after — the floor is
  a lower bound, not a poll interval; clients should apply jitter.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: rayhpeng <rayhpeng@gmail.com>
2026-07-14 07:37:59 +08:00
Ryker_Feng
00161811bd
feat: add workspace change review for agent runs (#3945)
* feat: add workspace change review

* chore: format workspace change files

* fix: optimize workspace change summaries

* style: refine workspace change badge scale

* fix: restore workspace change user context import

* fix(frontend): gate workspace change badge to assistant messages

Only pass run_id to assistant MessageListItems so the workspace-change
badge can never render under a user's prompt, which carries the same
run_id. Assert single badge render in the E2E flow.

* fix(workspace-changes): address review feedback on diff parsing and badge

- Restrict unified-diff header detection to "+++ "/"--- " (trailing space)
  in both backend _count_diff_lines and frontend getWorkspaceChangeLineClass
  so content lines beginning with +++/--- are counted/styled correctly
- Gate WorkspaceChangeBadge to ai messages so tool messages folded into an
  assistant group don't render a duplicate badge
- Add regression tests for both diff-classification fixes
- Apply prettier formatting to workspace-changes files flagged by CI

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-05 15:38:01 +08:00