* fix(frontend): retry a failed same-tab reconnect after refresh
After a refresh the SDK reconnects once from the tab's lg:stream pointer
and keeps that pointer when the join fails. Active-run recovery defers to
the SDK whenever the pointer matches the active run, and its error handler
only scheduled retries for joins it started itself, so a failed reconnect
(at connect time or mid-stream) left the tab without a live stream until
another refresh.
When an SDK reconnect fails, release the pointer and wake the recovery
effect, which rejoins the run if the runs read still reports it active,
with its existing bounded retries. Runs created by this hook's submit
(onCreated) keep their current error handling.
The new native-reconnect-recovery test drives the real SDK useStream, so
it also pins the SDK behavior this relies on: one reconnect per mount and
a pointer that survives a failed join.
* docs(changelog): reference #6400
* test(frontend): pin that a released reconnect skips finished runs
A run handed to recovery after a failed SDK reconnect was never streamed
by this hook, so completedRunIdsRef cannot guard it; only the
server-reported status in activeRunId does. Resolve the runs read with
each terminal status after the failed reconnect and assert recovery
never joins and the pointer stays cleared.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks
- Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal
streak boundary) and stop_condition (the user's stop rule, own column).
- Stop rule is normalized (stop_rule.py) and appended only at launch; the
stored prompt never contains it. Wording adapts when scheduler.tool_enabled
is off.
- Willem 1: page-created tasks get goal evaluation, <standing_notes>, the
previous-run reference and own-schedule stop; request_stop and the
scheduled capability no longer require an origin conversation.
- Repository: single reactivation choke point raising
ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop
condition/note changes (not on resume); goal-check failures neither count
nor reset the streak; pause of a finished task returns "finished"; trials
on a paused task keep the host pause marker.
- New repository reads: automatic_runs_used_for, active_run_status_for,
list_manageable_from_thread, thread_relation/thread_run on thread tasks,
task_ids_for_thread, run_number, and run_number/total_tokens/summary on
run history rows.
- REST: goal/max_runs/end_at/stop_condition on create and PATCH (null
clears, naive end_at = task timezone), resume recomputes from now with an
optional renewal body, once_time_passed, task_finished, limits_exhausted,
scheduler_not_running on create, trigger returns outcome/existing/thread_id,
task responses add automatic_runs_used and active_run_status.
- Coded errors {code, message, params} for routes, shared validation and the
chat capability (scheduled_task_errors.py), pinned by
contracts/scheduled_task_errors_contract.json; goal notes contract v2 with
check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py).
- /api/features adds scheduled_tasks {available, running, tool_enabled,
min_interval_seconds}; ScheduledTaskService.is_running.
- Docs: six README scheduler sections, API.md scheduled tasks section
(breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs.
- Tests: migration 0031, stop rule, error contract, run listing, lifecycle
over HTTP, repository, service and capability cases; updated tests for the
new error shape, finished-task pause and head revision.
* feat(scheduler): conversation update/resume, run-conversation authority and readable launches
- schedule_task: update and resume actions, stop_condition and clear_fields
parameters, plain parameter types (value rules come back as coded results),
rewritten docstrings for schedule_task and stop_scheduled_task.
- Capability: update/resume share the REST validation and error codes;
interactive turns in a task's run conversation may manage that task;
scheduled runs keep only own-stop; grants require a running scheduler.
- Results: compact JSON-safe task views with local times, display card/text,
last run in list, timezone_source/browser_timezone, now_local; no ISO, user,
lease or origin fields; stop_instructions removed.
- Timezone: explicit > browser (context.client_timezone, validated in
start_run and never forwarded to the run config) > not needed for intervals
and offset run_at > timezone_required.
- Trial gate accepts natural direct requests after acknowledgements and
particles; trial results report thread_id and existing.
- Scheduled launches: stable message id scheduled-<task_run_id>, server-owned
deerflow_scheduled_origin metadata with the user-language parts, run thread
title for fresh-thread tasks.
- Worker records goal continuations in the verdict; lead prompt language rule
covers stored task text and scheduled runs.
- Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs.
* feat(frontend): scheduled-task core types, formatting, errors and glossary
- throwGatewayApiError now throws GatewayApiError (status, code, params,
rawMessage); message stays the string detail or detail.message
- scheduled-task types gain stop_condition, automatic_runs_used,
active_run_status, origin/notes, run_number/total_tokens/summary, thread
relation, trigger result, renewal and scheduled origin; isTaskBusy and
hasActiveRun helpers
- api: fetchScheduledTask, resume with optional renewal body, typed trigger
result and update payload with null clears
- hooks: polling at 3 s while a run is active (list, thread list, run
history), useScheduledTask with live gating and final 404, useInView,
localized error toasts; run history refreshes tasks when a run finishes
- new pure modules: errors.ts (contract code to localized copy), format.ts
(relative task times, viewer time, display zone), status.ts, actions.ts,
run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked
goal checks, describeTaskOutcome and parseStopRunId
- features: scheduled_tasks block with defaults when missing
- buildRunContext sends client_timezone
- titleOfThread takes a localized untitled label; breadcrumb names scheduled
tasks and agents; reference chips use the localized untitled label
- schedule input: shared browserTimeZone, labelled selects, aria-pressed
type toggles, interval minimum from the hint template
- i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes,
recipe instructions moved into i18n
- e2e mocks follow the new task, trigger, resume, thread-task and features
shapes; expectNoRawIdentifiers helpers for e2e and DOM tests
- tests for errors, format, cron, status, actions, goal outcome, run errors,
hooks, run context, thread titles and features; scheduled e2e copy updated
* feat(frontend): list-first scheduled tasks page with outcomes and editable goals
- page.tsx is now a thin composition of components/workspace/scheduled-tasks:
status tabs (All/Active/Paused/Finished) and search over the task list,
?task_id= deep link with router.replace on selection, ?thread_id= chip
with Show all, empty state, scheduler-off notice and unavailable state
- task list rows show the schedule in words and one status line (next run,
running now, paused by agent, auto-paused, finished by its cap)
- task detail: status badge, actions from availableActions with reasons,
More actions menu (Duplicate, Copy task ID, Delete), outcome notice,
Runs / Stops when / Each run's goal / Does / Notes from chat / History,
origin chat link; success toasts for every action
- run rows: run number or Trial run, summary, goal and assumption badges,
continuations, tokens, Open chat link; raw errors and the evaluator
reason only behind Details
- create/edit/duplicate dialog with goal, stop condition, safety caps and
advanced agent/context options; PATCH sends only changed fields, null
clears a cap
- renew dialog raises or removes caps and resumes in one request; a
limits_exhausted Resume opens it
- removed i18n keys the old page used; added feedback.resumedNoTime
- recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for
the new page; DOM tests for the detail and the form dialog
- CHANGELOG entries and the frontend AGENTS.md scheduled-task note
* feat(frontend): live schedule card, run conversation provenance and header entry
- schedule_task results (create, update, pause, resume, trial, delete) become
an assistant:scheduled-task message group rendered as a live card: schedule,
next run, stop condition with the safety cap, goal, where results go, Run
once now, Pause/Resume and Open task; the card polls the task only while it
is on screen and shows deleted, scheduler-off and limits-exhausted states
- one card per task per assistant turn; the card group is not a turn boundary
for usage, branching, editing, outline or skill attribution
- a human message with deerflow_scheduled_origin stays a human group with
scheduledOrigin and renders as a "Scheduled run / Trial run" block with the
task instructions, stop condition and notes collapsed; the launched text is
never shown and the message is not editable
- readable step labels for schedule_task actions and stop_scheduled_task
- ThreadScheduledTasksButton replaces the header link: hidden without tasks,
shows the count, links to the task (or the chat filter), and a run's chat
links to its task
- recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread
e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card,
run block, header button, step labels and reconnect hydration
- i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and
frontend AGENTS.md notes
* fix(scheduler): address verification findings
Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check.
* fix(scheduler): address review findings
* fix(scheduled-tasks): polish from live acceptance
- zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒,
and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2
days) instead of 每 1 分钟 or 每 24 小时; several units keep the number
- zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between
the interpolated time and the Chinese after it; a copy test guards every
scheduled-task zh string against a placeholder glued to Chinese
- run summary: when the first line of the final reply only leads in to a list
(ends with ":" or "："), the list items follow it as plain text (bullets,
numbering, checkboxes, bold, code and links stripped), joined with "；" for
CJK text and "; " otherwise, within the 160-character cap
- a schedule_task result (create, update, pause, resume, trial, delete) that
arrives in a chat invalidates that chat's task list, the tasks list and the
named task, so the header button and cards update without waiting for the
idle poll; results already in the loaded history do not refetch
- e2e and unit fixtures are now the live acceptance recordings (DeepSeek
Flash, demo data, fictional names): live-weekday.json (create, trial, edit,
pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1
and run 2, paused by the agent), fetched with GET from the demo Gateway and
trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card,
run-thread and tasks-page specs run in both locales over the recorded data
* docs(changelog): reference #6378 for the scheduled-tasks entries
* docs(agents): shorten the scheduled-task pointer to stay under the guidance limit
* test(scheduled-tasks): pin the live takes to their recorded time zone
The live fixtures were recorded in Asia/Shanghai, and an interval task
reads in the viewer's zone, so the 16:48 assertions failed on UTC CI
runners. Pin the browser zone in the two live-take e2e specs and the
zh live-take unit test.
* fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in
- The edit, duplicate and renew dialogs keep a stored end time's instant
while its field still shows it. A minute-precision wall value cannot
keep seconds or name the later 01:30 of a DST fall-back, so a
title-only edit moved the cap an hour earlier, and raising the run
limit re-sent the end time without its seconds.
- Creating or duplicating a task from a chat's task view drops the chat
filter when the new task does not belong to that chat, instead of
showing "This task isn't one of this chat's tasks".
* fix(scheduled-tasks): tie pause notices to the pause they describe
- Paused by agent links and times only the run named in last_error. When
that run is not loaded, the notice shows no link and the task's own time
instead of an earlier run that also asked to stop.
- The outcome notice reads the latest history page whichever page is
shown, so browsing older runs no longer swaps the auto-pause reason and
"Open latest run" to an earlier miss. It shares that page's request.
* fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes
- The edit form compares a cron in the form its schedule input emits, so an
untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only
edit no longer re-arms or refuses a finished task.
- Chat resume asks for a timezone, as update does, when a naive end_at would
land in the UTC placeholder of a zone-free task.
- Re-arming a task clears a host pause marker, as resume does, so a later
manual pause no longer reads as "Paused by agent".
* fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages
useLatestScheduledTaskRuns kept React Query's default refetch on mount,
focus and reconnect, so going offline and back while an older history page
was shown fetched the latest page again. Refreshing stays with the history,
which refreshes the latest page only while it is shown.
* test(scheduled-tasks): let the viewer's time carry its own day label
Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days,
so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the
fixed "01:00 · 09:00 your time" assertion failed depending on the hour.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs(scheduler): document tool_enabled, goal notices and goal-evaluation change
Follow-up to #6229:
- CONFIGURATION.md: add scheduler.tool_enabled to the example and notes.
- CONFIGURATION.md, README (en/zh/pt): unmet goal runs send a goal-unmet
notice instead of a completion notice, and three unmet runs add an
auto-pause notice; the docs still said only success/failed notify.
- CHANGELOG (en/zh): the #6229 entry now states the default-on goal
evaluation change for scheduled/webhook/autonomous goal runs and that
evaluator usage counts toward run usage.
* feat(scheduled-tasks): show goal outcomes and end conditions in run history
Follow-up to #6229. The API already returns these fields; the tasks page
did not show them, so a met goal read as plain "Success", an unmet
reason code rendered as a red execution error, and the goal-unmet IM
notice pointed to a workspace showing neither objective nor verdict.
- Task detail shows the per-run goal, max automatic runs and end time
when a conversation-created task has them.
- Goal runs show "Goal met" (or met relying on stated assumptions);
unmet runs show a localized reason plus the host code in neutral
styling. The run whose agent asked to stop the schedule is marked.
- Runs without a goal, and failed runs, render as before.
Read-only and frontend-only: no API change.
* docs(changelog): add #6326 scheduled-task goal outcome entry
* fix(scheduled-tasks): show readable unmet reasons instead of raw codes
Unmet runs now show only the localized reason; the host code moves to the
tooltip so it can still be matched with the IM notice. Codes without a
label are still shown verbatim.
* fix(scheduler): show readable goal reasons instead of internal codes
Task detail now localizes the host-written last_error values from goal and
stop finalization (agent stop, automatic pause, unmet reason codes) under a
matching label, and Chinese copy is tightened. Goal-unmet and auto-pause IM
notices state the reason in words instead of printing the raw code; unknown
codes are still not forwarded.
* docs(changelog): describe readable goal reasons in #6326 entry
* test(scheduler): pin host goal strings with a cross-language contract
Address review on #6326:
- Add contracts/scheduled_goal_notes_contract.json listing the task
last_error strings and unmet reason codes the tasks page and IM notices
translate. A backend test checks finalization constants, blocker and
stand-down codes against it; a frontend test checks every listed value
has a label.
- Move the two last_error strings in finalization.py into constants.
- Frontend reason lookup strips the blocked: prefix like the IM map and
now covers goal_not_met_yet, with en/zh labels.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(extensions): add full Custom Agent team collaboration plugin
* fix(extensions): register agent teams in the capability catalog
* fix(agent-teams): recover incomplete teams and preserve request capacity
* fix(agent-teams): preserve clarification waits and active handoff lineage
* fix(agent-teams): align mention and input text limits
* feat(frontend): support extension providers in composer mentions
* fix(mentions): retain settled plugin suggestions during search
* fix(extensions): validate identifiers and skip empty mention searches
* test: stabilize history pagination after latest page refresh
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): ignore literal base tags in HTML artifact previews
* fix(frontend): address HTML preview base review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
extensionMap is keyed by extension and valued by language, but the Julia row
was written backwards as `julia: "jl"`. checkCodeFile tests membership with
`extension in extensionMap`, so ".jl" never matched and a Julia artifact was
not a code file at all, while ".julia" resolved to a language named "jl".
Add both `jl: "julia"` and `julia: "julia"`, the same pair form the map already
uses for haskell/hs and elixir/ex, so no extension that classified before stops
classifying.
* fix(frontend): strip closing hashes from web fetch titles
Signed-off-by: 97three <2212371308@qq.com>
* fix(frontend): scan web fetch heading suffixes in linear time
Signed-off-by: 97three <2212371308@qq.com>
---------
Signed-off-by: 97three <2212371308@qq.com>
* fix(threads): re-anchor re-persisted history rows to their earliest seq
reconcileThreadHistoryRows collapsed a repeated (run_id, identity) to
its latest-seq copy, deleting the earliest-seq row before
buildVisibleHistoryMessages could apply its earliest-seq-wins
anchoring. A message re-persisted under a newer seq therefore jumped
to the tail of the thread, contradicting the documented contract in
this function's docstring, in buildVisibleHistoryMessages, and in the
backend get_message_seqs earliest-seq-wins rule.
Re-anchor the surviving row to the earliest seq its identity held in
the merged window after deduping, and make the retained-history
stability check value-based so re-anchored rows do not force a state
update on every pass.
Fixes#6125
* fix(threads): address history reconciliation review
---------
Co-authored-by: liwenjie200543 <175601277+liwenjie200543@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(integrations): keep Lark runtime status consistent
Probe sandbox runtime readiness for every Lark mutation response outside the credential lock. Expose explicit sandbox_runtime_probed state so the frontend can distinguish conservative fallbacks without inferring semantics from runtime detail text, while cancelling stale reads before updating the cache.
* style(frontend): format Lark cache test
* fix(integrations): snapshot Lark credential status under the lock
Why this change is needed:
PR #5994 moved the post-mutation status read outside the per-user
credential lock so the new sandbox runtime probe would not extend the
critical section. As flagged in review, that lets a concurrent app
switch or auth flow land between the committed mutation and the status
read; in remote-provisioner mode the runtime probe can spend seconds
before probe_lark_auth runs, so the raced switch can clear the new
tokens and make a completed login report success=false, or mix an old
app_id with a newer auth state.
What changed:
- add _LarkCredentialSnapshot (app config + auth probe) and capture it
in the same _lark_credential_lock section that commits the mutation
- thread the snapshot through _get_lark_mutation_status into
get_lark_integration_status(credential_snapshot=...); only the
credential-independent runtime probe still runs after the lock
- apply to complete_lark_auth, complete_lark_config, and
set_lark_app_credentials; install keeps the one-shot status build
Problem solved:
Mutation responses stay tied to the just-committed credential
generation: a raced app switch can no longer flip a completed
device-code login to success=false or blend a stale app_id with newer
auth state, while the runtime probe stays off the lock.
* fix(integrations): address Lark status review follow-ups
- install_lark_integration: snapshot credential-derived status fields
under _lark_credential_lock so the install response cannot mix newer
credential state read across the unlocked runtime probe
- useInstallLarkIntegration: route the cache write through
cacheLarkMutationStatus so install cancels in-flight reads and keeps
probed runtime fields under frontend/backend version skew
- README: soften the UI claim - the status API exposes
sandbox_runtime_probed and the mutation cache honors it
* test(integrations): tighten Lark status fixtures and guidance docs
- cache.test: use a distinct mutation runtime mode in the unprobed-merge
case so the assertions prove the cached runtime fields win
- e2e mock: include broker in the sandbox_runtime_mode union cast so the
fixture cannot drift from LarkSandboxRuntimeMode
- AGENTS.md (gateway + skills): mention sandbox_runtime_probed alongside
the existing runtime readiness fields, keeping the gateway guidance
file under its byte budget
* fix(integrations): harden Lark mutation cache against skew and races
- cacheLarkMutationStatus: await cancelQueries before writing so an
in-flight GET whose fetch already resolved cannot clobber the fresher
mutation status, matching the existing threads/archive pattern
- self-heal version skew: an unprobed (pre-field backend) mutation
response now triggers a status refetch instead of risking the
conservative runtime fallback lingering until the next focus/remount
- api: normalize a missing sandbox_runtime_probed to false at the
boundary so the declared-non-optional type matches skewed runtimes
- tests: pin the skew contract (missing-field cases for cache and api)
and add a useInstallLarkIntegration regression test proving the
install cache write goes through cacheLarkMutationStatus
- README: describe the probed flag as cross-version semantics instead of
claiming the same-version API surfaces both states
- lark_cli: note that install's credential snapshot runs the auth probe
subprocess while holding the credential lock
* refactor(integrations): derive sandbox_runtime_probed inside the resolver
_resolve_sandbox_runtime_readiness now returns a fourth probed element
next to the branch that produces the unevaluated remote fallback, so the
'was readiness actually evaluated' invariant can no longer drift from the
fallback logic it mirrors. Callers and resolver-level tests updated to
pin probed per branch.
* fix(mcp): isolate personal connections from deployment configuration
* fix(mcp): preserve personal task ownership and repair plugin E2E fixtures
* fix(mcp): handle corrupt personal config and cache owned calls
* fix(mcp): pin personal endpoint addresses and handle invalid listings
* fix(mcp): recheck current authority for privileged personal connections
* fix(frontend): keep additional_kwargs on the optimistic bubble through an upload
When a send carried files, the upload-complete update replaced the
optimistic human message's additional_kwargs with only the uploaded
files. Quote and conversation reference chips vanished until the server
echoed the message, and project attachments staged with the send were
missing from the bubble both while uploading and after. The submitted
message was always intact.
The optimistic copy (before and after the upload) and the submit now
build additional_kwargs through one helper,
buildHumanMessageAdditionalKwargs, and share a single uploaded-files
value, so the bubble always matches what was sent.
* docs(changelog): reference #5982
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(agents): persist tool artifacts as resolvable handles (#4676)
MCP and sandbox tools return file paths, URLs, and task ids that are
lost when context compaction summarizes the conversation. This adds a
per-thread artifact handle registry so the model can reference
artifacts by short deterministic handles (art_xxxxxxxx) that survive
compaction and resolve to real references at the tool-call boundary.
- ThreadState.tool_artifacts: reducer channel capped at 100 entries,
same-handle latest-wins, survives summarization compaction
- artifact_registry: deterministic handle derivation from
(thread_id, tool_call_id, call_index); extraction precedence
structured_content -> file/image blocks -> conservative text scan;
HTML-escaped model-context rendering with char budget
- ArtifactCaptureMiddleware: before_model hook captures lightweight
metadata (never content) and tracks handle consumption
- ArtifactResolutionMiddleware: rewrites handles in tool-call args to
real references before execution; unknown handles untouched
- DurableContextMiddleware: projects the registry into the hidden
durable-context data block ([available]/[consumed] status)
- config: new tool_artifacts section (all switches default on)
- frontend: artifact badges on tool-call steps keyed by tool_call_id
Fixes#4676
* fix(agents): harden artifact registry after review (#4676)
Review findings on #4929:
- capture/consumption updates emitted in the same before_model call are
now concatenated instead of dict-merged; dict-merging clobbered the
shared tool_artifacts key and permanently dropped fresh captures
- handles embed a per-result ordinal so multiple references inside one
tool result never collide (two paths in one text block previously
collapsed into one handle)
- structured_content extraction pulls concrete refs from known keys
(file/path/url/task_id/..., string or list-of-string values) instead
of storing a truncated JSON preview, falls back to an untruncated
whole-object data entry, and no longer skips file/image content blocks
- plain-string tool results (e.g. bash/write_file output) are scanned
for sandbox paths / remote URLs too, gated by detect_refs_in_text
- wire max_entries through configure_tool_artifact_max_entries at
assembly (clamped 10..1000) and gate detect_refs_in_text scans
- resolution requires both enabled and resolve_handles_in_args,
matching its docs
- add real ToolCallRequest.override coverage and the spec 14.4
integration cycle (capture -> inject -> resolve -> consumption)
* fix(agents): address second-round artifact registry review (#4676)
Follow-up findings on #4929:
- drop generic `output`/`outputs` from structured ref keys: prose under
those keys no longer becomes a bogus file entry (falls back to the
whole-object data entry; path-shaped refs are still caught by the
free-text scan)
- strip closing quotes, brackets, and backticks from detected refs so
markdown- and JSON-wrapped paths resolve to clean real references
- remove the process-wide cap setter: the reducer enforces only an
absolute ceiling (1000) and ArtifactCaptureMiddleware applies the
configured max_entries per agent when emitting updates, so agents
with different configs in one process never share or flap a cap
- skip extraction for tool messages whose tool_call_id already has
registry entries and memoize known-empty results (bounded FIFO), so
steady-state capture cost drops to the new message tail instead of a
full-history rescan per model call
- correct the inverted ordering claim: capture is position-independent
in the wrap chain because it is a before_model hook reading state
messages; fixed the assembly comment, AGENTS.md item 15, and the
ordering test docstring
CI parity verified locally: ruff check + format --check, full offline
suite, blocking-io suite, collect-only, uv lock --check,
check_agent_guidance, frontend format/lint/typecheck/build/tests.
* fix(agents): sliding-window cap and URI gates for artifact registry (#4676)
Third-round review findings on #4929:
- max_entries is now a sliding retention window: fresh captures are
always registered, and when the projection exceeds the configured
cap the middleware appends a trailing {"op": "trim_to", "keep": N}
directive that the reducer applies by evicting the oldest entries.
The previous remaining-budget emission froze the registry at the cap,
permanently dropping post-compaction artifacts — the exact failure
mode #4676 targets — and silently disabled the round-2 rescan
optimizations at the point threads grew longest
- _track_consumption memoizes settled tool-call args in a bounded FIFO
alongside the existing empty-result memo, so historical AIMessage
args are not regex-rescanned every model call
- file/image content-block URLs must be http(s) or path-shaped;
data:/blob: URIs (MCP embedded resources carrying base64 payloads)
can no longer enter thread state, tool args, or crowd out the render
budget; protocol-relative //host forms are rejected as well
- structured-content ref values go through the same URI-shape gate as
content blocks, closing the embedded-payload hole via known keys such
as url/urls
- tool_artifacts.enabled now gates the whole middleware including
consumption tracking via an early return in before_model
Docs updated to describe the sliding window, URI gates, enabled scope,
and both bounded memos.
* fix(agents): final evictions and pending-aware consumption for artifact registry (#4676)
Fourth-round review findings on #4929:
- the sliding window interacted with the seen_call_ids skip to
resurrect evicted entries every round: once an entry was trimmed, its
tool_call_id left the seen set while its ToolMessage stayed in
context, so the next model call re-extracted it, re-registered it as
newest, and evicted something else — a never-converging rotation.
The empty-result memo is now a unified _processed_results memo that
records every extraction regardless of outcome, making evictions
final; steady state over a saturated thread is quiescent (pinned by
a multi-round test asserting no updates and zero extractions)
- the consumption scan built its handle map from pre-capture state, so
a call referencing a handle captured in the same before_model call
missed, got settled as quiet, and its consumption mark was lost
forever. _track_consumption now receives the pending capture entries
and resolves against state + pending; a call settles as quiet only
after every referenced handle resolved, so transient misses retry
next round instead of permanently suppressing the mark
AGENTS.md entry 15 updated to describe both memos and the
eviction-is-final semantics.
* fix(artifacts): address occurrence identity, PII and URL review findings
* fix(artifacts): enforce resolved policy checks and bounded registry handling
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
CommonMark forbids an info string on a closing fence, so "```python" inside a
plain ``` fence is content, not the end of the block. The closer check that two
fence consumers share only compared character and run length, which ends the
block early and pushes whatever follows back into prose: on the render path a
system marker inside the sample loses its tags and its content becomes visible
text, and on the export path stripInternalMarkers deletes real user content from
the thread markdown.
mermaid.ts already had the correct predicate privately. Move it into the shared
leaf module and use it from both consumers, so the three fence scanners in
core/streamdown and core/messages agree on what closes a block.
* fix(projects): label untitled threads in the conversation-files view
GET /api/projects/{id}/threads/files returns each member thread's
display_name as `str | None`: the thread meta row is created with NULL and
only filled once title generation has run after the first reply, or never
if that step fails. The frontend's ProjectThreadFileGroup declared the field
as a required string and ConversationFileGroup rendered it verbatim, so a
thread whose files were uploaded before its first reply showed them under
an empty heading. The sibling ProjectThread type already models the field
as nullable and every consumer of it falls back to t.projects.untitled.
Make the group type nullable to match the wire, and render the heading
through a shared threadTitle() helper that both the thread-name map and the
file group now use, so the "Untitled" rule lives in one place.
The DOM test covers null, missing and whitespace-only titles.
* docs(changelog): reference #5879 in the conversation-files entries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs(checkpoints): add a bilingual checkpoint storage manual
Add the missing docs-site coverage for dual-mode checkpoint storage (`database.checkpoint_channel_mode: full | delta`) as a ten-chapter manual under `harness/checkpoints/` in both languages: concepts, quick start, channel modes, snapshot cadence, history cache, resume and rollback, operating checkpoints, observability, troubleshooting by symptom, and a reference of every configuration key, error message, HTTP status mapping, public symbol, load-bearing test, and pinned upstream defect.
The index page is `asIndexPage`, so `/docs/harness/checkpoints` is the section itself and its `_meta.ts` lists only the nine child chapters. Discovery: `harness/_meta.ts` registers the section, the harness configuration module table gains a `database` row, and the thread persistence section of `agents-and-threads` links to the manual.
Content is grounded in the code. Only the three measured values recorded in `backend/docs/runtime-guidance-details.md` are quoted; the upstream-derived 41x storage figure in the untracked delta-channel RFC is deliberately not cited. Pinned upstream defects are documented with their issue and PR numbers and the versions they are pinned at (`langgraph 1.2.9`, `langgraph-checkpoint 4.2.0`, `langgraph-checkpoint-postgres 3.1.2`).
* docs(checkpoints): correct the history-cache restart requirement
The history cache is built with the checkpointer at Gateway startup (`langgraph_runtime()` -> `make_checkpointer()` -> `make_checkpoint_cache()`), so editing `database.checkpoint_cache.*` does not change a running process. The manual previously marked the whole section "not restart-required", which conflated "not frozen for correctness" with "hot-reloadable" and could mislead an operator changing capacity, credentials, or retention.
Distinguish the two properties everywhere the section is described: history-cache.mdx marks every field as needing a restart and adds a callout explaining the difference, reference.mdx redefines the Restart column and fixes the five rows and the effect list (adding the `asyncprov` evidence alias), troubleshooting.mdx replaces the "two keys need no restart" claim and gains a dedicated symptom entry, and observability.mdx notes that changed counters need a restart. The Chinese text is corrected in parallel.
Verified: the docs link test and the docs unit suite pass, frontmatter parses for all 20 pages, and every in-page anchor resolves on the rendered English and Chinese history-cache, troubleshooting, reference, and observability pages.
* docs(checkpoints): account for periodic snapshots in the storage-growth claim
Every periodic snapshot stores the entire message list as of that step (`_DeltaSnapshot(ch.get())`, persisted as a checkpoint blob), so with a fixed snapshot frequency K the retained total after N turns is O(N²/K + N), not O(N). The manual claimed unconditional linear growth, which understated capacity planning — the more so because nothing in DeerFlow prunes those snapshots yet.
Distinguish per-step delta writes from total retained storage in index.mdx (prose and table), modes.mdx (step-cost row), and operations.mdx; note in cadence.mdx that K is also the capacity knob that scales the snapshot bytes. The English and Chinese chapters are corrected in parallel. The reference chapter keeps the released #4292 wording as a verbatim quotation and now names the retained-storage bound beside it.
Verified: `_DeltaSnapshot(ch.get())` and its blob serialization in the pinned langgraph (channels/delta.py:196-198, checkpoint/serde/types.py:19-25) confirm the premise; the docs link test and docs unit suite pass; all ten changed pages render with resolving anchors and no stale linear-growth wording.
* docs(checkpoints): correct the run-error surfaces and repair two evidence pointers
Review follow-ups on the checkpoint manual, both languages.
Error-surface corrections:
- Run creation is not always asynchronous: when a thread's run-event feed is
empty but a checkpoint head exists, ensure_checkpoint_history_seeded reads
that head inside start_run, where only ConflictError (409) and
UnsupportedStrategyError (501) are mapped, so a mode mismatch returns a
plain 500 before admission. Troubleshooting, observability and the
reference route tables now scope the "does not pre-gate" claim.
- The 503 symptom is retargeted. Only Gateway startup, make_lead_agent and
DeerFlowClient.__init__ call the freeze helpers, so a config.yaml edit
cannot make a state/history/thread route answer 503. A mode edit is
injected over by the run worker and ignored until a restart; a cadence edit
fails the next run with the restart-required message. The router's 503
mapping is marked defensive in modes and the reference.
- The regenerate section title no longer groups the state endpoints, which
map a mismatch to 409.
- operations: the Textual TUI catches the mismatch through stream_actions();
only the headless --print/--json runners print a traceback.
Evidence-pointer repairs:
- reference: this branch's CHANGELOG insertion shifted the #5734 entry from
3032-3039 to 3040-3048; the citation is re-pointed.
- reference: the #8548 row cited an untracked worktree plan file; replaced by
the in-repo test anchor, and "mirror issue #8551" reworded to "related".
- reference: new `agent` alias for the agent-factory freeze sites.
- history-cache: the bare `deps:`/`asyncprov:` aliases are expanded to full
paths; the alias legend lives only on the reference page.
* docs(checkpoints): say a delta fork is accepted, not refused
"Delta mode cannot fork" overstated the constraint. Nothing in the stack
rejects the fork: LangGraph stores the second child, no saver checks for an
existing sibling, and ensure_checkpoint_mode_compatible returns immediately
when the process mode is delta. What breaks is the read — the walk replays
the abandoned sibling's writes stored under the shared parent.
Reproduced on the pinned stack (langgraph 1.2.9 / langgraph-checkpoint
4.2.0, AsyncSqliteSaver) by running this repo's own #4458 fixture without
the linearization call: the forked resume returns
["h1", "a1", "h2", "a2", "a2-new"] instead of ["h1", "a1", "h2", "a2-new"],
i.e. the superseded answer comes back.
- resume-and-rollback: title/description/heading now say "cannot fork
correctly"; new "Forks are accepted, not refused" subsection covering the
unguarded write side.
- troubleshooting: the "old answer reappears" cause states the same split.
- reference: the POST /threads/{id}/state row notes that a client-supplied
checkpoint_id older than the head is applied as a delta fork with no
linearization — the write shape upstream tracks as #8551 (open), which the
run-path linearization does not cover.
* docs(checkpoints): scope the unguarded state-update fork to what the probe showed
The previous commit said a mutation against an older checkpoint stores its
writes under the shared parent and implied the sibling branch suffers for it.
Probing the pinned stack (AsyncSqliteSaver, delta messages channel) shows:
- the write does land under the shared parent and joins the write set the
parent's other children walk (raw walk for a sibling includes the abandoned
head's own write);
- but reducer channels are written with Overwrite, which replaces the
accumulated value, so the forked branch reads clean and the abandoned
branch is unchanged in that shape — #8551 did not reproduce;
state the masking mechanism and the upstream tracking issue, not a corruption
claim.
* docs(checkpoints): fix a frontmatter description that broke the docs build
`resume-and-rollback.mdx` shipped `description: Delta mode cannot fork a
thread correctly: a forked delta read ...` in the previous commit. The
unquoted ": " turns the value into a nested mapping, so Nextra's YAML
frontmatter parse failed and the e2e lane could not start its web server
(YAMLParseError at line 2, column 14) — the exact failure class this manual
warns docs authors about, introduced by me two commits ago.
- Replace the colon with an em dash in the English description (the Chinese
one uses a full-width colon and was unaffected).
- Add tests/unit/content/docs-frontmatter.test.ts: every `src/content/**/*.mdx`
frontmatter block must be YAML-parseable, implemented as the one rule that
actually breaks it (an unquoted scalar containing ": "), because the unit
suite never compiles MDX and only a build or a dev render catches it.
Verified the guard fails on the broken line and passes on the fix.
Verified with a clean `BETTER_AUTH_SECRET=local-dev-secret pnpm build` on this
tree: all pages prerender, resume-and-rollback EN/ZH included.
* feat(models): add declarative reasoning capability contract (#5073)
Introduce an optional per-model `reasoning:` block beside the legacy
`supports_thinking` / `supports_reasoning_effort` booleans. It declares
thinking availability (unsupported | optional | required), the accepted
effort values with aliases, default, and serialization path, the payload
dialect, and the reasoning-history requirement. When present the booleans
are derived from it and contradictory profiles fail at config load.
`deerflow.models.reasoning` normalizes legacy and declared profiles into
one immutable contract and resolves every caller's generic request against
it. `create_chat_model` is the single enforcement point: required-thinking
models never receive a synthesized disable payload, unsupported effort
values never reach the provider, and the dialect synthesizes the on/off
payload when no template exists. Legacy profiles keep the historical path
byte-for-byte. The lead agent and subagent descriptor resolve the same
policy first so run metadata reports effective values.
`/api/models` and `DeerFlowClient` project the contract as `reasoning`;
the frontend derives mode clamping and effort choices from it through
`core/models/reasoning.ts`, and user preferences accept provider-specific
effort tokens. The setup wizard's Z.AI GLM-5.3-Flash profile migrates to
the contract, restoring its low/high/max effort control.
* docs(models): move the reasoning contract guidance into models/AGENTS.md
backend/AGENTS.md sits within 46 bytes of the 96 KiB effective-chain hard
limit for the middlewares, runtime, sandbox, and subagents guides, so the
new section pushed scripts/check_agent_guidance.py over it. The models
guide is the nearest file for the factory and the contract module, and
its chain has ample room.
* fix(models): address review on the reasoning contract
- vLLM dialect: the enable path now mirrors whichever chat_template_kwargs
switch the template declares, like the disable path already did, so a
template using the older `thinking` key no longer also grows
`enable_thinking`.
- Validate operator-supplied effort values at `effort.path` inside
`when_thinking_enabled`, `when_thinking_disabled`, and the `thinking`
shortcut against the contract, not only the profile-level value; they are
forwarded when the caller chooses nothing, so the "unknown values never
reach the provider" guarantee now holds for them too.
- Constrain `effort.path` to dotted identifiers and reject single-segment
paths that would replace a whole mapping (`extra_body`, `thinking`,
`model_kwargs`, `default_headers`, `default_query`).
- Z.AI GLM-5.3-Flash profile: `default: high` instead of `max`, because the
default also governs summarization, title generation, and subagent calls
that never choose an effort; `max` stays selectable. Documented in the
config example, the configuration manual, and the design note.
* fix(models): close reasoning contract review gaps
* fix(models): preserve native Ollama reasoning config
* fix(models): keep native reasoning strings and project reasoning into the descriptor
- `ModelConfig.reasoning` also accepts the level string ChatOllama takes
for gpt-oss style models (`low` / `medium` / `high`); the factory forwards
both the boolean and the string on the legacy path, so profiles that
passed `reasoning: high` through before the contract field existed keep
loading and keep sending it.
- The assembly descriptor no longer excludes `reasoning` from
`model_parameters`: the native ChatOllama value and a declared contract's
dialect/history/effort change the request payload, so they now move the
assembly fingerprint.
- e2e: the sidecar spec pinned the pre-contract body where a model without
effort control still received `reasoning_effort: minimal`; the preset is
now dropped for such models (the Gateway discarded it anyway), so the
assertion expects the key to be absent.
* fix(composer): keep IME-composition Enter out of the slash-skill catalog
`handlePromptTextareaKeyDown` called `handleSkillSuggestionKeyDown` with no
composition guard, unlike its two siblings: the inline-skill editor wraps the
same call in `isIMEComposing(...)` with a comment naming this exact hazard, and
`handlePromptHistoryKeyDown` checks it too. `PromptInputTextarea` calls the
prop's `onKeyDown` *before* its own `isIMEComposing` check, so the suggestion
handler won the event.
With an IME active and the `/` catalog open, pressing Enter to commit the
composition instead ran `event.preventDefault()` and applied the highlighted
suggestion — discarding the text still being composed and selecting a skill the
user never chose. The catalog opens from the live `textInput.value`, and React
`onChange` fires during composition, so a CJK user typing `/res` to reach a
skill hits it directly.
Guard the main composer's call site the same way the inline-skill editor does,
and pin it with a happy-dom regression test: `keyCode: 229` (the keydown a
browser reports for a key an active IME consumed) must leave both the catalog
and the draft untouched, while a plain Enter still selects the highlighted
skill.
Verified:
- new test `tests/unit/components/workspace/input-box-skill-suggestion-ime.dom.test.tsx`
fails on main (the IME case applies the skill) and passes here; the plain-Enter
control case passes on both
- `rstest run --pool threads tests/unit/components/workspace` -> 69 files / 448
tests passed
- `tsc --noEmit`, `eslint`, `prettier --check` clean on the touched files
* fix(composer): ignore the Enter that follows IME compositionend
Safari emits the confirming Enter after compositionend, with neither
isComposing nor keyCode 229. Drop that Enter in PromptInputTextarea for
a short window, before the skill catalog or prompt history can treat it
as a plain key.
---------
Co-authored-by: Shxiao101 <Shxiao101@users.noreply.github.com>
* feat(memory): add cognitiveStyle for collaboration preferences
Extend cross-session memory with user.cognitiveStyle and cognitive facts,
backward-compatible normalization, prompt injection as Thinking Style, and
settings UI. Add design notes and unit tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(memory): normalize legacy memory imports on frontend
Add normalizeMemoryPayload aligned with backend normalize_memory_data,
use it for Settings import and API reads, and add unit/regression tests.
Document schema-evolution steps and sync FACT_EXTRACTION_PROMPT cognitive category.
* fix(memory): harden legacy memory normalization
* fix(memory): keep import envelope validation strict
* fix(memory): keep unknown fields through frontend import normalization
normalizeMemoryPayload rebuilt the memory document from a section whitelist,
so every key it did not name was dropped. Because readMemoryResponse routes
all memory API responses through it, the loss showed up on read, export, and
import alike: revision vanished, and any future backend field would silently
break lossless export/import.
The Gateway models are extra="allow" and deermem normalizes additively, which
left the frontend as the only lossy hop in the chain. Spread before overriding
at each nesting level so normalization only validates and defaults
contract-owned fields, leaving the envelope check as strict as before.
Covered by a unit test on the strict import path, two backend tests for
extension preservation and caller immutability, and an E2E round-trip that
asserts on the captured import request body.
* docs(memory): state the normalization invariant that actually holds
The docstring cited `display` as a backend-driven field alongside
`revision`, but #4546 was closed unmerged and no such field exists in the
contract. It also promised lossless export/import, which the Gateway's
`extra="ignore"` models preclude for any undeclared field.
Replace both with the invariant the code is really defending: the frontend
must not be narrower than the Gateway contract, using the declared
top-level `revision` as the concrete example.
* fix(memory): preserve legacy migration and reject lossy imports
* fix(memory): align legacy fact normalization defaults
* test(memory): use authenticated app for import regression
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(authz): cover create/run/memory/agent routes with permission checks; scope USER.md per user
Route permissions: the AuthorizationProvider model only applied to routes
carrying @require_permission, so POST /api/threads, /api/threads/search,
POST /api/runs/stream and /api/runs/wait (runs:create), every /api/memory
route and the custom-agent routes ran with authentication only — a
provider configured to deny threads:write/runs:create/... could not
enforce those decisions. Add the missing decorators (threads:write,
threads:read, runs:create, memory:read/write, agents:read/write) and
register the new permission names in authz.Permissions. No owner checks
are added: per-user data scoping stays in the repository layer.
User profile: GET/PUT /api/user-profile read and wrote a single global
{base_dir}/USER.md, so in a multi-user deployment any authenticated user
could overwrite the prompt context injected for everyone else (and read
it). Scope the file to the caller's bucket
({base_dir}/users/{user_id}/USER.md) like user-scoped skills; no other
consumer of the old global path exists in the tree (verified by grep and
the updated tests). test_put_user_profile asserted the wrong effective
user under the autouse conftest user fixture; fixed to test-user-autouse.
* fix(authz): bind positional args in require_permission; migrate legacy USER.md
Review follow-up on #4989 (willem-bd):
- require_permission's wrapper only looked for request in keyword
arguments, so direct positional calls like
create_thread(body, request) made the injected keyword stub collide
with the positional request (TypeError: multiple values). The wrapper
now binds the wrapped signature via inspect.signature().bind() and
honors a positionally-passed request (and thread_id) instead of
assuming kwargs; the test-stub injection only fires when request is
absent everywhere. The two positional callers in
tests/test_threads_router.py pass again (8/8 channel tests).
- migrate_user_isolation.py now claims the legacy global USER.md for
--user-id (default 'default') like the other unowned legacy
artifacts; without it, upgrading installs with an existing profile
would read content: null and later strand the old file beside the
new per-user one. Conflict handling mirrors migrate_memory (rename to
USER.legacy.md).
* style(scripts): keep migrate_user_isolation help within the line budget
* test(authz): give internal-request stubs realistic auth fields
The create_thread permission wrapper added in this PR authenticates
the direct calls in the internal-owner tests; their SimpleNamespace
requests had state.user but no auth_source and no cookies, so
get_current_user_from_request fell through to request.cookies.get and
raised AttributeError (verified introduced by this branch: both tests
pass on the base commit).
The stubs now carry cookies={} and
state.auth_source=AUTH_SOURCE_INTERNAL, which is exactly what
AuthMiddleware stamps on real internal requests, so state.user is
honored without the JWT path. All 80 tests in the file pass.
* fix(pat): keep memory/agent permissions out of PAT scopes by design
The route permissions exist (they guard the memory/agent routers) but PATs
govern the thread/run lifecycle only: _PAT_ROUTE_RULES default-denies those
routers for PAT callers regardless of scopes. The alignment invariant
becomes a subset check plus a pinned exclusion, and pat.py documents that
opening these scopes is a product decision requiring three synchronized
changes.
* test/docs: cover migrate_user_profile; drop the unclaimed USER.md injection wording
Five-scenario test class mirroring the sibling migration steps (move,
conflict-rename, noop, both under dry-run). The GET/PUT descriptions no
longer say USER.md is 'injected into agents' — nothing at this head consumes
it for prompts; the routes are storage/retrieval only. The memory AGENTS.md
migration pointer now enumerates skills/ and the global USER.md.
* docs(paths): align the USER.md layout comment with the injection disclaimer
* test(authz): align effective permission contracts
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
In grouped mode the sidebar indents a project's thread rows with a nested
SidebarMenu carrying ml-4, but SidebarMenu defaults to w-full. Width 100%
plus a 16px left margin overflows the sidebar by 16px, so the absolutely
positioned row kebab (right-1) lands past the visible edge and gets
clipped. The Archived group nests one level deeper and overflows 32px,
hiding its kebabs entirely.
Swap w-full for w-auto on both indented menus so the block-level flex
container fills the remaining width minus its margin. A dom test renders
ProjectsSection in grouped mode from seeded query caches and pins that
every indented menu drops w-full while the root menu keeps it.
Fixes#5681
* fix(frontend): read web_fetch titles that start with blank lines or indented headings
* docs(frontend): describe the indented-code guard as it actually behaves
* fix(frontend): reject mixed code indentation in web-fetch titles
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(authz): gate composer send on runs:create (Phase 4 follow-up, #4063)
Mirrors #5294's stop gating on the send side: both chat routes pass
canCreateRuns (from PERMISSIONS.RUNS_CREATE, lockstep with the backend
enum) into the shared composer. The gate sits at the top of
submitThreadMessage — the single choke point every composer entry
(submit button, Enter, goal-set-triggered run) funnels through — and
denies with a toast plus a rejected promise so PromptInput keeps the
text. The idle submit button is disabled and explains the boundary via
conditionally-spread aria-label/title (startTurnUnavailable, en/zh);
while streaming the button stays the runs:cancel stop affordance.
Also removes the unreachable kind === "stop" branch in handleSubmit
(flagged during #5294's review): the Enter path early-returns with the
streaming toast before the classifier runs.
* fix(frontend): reject denied goal starts before saving state
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
