github/deer-flow
1
0
Fork 0
mirror of https://github.com/bytedance/deer-flow.git synced 2026-08-10 23:08:45 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,642 Commits 50 Branches 8 Tags
Commit Graph

501 Commits

Author SHA1 Message Date
qin-chenghan
ad45f59d66
feat(memory): pluggable memory abstraction with self-contained DeerMem backend (#4122) 
* feat(memory): pluggable + self-contained memory system (MemoryManager plan phases 1 & 2)

Phase 1 — Pluggable (steps 0-10):
- ABC MemoryManager (9 methods) + singleton factory + drop-in backend discovery
- DeerMem default backend with core/ (storage/queue/updater/prompt/message_processing)
- NoopMemoryManager backend (proves pluggability)
- All call sites (middleware/hook/prompt/gateway/client/app) routed through manager
- hasattr capability probing for DeerMem-internal methods (no hard imports)
- MemoryConfig gains manager_class field; shared vs DeerMem-private annotated

Phase 2 — Self-contained DeerMem (steps 11-18):
- backend_config passthrough + DeerMemConfig (all DeerMem-private fields moved off MemoryConfig)
- DI: DeerMem owns storage/queue/updater/llm as instance attributes (no global singletons)
- Storage independence: core/paths.py with own root (~/.deermem or ),
  factory auto-injects deer-flow's runtime_home() as absolute base_dir (zero-config)
- LLM independence: core/llm.py via langchain init_chat_model (no create_chat_model)
- Trace independence: optional tracing_callback replaces inject_langfuse_metadata/request_trace_context
- Message processing independence: hide_from_ui default-skip + optional should_keep_hidden_message hook
- Internal imports → relative (only deer_mem.py ABC import is host-relative)
- Carrier (deer_mem.py adapter) / portable (deermem/ config+core) split
- New tests: test_deermem_self_contained + test_memory_manager_pluggable; all memory tests migrated
- Other-agent demo: samples/other_agent_demo/ + automated portability test
- config.example.yaml memory section updated to phase-2 schema

* feat(memory): port consolidation + staleness fix into self-contained DeerMem; phase-2 host hooks

Port upstream #3996 (memory consolidation) and #3993 (staleness KeyError fix)
from origin/MemoryManager into the pluggable, self-contained DeerMem structure
(backends/deermem/deermem/), adapted to the DI MemoryUpdater (config injected,
not get_memory_config globals):

- DeerMemConfig: add consolidation_enabled (opt-in, default false) /
  consolidation_min_facts / consolidation_max_groups_per_cycle /
  consolidation_max_sources
- prompt.py: factsToConsolidate JSON field + {consolidation_section} placeholder
  + CONSOLIDATION_PROMPT constant
- updater.py: _coerce_source_confidence / _select_consolidation_candidates /
  _build_consolidation_section module helpers (matching the existing
  _select_stale_candidates style); consolidation normalization in
  _normalize_memory_update_data; consolidation apply in _apply_updates (after
  max_facts trim, with apply-time guardrails mirroring staleness); staleness
  KeyError fix (f["id"] -> f.get("id") is not None) applied to both the
  staleness guardrail and the consolidation allowed_source_ids comprehension
- config.example.yaml: consolidation section under memory.backend_config
- tests/test_memory_consolidation.py: 40 DI-adapted tests (running, not skipped)
  incl. the staleness KeyError regression

Also includes in-flight phase-2 host-integration work: storage_path semantics
(any absolute/relative value = root dir) and host-default tracing_callback /
should_keep_hidden_message hooks injected into backend_config by the factory.

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(memory): add noop backend template and backends guide

- backends/noop/: complete drop-in template (config.py with zero deer-flow
  imports, noop_manager.py with a 6-step new-backend walkthrough in its
  docstring, commented optional fact-CRUD capabilities).
- backends/README.md: which files to touch when adding/swapping a backend,
  the 5-item backend contract, and common pitfalls.
- manager.py: generalize backend examples in comments (drop mem0-specific
  references).

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(frontend): guard formatTimeAgo against invalid timestamps

Return a neutral placeholder when the input date is invalid (e.g. an empty lastUpdated from a backend with no memories) instead of throwing 'Invalid time value' from date-fns.

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(memory): wire tool-driven memory mode through the MemoryManager ABC

tools.py (memory_search/add/update/delete) now calls get_memory_manager()
instead of the removed host memory module, so tool mode (memory.mode: tool)
works for any backend. DeerMem.search is implemented (case-insensitive
substring match, ranked by confidence) as a stand-in for the planned
semantic retrieval; noop.search returns [] (unchanged). Fact-CRUD tools
use getattr+callable probing -- backends lacking those ops (noop) get a
clear JSON error instead of crashing.

Tests: test_memory_tools rewired to mock the manager (handler tests) +
TestModeGating retained; test_memory_search now covers DeerMem.search;
pluggable stubs test updated (search no longer a stub).

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: resolve lint errors (import sorting, type annotation quotes, E402 in skipped tests)

* docs: restore explanatory comments in config.example.yaml memory section

* fix(security): port html-escape memory facts fix (#4097) to vendored DeerMem prompt.py

* fix(memory): address review + port dropped upstream memory fixes

Review blockers (vendored DeerMem):
- #4044 restore _escape_memory_for_prompt (current_memory blob in
  MEMORY_UPDATE_PROMPT) - prevents </current_memory> breakout
- #4028 html.escape staleness-section cat/content in _build_staleness_section
- #4119 add _escape_summary for injection-path summaries (Work/Personal/
  Current Focus/Recent/Earlier/Background)
- default-model silent no-op: factory injects host default chat model via a
  new host_llm slot (create_chat_model(name=None)); DeerMem prefers host_llm
  over build_llm(model). Zero-config extraction works out of the box again
- MemoryConfigResponse: fix stale docstring (backend-agnostic shape; DeerMem
  knobs live under backend_config, not top-level - restoring flat would
  re-couple the API to DeerMem). Frontend audited: does not read /memory/config
- _host_default_tracing_callback: restore langfuse assistant_id/environment
- search: push category onto the ABC signature; DeerMem filters BEFORE the
  top_k slice (was filtered client-side after slicing -> starved results)
- _do_update_memory_sync: split into wrapper+impl; bind trace_id into the
  request-trace ContextVar on the Timer/executor worker via a new
  trace_context_manager host hook (None trace_id left unbound - no fabrication)
- client.py fact-CRUD now passes user_id (was writing to the global bucket
  while get_memory reads per-user)
- _resolve_manager_class: fail-fast (raise ValueError) on an unresolved
  explicit manager_class instead of silently falling back to DeerMem (memory is
  persistent state - a wrong store is a silent data-integrity footgun)

Upstream memory fixes dropped by the host->vendored rename conflict, re-ported
to backends/deermem/deermem/core/ (+ deer_mem.py):
- #4073 queue busy-timer-spin -> _reprocess_pending flag (core/queue.py)
- #4074 null source.confidence in staleness -> _coerce_source_confidence
  (core/updater.py: _build_staleness_section + _apply_updates stale sort)
- #4075 factsToRemove is optional (drop from _REQUIRED_MEMORY_UPDATE_TOP_LEVEL_KEYS)
- #4076 null confidence in search ranking -> _coerce_source_confidence
  (deer_mem.py DeerMem.search)

host_llm + trace_context_manager are host-injected via backend_config (factory
in manager.py), keeping backends/deermem/ at exactly one `from deerflow` line
(the ABC contract) - portability test preserved.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: resolve lint errors (F541 f-string without placeholders, E501 line too long)

* fix(memory): restore hide_from_ui clarification preservation, expose mode

Two memory-system fixes (F541/E501 lint was already fixed on this branch):

- filter_messages_for_memory: restore default preservation of well-formed
  human_input_response clarification answers (v2 regression). The
  self-containment refactor made the bare function skip ALL hide_from_ui when
  no hook was passed, but upstream preserves well-formed clarification
  responses by default (test_hide_from_ui_human_input_response_is_preserved).
  Inline a host-agnostic _is_human_clarification_response mirror of
  read_human_input_response as the default keep-decision; the host-injected
  should_keep_hidden_message hook still overrides (production path unchanged).
  Portable package stays zero `from deerflow`.

- /memory/config: expose `mode` (middleware|tool) in MemoryConfigResponse +
  the config/status endpoints + client.get_memory_config. mode is a host-
  shared, behavior-determining field missing from the response projection.
  Sync tests (mock .mode; e2e assert mode present).

- Align manager_class field docstring with fail-fast behavior.

Tests: filter/self-contained/portability (35) + memory-config (4) pass;
ruff clean.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): resolve ruff format failures in memory module + tests

`make lint` runs `ruff format --check` in addition to `ruff check`; 8 memory
files had pending format changes -- 7 pre-existing (deer_mem, updater, tools,
test_memory_queue/router/search/tools) + message_processing from the
hide_from_ui fix. Apply `ruff format`: whitespace/wrapping only, no logic
change. 109 memory tests pass; ruff check + format --check both clean.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): address PR review - legacy field migration, fact_id contract, path/docs

Address willem-bd's review on PR head bc8bf0d4 (risk:high, persistent state):

- config: auto-migrate pre-abstraction top-level memory.* DeerMem fields
  (storage_path, max_facts, debounce_seconds, model_name, token_counting,
  staleness_*, consolidation_*) into backend_config on load + warn, so an
  upgrade does NOT silently revert customized settings (was: silent
  extra='ignore' drop). model_name -> backend_config.model.model. Unknown
  top-level keys warned.
- factory: resolve a relative backend_config.storage_path against runtime_home()
  (base_dir-relative, CWD-independent) to preserve pre-abstraction semantics;
  paths.py stays portable (no runtime_home import).
- tools: memory_add uses the fact_id returned directly by create_fact instead of
  re-deriving it via content-key matching (coupled the tool to the backend's
  content normalization; could misreport a storage cap). create_fact now returns
  (memory_data, fact_id); gateway/client/tool updated. Fix terse
  {"error":"content"} -> {"error":"empty content"}.
- app.py: update stale token_counting=="char" warm-up comment to point at
  manager.warm (DeerMem.warm re-checks char and returns early).
- router: comment explaining reload_memory silent fallback vs fact 501 asymmetry
  (read-only degrade vs write fail-loud).
- CHANGELOG: document breaking changes (/memory/config + client.get_memory_config
  shape flat->backend_config; custom storage_class path moved + __init__ must
  accept config) and the legacy-field auto-migration.
- tests: add regression test pinning the per-user memory path
  ({storage_path}/users/{safe_user_id}/memory.json == host make_safe_user_id)
  across the abstraction; update create_fact mocks for (memory_data, fact_id).

Tests: 273 passed (memory suite); ruff check + format clean.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): address PR review - storage_path, max_facts, tracing, parsing

Six review findings (willem-bd), each verified against upstream:

- storage_path semantics (file -> root dir): migration drops file-style
  (.json) legacy values with a warning; factory raises if storage_path
  resolves to an existing file (avoid silent NotADirectoryError write
  failure). CHANGELOG + config.example.yaml comment updated.
- create_memory_fact enforces max_facts again (via _trim_facts_to_max) and
  returns (memory, None) when the cap evicts the new fact; memory_add tool
  reports "not stored", client raises ValueError, POST /memory/facts -> 409.
- max_facts trim uses _coerce_source_confidence (was raw f.get("confidence",
  0) -> TypeError on non-float imported/legacy confidence, swallowed as
  silent update failure).
- memory-tracing assistant_id restored to "memory_agent" (was "lead-agent"
  copy-paste; matches upstream + DeerMem run_name).
- _is_human_clarification_response cross-checked against
  read_human_input_response (drift guard test).
- empty-string legacy values skipped silently in migration (narrow fix, not
  broad "if not value" which would skip explicit bool False).

8 new regression tests. make lint + 406 memory tests pass.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): address internal review - storage fail-fast, build_llm degrade, config warn, noop template

Addresses 4 findings from the PR #4122 internal supplemental review
(parallel to willem-bd's review, no overlap):

- create_storage fail-fast: a misspelled/unimportable storage_class now
  raises ValueError instead of silently falling back to FileMemoryStorage.
  Memory is persistent state, so a wrong store is a data-integrity footgun;
  mirrors the existing manager_class resolution policy. (storage.py)

- noop template create_fact signature: the commented template used
  keyword-only `content` and returned a bare dict, while DeerMem's actual
  create_fact takes positional `content` and returns tuple[dict, str|None]
  (the memory_add tool passes content positionally; gateway/client/tools all
  tuple-unpack). A backend copied from the template would 500 on fact-CRUD.
  Template fixed; delete_fact/update_fact templates left (callers compatible).
  (noop_manager.py)

- build_llm graceful degrade: wrap init_chat_model in try/except, degrade to
  None + WARNING on failure (mirroring _host_default_llm) so a misconfigured
  explicit model does not crash app startup -- non-LLM memory ops still work
  and an update raises at runtime with the error logged. (llm.py)

- from_backend_config unknown-key warning: log a WARNING for unknown
  backend_config keys (mirrors the host layer's load_memory_config_from_dict)
  so a typo like `storage_pat` does not silently fall back to the default and
  write memory to an unintended location. (config.py)

Tests: rewrote 3 create_storage fallback tests to expect ValueError; added 4
tests (build_llm zero-config/degrade, from_backend_config warn/silent).
make lint green; full memory suite passes.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: lllyfff <2281215061@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: lllyfff <122260771+lllyfff@users.noreply.github.com>
 2026-07-15 11:21:04 +08:00
Huixin615
24648194ae
fix(frontend): show branch action only for completed turns (#4147) 
* fix(frontend): gate branch action by completed turn

* test(frontend): clarify branch turn boundaries
 2026-07-14 11:11:43 +08:00
AnoobFeng
446fa03801
fix(context): resolve context compress bug (#4065) 
* fix(runtime): persist original human input outside model sanitization

* refactor(history): load thread messages by global event sequence

* fix(frontend): make summarization rescue a transient history bridge

* fix(frontend): old message not append tail

1. add identity anchor
2. add bridgeOrder

* fix(frontend): lint error fix

* fix: address review feedback and harden pagination coverage

- defer transient history ref writes until after render commit
- cover large middleware-only history scans
- verify infinite-query refetch recalculates page cursors
- document AI event types and anchor-weaving differences

* fix: harden message pagination and enrichment

- append unmatched live tails after canonical history
- warn and stop when pagination has_more lacks a cursor
- deep-copy restored UI messages to isolate model-facing content
- log invalid event sequence and non-advancing cursor errors
- pass user_id explicitly through event-store history queries
- cover middleware-only AI runs across memory, JSONL, and DB stores

* fix: address pagination review feedback

* fix(frontend): checkpoint has unknow redener content, optimize the anchor policy

* fix(frontend): unit test issue missed previously, remove the TanStack cache trimming

* fix(gateway): harden message history queries and provenance

- reject externally forged original_user_content metadata
- validate provenance metadata in upload and sanitization middleware
- make run lookups fail closed by default
- batch feedback queries by run ID
- align memory message filtering with persistent stores
 2026-07-14 10:43:13 +08:00
DanielWalnut
4e209827f3
feat(agent): Add subagent total delegation cap (#4115) 
* fix subagent total delegation cap

* fix embedded subagent run cap context

* fix subagent cap config consistency

* fix resumed subagent run cap boundary

* fix legacy resume subagent boundary

* address subagent cap review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-13 19:26:07 +08:00
Xinmin Zeng
95a35f3725
docs(deployment-guide): fix stale userdata PVC subPath (#4139) 2026-07-13 19:08:13 +08:00
nonoge
63e2da1401
fix landing hero background positioning (#4092) 2026-07-13 18:38:49 +08:00
Zheng Feng
a9145e426c
feat(frontend): show real project version on About page (#4126) 
* feat(frontend): show real project version on About page

The About page heading was hardcoded "About DeerFlow 2.0" while the real
version is 2.1.0, kept in lockstep across backend/pyproject.toml,
frontend/package.json, and Chart.yaml by scripts/verify_versions.sh.
With the nightly workflow shipping unreleased main daily, a hardcoded
version drifts further from reality every day.

Wire the heading to the real version, nightly-aware:
- src/version.ts reads NEXT_PUBLIC_APP_VERSION (nightly override) with a
  package.json fallback for release/local builds.
- about-content.ts interpolates APP_VERSION into the heading; the
  "DeerFlow 1.0 and 2.0" milestone copy in acknowledgments is left as-is.
- Dockerfile adds ARG APP_VERSION / ENV NEXT_PUBLIC_APP_VERSION in the
  builder stage so nightly CI can stamp the version.
- nightly.yaml computes the nightly string once in prepare
  (<base>-nightly.<date>-<sha>, mirroring the chart scheme) and feeds it
  to both the frontend image build-arg and the chart publish (refactored
  to consume the shared output, so the displayed version and chart
  version can't drift). Release builds (container.yaml) are untouched
  and fall back to package.json.

Tests cover the version branches and the heading interpolation;
RELEASING.md notes the About page as a derived version consumer.

* test(frontend): assert positive About-page version value

Address review feedback on #4126: the unset-env assertion was negative-only
(checked the stale "2.0" literal was gone), so it would still pass if
APP_VERSION interpolated to "" or undefined. Mirror version.test.ts by
asserting the heading carries the real resolved APP_VERSION.

* ci(nightly): fail loudly on empty base version

Address review feedback on #4126: the prepare step's BASE (parsed from
Chart.yaml) had no empty-guard. A malformed/missing version would yield
`-nightly.<date>-<sha>` (invalid semver) that now flows into both the
chart publish and the frontend About-page version. Add `set -euo pipefail`
plus a `test -n "$BASE"` guard mirroring publish-chart. `pipefail` matters
- without it the pipeline's exit code is awk's, so `set -e` alone wouldn't
catch a grep miss.
 2026-07-12 22:54:03 +08:00
DanielWalnut
65d474202f
fix(front): Show assistant text during tool steps (#4114) 
* fix: show assistant text during tool steps

* test: cover collapsed tool step assistant text

* test: cover tool-call text review cases
 2026-07-12 19:50:15 +08:00
Huixin615
7bdf9412cc
fix(front): stabilize artifact paths during streaming (#4094) 2026-07-12 09:00:12 +08:00
chaoxi007
97935b081b
fix(front): resolve relative artifact image paths (#4038) 
* fix: resolve relative artifact image paths

* fix: address artifact image review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-11 23:23:35 +08:00
Huixin615
2839a3632e
fix(frontend): avoid highlighting code while streaming (#4068) 
* fix(frontend): avoid highlighting code while streaming

* fix(frontend): harden streaming code rendering
 2026-07-11 22:56:29 +08:00
Nan Gao
aafd5077b2
feat(subagents): show effective model and token usage on task cards (#4049) 
* feat(subagents): show runtime metadata on task cards

* fix(subagents): stop task-card render loop and dedupe model fetches

Address code review on the runtime-metadata cards:

- P1 render loop: the terminal ToolMessage is re-parsed on every
  MessageList render and always carries modelName/usage, so the
  presence-based setTasks condition fired a fresh state object each
  render -> "Maximum update depth exceeded". computeNextSubtask now
  returns a value-compared `changed` flag and a pure subtaskNotification()
  routes terminal transitions through the deferred after-render path
  while skipping no-op re-parses.

- Per-card useModels refetch: add staleTime: Infinity to the ["models"]
  query so every subtask card shares one /api/models fetch instead of
  refetching on each mount.

* make format

* refactor(subagents): dedupe token-usage validators + tidy event narrowing

Address PR review follow-ups:

- DRY: extract one shared token-usage validator per side. Backend
  status_contract.normalize_token_usage() now backs both the terminal
  ToolMessage metadata and the subagent.step/.end run events
  (step_events.py), and frontend messages/usage.normalizeTokenUsage()
  backs both the live task_running event (lifecycle.ts) and the terminal
  ToolMessage metadata (subtask-result.ts). Prevents the input/output/
  total_tokens validation from drifting across the four former copies.

- Nit: onCustomEvent narrows event.type once instead of re-checking the
  object shape per branch; the redundant task_started early-return
  (already validated by taskEventToSubtaskUpdate) is dropped.
 2026-07-11 15:41:57 +08:00
Vanzeren
c2002d9fac
feat(memory): add memory tool sets (#4023) 
* feat: add memory-as-tool mode alongside existing middleware mode

- Add memory.mode config field (middleware|tool, default middleware)
- Add search_memory_facts() for case-insensitive fact lookup
- Add 4 memory tools: memory_search, memory_add, memory_update, memory_delete
- Wire mode gating in factory.py and lead_agent/agent.py
- 256 memory tests passing, zero regressions

* fix: harden tool-mode memory scoping and docs

* fix: address memory tool mode review feedback

* fix(memory): address tool mode review feedback

* fix: update config_version to 22 in values.yaml
 2026-07-11 15:28:16 +08:00
Ryker_Feng
be637163a6
feat(frontend): add voice dictation input (#4036) 
* feat(frontend): add voice dictation input

* fix(frontend): address voice input review feedback

* ci: rerun frontend checks
 2026-07-10 23:13:06 +08:00
hataa
c9fb9768d4
fix(subagents): unify guardrail caps on additive stop_reason + add token_budget (#3875 Phase 2) (#3980) 
Phase 2 of #3875. Two guardrail axes can end a subagent run early — the turn
budget (GraphRecursionError) and the token budget (TokenBudgetMiddleware) —
and both now surface *why* through one additive `subagent_stop_reason` field
instead of a status enum.

This completes and course-corrects Phase 1 (#3949), which shipped the
turn-budget cap as a `max_turns_reached` status enum. The agreed Phase 2
design replaces that enum with an optional `stop_reason` field
(token_capped | turn_capped | loop_capped): a new enum value would break v1
consumers, while an additive field is ignored by older frontends and ledger
readers. `max_turns_reached` and SubagentStatus.MAX_TURNS_REACHED are removed.

- subagents.token_budget config (default enabled, 2,000,000 tokens, warn 0.7)
  with per-agent override; TokenBudgetMiddleware is now attached in
  build_subagent_runtime_middlewares so the cost-ceiling backstop engages for
  every subagent. The hard-stop does not raise — it strips tool_calls and
  lets the run finish with a final answer, recording the cap on a per-run
  consume_stop_reason() accessor.
- executor.py: on normal completion it reads consume_stop_reason() and stamps
  completed + token_capped when the budget fired; on GraphRecursionError it
  recovers the last AIMessage partial (completed + turn_capped) or, if nothing
  usable survived, failed + turn_capped. SubagentResult gains stop_reason.
- status_contract.py / contracts/subagent_status_contract.json (v2) /
  frontend subtask-result.ts: additive subagent_stop_reason field, pinned by
  test_status_values_match_contract / test_stop_reason_values_match_contract.
- task_tool.py + delegation_ledger.py: drop the max_turns_reached paths; the
  ledger captures stop_reason and renders model-facing "capped" guidance so
  the lead reuses a capped completion knowingly.

The 2,000,000-token default is deliberately loose (tighten to taste) — it
would have roughly halved the reported 4.4M burn while leaving legitimate
deep-research runs (max_turns=150) room. Subagent summarization is a follow-up.
 2026-07-08 22:26:06 +08:00
Ryker_Feng
c640b52a7d
feat(frontend): render slash-skill activations as inline chips (#3981) 
* feat(frontend): render slash-skill activations as inline chips

Show an explicit `/skill` activation as a compact inline chip in both the
composer and the chat transcript instead of raw slash text.

- Composer: selecting a skill suggestion stores it as a removable chip
  aligned inline with the textarea; the leading `/skill ` prefix is
  reattached only at submit time, so the backend activation protocol is
  unchanged. Backspace on an empty input or the chip's close button clears
  it; history navigation is disabled while a chip is active.
- Transcript: human messages that begin with `/skill` render the skill as a
  read-only chip followed by the task text.
- Add a shared `core/skills/slash.ts` (`parseSlashSkillReference` +
  `resolveSlashSkillDisplay`) mirroring the backend `slash.py` gate, so the
  transcript only shows a chip when the skill actually exists and is enabled.
  This removes a duplicated regex/reserved-name list and keeps display
  semantics consistent with backend activation.

Add unit tests for the shared slash parser and extend the chat e2e to assert
the composer still submits `/skill <task>` after showing a chip.

* chore(frontend): format chat e2e test

* refactor(skills): address slash-skill chip review feedback

Follow-up to the inline slash-skill chip PR, resolving three second-order
review findings:

- Drive the reserved-command set and skill-name grammar from a shared
  contracts/slash_skill_contract.json instead of a hand-copied
  "keep in sync" pair. slash.ts and slash.py now reference the fixture, and
  contract tests on both sides fail CI if either drifts.
- Extract a shared SlashSkillChip so the composer and transcript chips stay
  in lockstep, and normalize the off-scale /8 and /12 opacity steps to the
  standard /10 and /20 tokens.
- Split HumanMessageText into a pure parse gate plus a slash-only subtree
  that owns the useSkills() lookup, so a skill-enabled toggle no longer
  re-renders every plain-text human turn.

Verified: frontend eslint + tsc clean, pnpm test 572 pass (incl. new
slash-contract test); backend slash contract + slash-skills tests 31 pass.

* style(tests): sort slash skill contract imports

* fix(composer): inline the slash-skill text so the chip aligns with input

Address the "composer body layout change" review on #3981 by rendering the
active skill as an inline chip in the same text flow as the prompt, rather
than a separate flex row that drifted the box model across states.

- Render the chip + prompt inside one leading-6 wrapper and edit the prompt
  through a `contentEditable` span, so the chip sits inline with the first
  line and long/multi-line input wraps naturally back to the container edge.
- Align the chip with `align-top`: its h-6 (24px) height matches the text
  line height, so chip and first-line centers coincide exactly (measured
  delta 0), fixing the chip being raised above the baseline.
- Restore the placeholder in chip mode via a `data-empty` CSS `::before`,
  which also gives the empty editable span width so it is no longer treated
  as hidden.
- Widen the IME helper to `HTMLElement` and route the span's keydown/paste
  through the shared skill-suggestion, prompt-history, backspace-to-clear,
  and IME-composition handlers so contentEditable behaves like the textarea.
- Extend chat.spec.ts to drive the inline skill editor instead of the
  textarea after a chip is shown.

* style(frontend): fix composer class order formatting

* fix(composer): break long unbroken input inside the slash-skill row

The inline slash-skill editor wrapped with `break-words`
(overflow-wrap: break-word), which only moves an over-long token to the
next line before breaking it. A long unbroken string therefore started
on the line below the chip, and when the string contained a break
opportunity such as a hyphen the browser wrapped there and pushed the
remaining run to the next line, leaving a wide gap on the right.

Switch to `break-all` (word-break: break-all) so the text fills each
line from the chip and packs tightly regardless of hyphens or CJK.
 2026-07-08 21:58:33 +08:00
Ryker_Feng
01dc067997
feat: add composer input polishing (#3986) 
* feat: add composer input polishing

* Revert "Merge branch 'main' into feat/input-polish"

This reverts commit 5b6ceccf0db3092bc62fde3b05e7816829601756, reversing
changes made to 45fbc57fef5fa5fd878cf0176c37f3e3bc7ebef6.

* Merge main into feat/input-polish

* style(frontend): format input helper polish guard

* fix(input-polish): address composer polish review findings

Frontend
- Add a cancel affordance to the in-flight polish status pill that calls
  abortInputPolishRequest(), so a slow/hung provider no longer hard-locks the
  composer for up to stream_chunk_timeout with a page reload (and draft loss)
  as the only escape.
- Reset promptHistoryIndexRef/promptHistoryDraftRef when a rewrite is applied
  (and on undo), so a stale history-browse index can no longer let the next
  ArrowDown silently overwrite the polished draft.
- Disable polishing while an open human-input card is present, matching the
  frontend/AGENTS.md rule that composer entry points defer to the card so
  card-reply metadata is preserved.
- canPolishInput now reuses parseGoalCommand/parseCompactCommand instead of a
  third hardcoded reserved-command regex, and drops the phantom /help entry
  (no /help parser exists in the composer), so future builtins only need to be
  taught to the existing parsers.

Backend
- Extract the non-graph one-shot LLM path (build model + inject Langfuse
  metadata + system/user invoke + text extract) into
  deerflow.utils.oneshot_llm.run_oneshot_llm, shared by the input-polish and
  suggestions routers so tracing-metadata and invocation shape cannot drift
  between the two copies.
- strip_think_blocks gains truncate_unclosed (default True, preserving the
  suggestions/goal JSON-prep behavior); input polish passes False so a draft
  that legitimately contains a literal <think> substring is no longer
  truncated into a partial rewrite or a spurious 503.
- Validate the empty-check and max_chars boundary against the same stripped
  view of the draft that is sent to the model, so the user-facing length
  boundary and the model input can no longer disagree.

Tests / docs
- Backend: literal-<think> preservation, whitespace-only rejection, and
  normalized-length/model-input agreement cases; suggestions tests repoint the
  create_chat_model patch to the shared helper module.
- Frontend: helper unit tests updated for the /help/reserved-command change; a
  new Playwright case covers cancelling an in-flight polish request.
- backend/AGENTS.md documents the shared one-shot helper and the polish
  normalization/think-tag behavior.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-08 17:10:27 +08:00
Ryker_Feng
26d7a5970d
feat: add manual context compaction (#3969) 
* feat: add manual context compaction

* fix: harden manual context compaction
 2026-07-07 19:55:33 +08:00
Huixin615
4915b5e4cf
fix(frontend): enable regenerate in custom agent chats (#3967) 2026-07-07 07:10:48 +08:00
AnoobFeng
47b0f604f4
feat(frontend):enhance the ask_clarification interaction with visualized card (#3956) 
* feat(frontend): add structured human input cards for ask_clarification

Implement a reusable Human Input Card flow for ask_clarification while keeping
the existing text fallback for older clients and IM channels.

Backend:
- Add structured ToolMessage.artifact.human_input payloads for clarification requests.
- Preserve ToolMessage.content as the readable Markdown/text fallback.
- Normalize clarification options from native lists, JSON strings, plain strings,
  mixed scalar values, None, and missing options.
- Derive input_mode as choice_with_other when options exist, otherwise free_text.
- Keep disable_clarification non-interactive behavior as a plain ToolMessage with
  no human_input artifact.
- Cover artifact persistence and Gateway message metadata preservation in tests.

Frontend:
- Add human input protocol types, runtime guards, extractors, response builders,
  and thread-state helpers.
- Add reusable HumanInputCard with option buttons, free-text input, pending,
  read-only, disabled, and answered states.
- Render structured clarification cards from artifact.human_input, with Markdown
  fallback for malformed or legacy tool messages.
- Preserve line breaks in structured question/context/option text.
- Hide submitted clarification bridge messages from the chat UI via
  additional_kwargs.hide_from_ui.
- Send structured human_input_response metadata through the fourth sendMessage
  options argument, preserving run context in the third argument.
- Wire submissions for normal chats, custom agent chats, agent bootstrap chats,
  and sidecar chats.
- Derive answered state from raw thread.messages so hidden replies still update
  the original card.
- Clear pending state when the hidden reply arrives, dispatch is dropped, or a
  later async stream failure appears on thread.error.

* perf(frontend): optimize HumanInputCard UI interactions

- Support Enter key to submit text input (Shift+Enter for newline)
- Render question and context fields as Markdown instead of plain text
- Replace deprecated FormEventHandler type with structural typing

* test(frontend): add unit test cover optimize HumanInputCard UI interactions

* feat(frontend): disabled chatbox when has new human-input-card

* fix(style): lint error fix

* fix: sanitize hidden human input replies

- Preserve IME composition safety for human input card Enter submits
- Treat hidden human input responses as genuine user messages for sanitization
- Keep hidden card replies in memory filtering while excluding malformed/internal hidden messages
- Add regression coverage for card IME handling and hidden reply sanitization

* fix: tighten human input response validation

- Reject empty hidden human input response values
- Remove invalid list ARIA role from human input card options
- Add backend coverage for empty response payloads

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-06 22:34:41 +08:00
Ryker_Feng
34f87f6c92
fix(sidecar): panel button deletes the side chat instead of hiding it (#3961) 
* fix(sidecar): make panel button delete the side chat instead of hiding it

The side chat panel's top-right button called `sidecar.close()`, which only
hid the panel — duplicating the header `SidecarTrigger` toggle. Repurpose the
button so the panel owns deletion and the header toggle owns hide/show.

- When a conversation exists, the button shows a trash icon and opens a
  confirmation dialog before deleting via `useDeleteThread` (backend cascade).
- In the draft state (references only, no thread yet) the header trigger is not
  rendered, so the button falls back to a plain close (X) that discards the
  draft without a confirm — there is nothing persisted to delete.

Also fix a latent double-delete bug surfaced by the new dialog:
`useDeleteThread` deletes through the LangGraph route (which drops the
thread_meta row) and then calls `deleteLocalThreadData`, which hits the same
gateway handler whose `require_existing=True` guard now 404s. Treat that 404 as
idempotent success. The recent-chat delete used fire-and-forget `mutate`, so it
swallowed this error; the sidecar's `mutateAsync` surfaced it as a toast.

The e2e mock now mirrors the gateway ownership guard (second DELETE 404s once
the thread is gone) so the regression stays covered. Adds tests for delete,
draft-state close, and header-owned hide/show.

* fix(sidecar): lock delete dialog while deletion is in flight

The delete confirmation dialog disabled Cancel during an in-flight
delete but still allowed dismissal via the overlay, Esc, and the
built-in close (X). Those paths could imply the delete was cancelled
when it was actually still running. Ignore dismissals and drop the
close button while `isDeleting` so only the (disabled) Cancel path
controls the dialog.
 2026-07-06 22:14:44 +08:00
Ryker_Feng
2ebe5cf048
fix(composer): stop bottom mask strip from clipping the focus ring (#3962) 
On conversation pages the composer renders an opaque `bg-background` strip
just below itself to mask scrolled content peeking past the rounded corners.
The strip was a child of `PromptInput`, the element that draws the focus ring.
A parent's box-shadow always paints beneath its own descendants, so the strip
covered the bottom ~3px of the ring — the blue focus outline looked cut off
along the bottom edge whenever the composer sat flush at the viewport bottom.

Move the strip out of `PromptInput` to be a sibling with a lower stacking order
and give the composer `relative z-10`, so the ring composites above the strip.
The strip still masks the same region; only the paint order changes. Welcome
mode is unaffected (it never renders the strip).
 2026-07-06 21:11:52 +08:00
MeloMei
f122594419
refactor(frontend): extract placeholder detection utility with unit tests (follow-up to #3764) (#3783) 
* refactor(frontend): extract placeholder detection utility with unit tests

* fix(frontend): pass prompt text directly to onSelectPlaceholder to avoid stale DOM read

The onSelectPlaceholder callback was reading textarea.value immediately
after textInput.setInput(prompt), but React state updates are async so
the DOM had not yet reflected the new value. This caused the placeholder
auto-selection to silently fail when the textarea was previously empty.

Fix: accept the new text as a parameter instead of reading from the DOM.

* fix(frontend): resolve duplicate findSuggestionTemplatePlaceholder identifier

After rebasing onto main, the function existed both inline in
input-box-helpers.ts (from #3764) and as an import from our new
placeholders module, causing TS2300 duplicate identifier errors.

- Remove duplicate import in input-box.tsx
- Replace inline function in input-box-helpers with re-export from
  @/core/suggestions/placeholders
- Export SUGGESTION_TEMPLATE_PLACEHOLDER_PATTERN from placeholders module

* refactor(frontend): remove dead hasUnreplacedPlaceholder export

No production call site uses this boolean wrapper — both existing
checks need the {start,end} range from findSuggestionTemplatePlaceholder.
Drop the function and its two unit tests.
 2026-07-06 15:44:24 +08:00
Ryker_Feng
cb83edb0da
fix(sidecar): correct text-selection toolbar actions inside the side chat (#3959) 
* fix(sidecar): correct text-selection toolbar actions inside the side chat

The sidecar panel reuses MessageList, but it did not distinguish the side
chat surface from the main conversation, so selecting text inside the side
chat behaved as if it were the main list:

- The "Ask in side chat" action was shown even though the user is already
  in the side chat, which is a no-op interaction.
- "Add to conversation" routed the snippet to the main composer's quotes
  (conversationQuotes) instead of the side chat's own composer, so the
  reference landed in the wrong input box.

Add a `sidecarSurface` prop to MessageList. On the sidecar surface, hide
"Ask in side chat" and route "Add to conversation" to `sidecar.openContext`
so the snippet attaches to the side chat's own composer (activeReferences).
Main-list behavior is unchanged.

* docs(sidecar): drop AGENTS.md note for the toolbar surface change

The sidecarSurface behavior is self-evident from the code; no dedicated
Interaction Ownership entry is needed.
 2026-07-06 14:37:31 +08:00
hataa
0664ea2243
fix(subagents): surface turn-budget cap as MAX_TURNS_REACHED with partial result (#3875 Phase 2) (#3949) 
* fix(subagents): surface turn-budget cap as MAX_TURNS_REACHED with partial result (#3875)

Phase 2 of #3875. When a subagent exhausts its turn budget
(recursion_limit == max_turns), LangGraph raises
GraphRecursionError from agent.astream. The generic
except Exception in _aexecute misclassified it as FAILED and
discarded the partial work already streamed into final_state, so the
lead could not tell 'broken subagent' from 'out of budget' and got an
empty failure.

Catch GraphRecursionError specifically (before the generic handler)
and set a distinct SubagentStatus.MAX_TURNS_REACHED terminal status,
recovering the partial result from the last streamed chunk via a shared
_extract_final_result helper (refactored out of the normal-completion
path so both paths render content identically).

Extend the cross-language status contract so the new value travels on
additional_kwargs.subagent_status: a capped run is result-bearing, so
make_subagent_additional_kwargs / read_subagent_result_metadata
carry subagent_result_brief + subagent_result_sha256 (the
recovered work, like completed) AND the cap notice on subagent_error
-- the one status that carries both. task_tool.py returns it via the
shared _task_result_command; the delegation ledger prefers the
partial result_brief and renders model-facing guidance (reuse / retry
tighter / raise max_turns). Frontend collapses max_turns_reached to the
failed pill with the cap notice on error.

No agent-loop, runner, or persistence behavior touched; default
max_turns is unchanged.

* refactor(subagents): consolidate content-stringify onto shared helper

Address review feedback on #3949 (willem-bd, copilot-pull-request-reviewer):

- executor.py: drop the private `_stringify_message_content` — a third
  near-duplicate of `utils/messages.py::message_content_to_text`.
  `_extract_final_result` now delegates to that canonical helper; the
  "No response generated" sentinel is pushed down to the consumer (the
  shared helper returns "" for no-text, matching every other call site).
- task_tool.py: align the live `task_failed` event's error string with
  the canonical "Reached max_turns=N" used by the logger, the structured
  `error=`, and the executor (was "Reached max turns (N)").

Behavior for real AIMessage content is unchanged; only atypical edge inputs
(consecutive bare-string list items; empty content) now match the canonical
helper that every other call site already uses.

`extract_response_text` is intentionally left as-is: it filters by OpenAI
content-block `type`, a different shape with many callers and its own tests.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-06 07:55:32 +08:00
Ryker_Feng
186c6ea463
feat: add branching support for assistant turns (#3950) 
* feat: add assistant turn branching

* fix(threads): skip workspace clone when branching from historical turn

Workspace files are not checkpointed, so cloning them onto a branch
rooted at an older assistant turn leaked files created in a later
timeline. Restrict the best-effort workspace copy to branches taken from
the latest turn; historical-turn branches now report
workspace_clone_mode="skipped_historical_turn" and keep only the
restored message history.

* style(frontend): fix prettier formatting in e2e mock-api

Collapse the branch-title normalization chain onto a single line to
satisfy the frontend lint (prettier --check) CI gate.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-06 07:40:41 +08:00
Ryker_Feng
7a985f441c
Fix(frontend): Fix mobile workspace and accessibility blockers (#3740) 
* fix(frontend): address mobile workspace polish blockers

* fix(frontend): prevent mobile landing overflow

* fix(frontend): keep landing sections within mobile viewport

Section titles used a fixed text-5xl with no word breaking, and the
<section> flex items had default min-width:auto, so wider Linux font
metrics in CI pushed content past the viewport (scrollWidth 345 > 320).
Make titles/subtitles responsive with break-words, constrain section
width with min-w-0, and add an overflow-x-clip guard on the page root.

* fix(frontend): address review feedback on mobile landing PR

- add hamburger Sheet nav below sm: so mobile users keep docs/blog access
- switch useIsMobile to useSyncExternalStore to avoid hydration swap flash
- memoize artifactContent so it isn't rebuilt on every streamed token
- use slot-based key in HeroWordRotate; drop flex-wrap so SuperAgent never orphans at 320px
- delete unused word-rotate.tsx dead code
- move section gutter to <main> for a uniform mobile padding contract
- harden e2e: per-viewport overflow tests, locale-stable artifact selector, focus-ring asserts light vs dark differ

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-06 07:15:35 +08:00
Ryker_Feng
00161811bd
feat: add workspace change review for agent runs (#3945) 
* feat: add workspace change review

* chore: format workspace change files

* fix: optimize workspace change summaries

* style: refine workspace change badge scale

* fix: restore workspace change user context import

* fix(frontend): gate workspace change badge to assistant messages

Only pass run_id to assistant MessageListItems so the workspace-change
badge can never render under a user's prompt, which carries the same
run_id. Assert single badge render in the E2E flow.

* fix(workspace-changes): address review feedback on diff parsing and badge

- Restrict unified-diff header detection to "+++ "/"--- " (trailing space)
  in both backend _count_diff_lines and frontend getWorkspaceChangeLineClass
  so content lines beginning with +++/--- are counted/styled correctly
- Gate WorkspaceChangeBadge to ai messages so tool messages folded into an
  assistant group don't render a duplicate badge
- Add regression tests for both diff-classification fixes
- Apply prettier formatting to workspace-changes files flagged by CI

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-05 15:38:01 +08:00
Nan Gao
c05c1899b5
fix(frontend): Fix uploaded file metadata in message copy (#3944) 
* fix uploaded file metadata copy

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-07-05 10:04:14 +08:00
Ryker_Feng
6a4e5a3bb2
feat(frontend): add side conversations for quoted follow-ups (#3934) 
* feat(frontend): add side conversations for quoted follow-ups

* style(frontend): apply prettier formatting to sidecar-chat files

* fix(frontend): surface sidecar cascade cleanup failures via console.warn

Previously deleteSidecarThreadsForParent silently swallowed both
lookup errors and per-thread deletion failures, so parent thread
deletions could succeed while orphaning sidecar threads with no
signal to the caller. Log a warning that includes the parent id
and the failed thread ids/reasons so the leak is discoverable in
telemetry, matching the existing console.warn/error pattern in
this file.

* fix(frontend): address all sidecar review feedback

Resolve every reviewer comment on PR #3934:

- input-box/hooks/sidecar-panel: clear quoted references only via an
  `onSent` callback that fires after the in-flight guard, so a dropped
  send no longer silently discards quotes (willem-bd #3550).
- message-list: flip the selection toolbar below the selection when it
  would clip above the viewport (willem-bd #3551).
- reference-metadata/thread/input-box: keep referenced ids, roles, and
  count arrays 1:1 parallel instead of deduping ids (willem-bd #3552).
- message-list: widen selection containment to the shared assistant-turn
  container and hint when a selection crosses messages (willem-bd #3553).
- sidecar/api: coalesce concurrent sidecar creates for one parent behind
  a single in-flight promise to prevent duplicates (willem-bd #3554).
- sidecar-trigger/context: force-restore on trigger click so a sidecar
  deleted elsewhere self-heals instead of opening a dead thread
  (willem-bd #3555).
- threads/hooks: surface sidecar cascade cleanup failures via
  console.warn for both lookup and per-thread deletes (Copilot).

Add unit + e2e coverage for parallel metadata, atomic create, and
trigger self-healing.
 2026-07-05 00:12:16 +08:00
Xinmin Zeng
4fc08b4f15
feat: add scheduled tasks MVP (#3898) 
* feat: add scheduled tasks MVP

* fix: harden scheduled task execution semantics

* feat(scheduled-tasks): preset-driven schedule form with timezone and live preview

Replace the raw cron input with a preset Select (hourly/daily/weekly/monthly/custom)
plus structured inputs (time picker, weekday toggles, day-of-month), datetime-local
for one-time tasks, a timezone selector defaulting to the browser timezone, and a
live human-readable preview. Reuses one ScheduledTaskScheduleInput for create and
edit; backend contract unchanged; zero new deps (pure Intl + DST-safe offset helpers).

* feat(scheduled-tasks): full-page i18n + recipe templates + E2E locale pin

Localize the rest of the scheduled-tasks page (filters, detail pane, actions,
edit form, run list, enum values) via t.scheduledTasks.* in en/zh. Add four
built-in recipe templates (GitHub Trending, news digest, issue triage, weekly
report) exposed as a chip row that pre-fills title + prompt + schedule. Pin
Playwright locale to en-US so E2E selectors stay stable against i18n. No backend
change, no new deps.

* fix(scheduled-tasks): idempotent 0003 migration, update head constants, future-date once test

Merge with main surfaced three CI failures:
- 0003_scheduled_tasks create_table collided with legacy test seeds that
  build from full metadata; guard with inspector.has_table so the revision
  no-ops when the table already exists (0004/0005 are already idempotent via
  _helpers.py).
- persistence bootstrap concurrency/regression tests pinned HEAD to main's
  0002_runs_token_usage; bump to the new head 0005_scheduled_task_thread_nullable.
- once-task router test used a fixed past run_at and tripped the
  must-be-in-the-future validation; use a future date.

* address review: ok-check, 502 for trigger failure, mock fields, migration filename, doc fences

- fetchThreadScheduledTasks now checks response.ok like the other fetchers.
- trigger endpoint returns 502 (not 409) when dispatch fails outright, so
  clients can distinguish a real conflict from a server-side failure.
- E2E mock normalizes scheduled-task objects with context_mode/last_thread_id
  and nullable thread_id, matching the backend contract the UI renders against.
- Rename 0002_scheduled_tasks.py -> 0003_scheduled_tasks.py to match its
  revision id (file was renamed in spirit already; filename now follows).
- CONFIGURATION.md: close the Tool Groups yaml fence and drop the stray fence
  after the Scheduler notes so the sections render correctly.

* fix(scheduled-tasks): harden lease, poller, config, and frontend UX after review

* fix(scheduled-tasks): harden run lifecycle, overlap skip, non_interactive gating, and DST conversion after review

- defer a once task's terminal status to the run-completion hook; the task
  stays running until the real outcome, and a startup sweep cancels once
  tasks orphaned by a crash (launch-time 'completed' could stick forever)
- record interrupted runs as a distinct 'interrupted' run status with a
  readable message; an interrupted once task ends 'cancelled', not 'failed'
- enforce overlap_policy=skip for fresh_thread_per_run via an active-run
  pre-check (same-thread ConflictError can never fire across fresh threads)
- protect terminal run statuses from the late launch-path 'running' write
- honor context.non_interactive only for internally-authenticated callers;
  arbitrary clients can no longer strip ask_clarification
- fix DST-stale timezone offset in zonedLocalToUtcIso by re-deriving the
  offset at the resolved instant (once tasks fired an hour late around
  spring-forward and the create->edit round-trip diverged)
- drop dead ScheduledTaskRunRepository.update_by_run_id; share one Gateway
  API error helper between channels and scheduled-tasks frontends

* fix(scheduled-tasks): close review round-3 gaps in guards, concurrency, and API ergonomics

- scrub internal-only context keys (non_interactive) from the assembled run
  config for non-internal callers: gating body.context alone left the same
  key smuggle-able through the free-form body.config copied verbatim by
  build_run_config
- guard update_after_launch with protect_terminal so the launch bookkeeping
  write cannot clobber a once task already finalized by a fast-failing run's
  completion hook (parent-row sibling of the run-row guard)
- reject a manual trigger while the task has an active run (409) instead of
  launching a duplicate concurrent run on fresh_thread_per_run
- re-arm a terminal once task to enabled when PATCH pushes run_at into the
  future; previously the endpoint returned 200 with a next_run_at that could
  never be claimed
- make max_concurrent_runs a real global cap: each poll claims only into the
  remaining budget of active (queued/running) scheduled runs
- paginate GET /scheduled-tasks/{id}/runs (limit<=200, offset) and push the
  thread filter of /threads/{id}/scheduled-tasks into SQL
- stamp context.user_id on scheduler-launched runs, matching IM channels, so
  user-scoped guardrail providers see the owning user

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-04 21:51:57 +08:00
Vanzeren
6060d95ee0
fix(wizard): update DeepSeek provider models to v4 (#3939) 
Update DeepSeek references from deprecated model names to the V4 lineup:
- deepseek-reasoner → deepseek-v4-pro
- deepseek-chat → deepseek-v4-flash

Keep docs and frontend mocks aligned with the wizard provider list.
 2026-07-04 21:44:22 +08:00
Zhipeng Zheng
53a80d3ad1
feat(skills): per-user custom skill isolation with sandbox mounting (#3889) 
* feat(skills): per-user skill isolation (#2905)

Implement user-scoped skill storage that isolates custom skills between
users while sharing public skills globally.

Key changes:
- Add UserScopedSkillStorage class for per-user custom skill directories
- Introduce get_or_new_user_skill_storage() factory with user_id context
- Auth middleware sets effective_user_id for request-scoped storage
- Agent/prompt/middleware now use user-scoped storage and prompt cache
- Sandbox mounts user-scoped skill directories for search/read tools
- Add validate_skill_file_path() to SkillStorage for path security
- Migration script supports --all-users bulk migration
- Frontend: add editable field to Skill type, error check in enableSkill
- All skill categories can be toggled (custom skills default to enabled)
- Update skill-creator SKILL.md with isolation-aware instructions

Tests:
- Add test_user_scoped_skill_storage.py (new)
- Update all existing skill tests for user-scoped storage
- Update sandbox, client, and router tests

* fix(skills): address second-round PR review feedback (#3889)

- P1-1: restrict legacy skill mount to users without custom skills
- P1-2: fail-closed for _is_disabled_skill_path (OSError → return True)
- P2-1: AND-merge global extensions_config skill disabled state
- P2-2: atomic write for _skill_states.json (mkstemp + replace)
- P2-3: normalize X-DeerFlow-Owner-User-Id in trusted boundary
- P2-4: LRU-bounded _enabled_skills_by_config_cache (OrderedDict, maxsize=256)
- P2-5: clear global prompt cache on PUBLIC skill toggle
- P2-6: invalidate skill caches on client.update_skill

* fix(tests): correct tool policy test after merge

* fix(skills): use DEFAULT_SKILLS_CONTAINER_PATH in UserScopedSkillStorage

The "/mnt/skills" literal in UserScopedSkillStorage.__init__ triggers
test_skill_container_path_defaults::test_mnt_skills_literal_is_owned_by_skill_constants_module
on CI. Migrate the default to the existing deerflow.constants constant,
matching the pattern already used by LocalSkillStorage, SkillStorage, and
the durable/tool_error middlewares.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-04 13:54:04 +08:00
Chetan Sharma
327695b8c5
fix(frontend) : prevent stream cancellation on concurrent message submit (#3878) 
* fix: prevent stream cancellation on concurrent message submit

* feat: add i18n support for streaming wait message
 2026-07-04 11:36:09 +08:00
AochenShen99
66b9e7f212
feat: emit structured runtime metadata (follow-up#3887) (#3906) 
* feat: emit structured runtime metadata

* fix: avoid subagent import cycle in replay gateway

* fix: preserve legacy subtask result parsing

* refactor: tighten runtime metadata contracts

* fix(middleware): keep recovery hint on task exception wrapper content

The structured-metadata stamp overwrote the wrapper text with the bare
task-failure message, dropping the model-facing 'Continue with available
context, or choose an alternative tool.' guidance that every other tool
exception keeps. Append the shared hint after the formatted message.

* fix(subagents): require lowercase hex for result_sha256 reader

Length-only validation accepted any 64-char string; a faulty serializer
or relaying wrapper could store a non-digest value in the delegation
ledger. Enforce the producer's hexdigest shape with a fullmatch.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-04 11:27:19 +08:00
lllyfff
a817a0ed87
Fix(frontend): stale run reconnect and cancel handling (#3908) 
* Fix stale run reconnect and cancel handling

* fix the frontend

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix prettier formatting and document error/timeout reconnect intent

Wrap the over-long assertion in api-client.test.ts so the CI prettier --check
job passes, and add a comment above TERMINAL_RUN_STATUSES noting that
error/timeout short-circuit intentionally drops the transient onError toast on
reload (the persisted error state still loads from the checkpoint via
useThreadHistory).

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-04 09:15:36 +08:00
Ryker_Feng
8a26b5c9a4
feat(frontend): add citation sources evidence panel (#3907) 
* feat(frontend): add citation sources evidence panel

Inline [citation:Title](URL) links render as badges, but with many
citations the reader has no consolidated, deduplicated list of what a
message or report actually drew on. Add a collapsible "sources" panel
that extracts citation links from AI messages and markdown artifacts,
dedupes by URL, counts occurrences, and offers per-source copy of a
reusable markdown reference.

- extractCitationSources: parse [citation:...](url) links, skip images
  and fenced code, dedupe by normalized URL, fall back to domain for
  generic labels
- CitationSourcesPanel: collapsible list with per-source cite counts,
  internal scroll for long lists, and copy-to-clipboard reference button
- Wire panel into AI message content and markdown artifact preview
- Add en-US/zh-CN citation strings and types
- Unit tests for extraction and panel rendering

* fix(frontend): preserve default link styling in message content override

MessageContent_ passes a custom `a` renderer to MarkdownContent, whose
default `a` (primary underline + external target/rel) is overridden
because MarkdownContent spreads props components last. Restore that
styling/external behavior in the fallback branch so normal links in
messages aren't regressed, while keeping citation: and /mnt/ handling.

* fix(frontend): harden citation source extraction and dedupe link renderer

Address review findings on the citation sources panel:
- Use a non-consuming lookbehind so back-to-back citations no longer drop
  every other source.
- Match balanced parenthetical groups in URLs so disambiguation links like
  .../Foo_(a)_(b) are no longer truncated.
- Mask inline code (and unclosed streaming fences) so example citations in
  code aren't scraped as real sources; masking preserves indices.
- Extract a shared createMarkdownLinkComponent factory used by both message
  content and markdown content, removing the duplicated `a` renderer.
 2026-07-03 18:03:43 +08:00
ly-wang19
ddca86411b
docs: add production database guidance (#3830) 
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
 2026-07-03 15:41:49 +08:00
NekoPunch
629477fd5c
docs: fix stale docs and typos (#3913) 2026-07-03 15:16:20 +08:00
ly-wang19
c05dd46e25
docs: clarify docker sandbox mount paths (#3833) 
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
 2026-07-03 10:55:20 +08:00
DanielWalnut
25ea6970a6
feat(runtime): implement goal continuations (#3858) 
* implement goal continuations

* fix(goal): address review findings for goal continuations

- goal: key the no-progress breaker on a signature of the latest visible
  assistant evidence instead of the evaluator's volatile free-text, so it
  actually fires on stalled turns; thread the signature through every
  worker persist / no-progress call site
- goal: align _stand_down_reason default caps with should_continue_goal
  (8 / 2) so the two gate functions agree on goals missing the fields
- runtime: offload the synchronous checkpointer fallback via
  asyncio.to_thread (goal.py + worker.py) to keep blocking IO off the loop
- frontend: i18n the GoalStatus "Goal" label (goalLabel in en/zh/types)
- frontend: extract pure composer helpers into input-box-helpers.ts with
  unit tests (parseGoalCommand, readGoalResponseError, skill suggestions)
- tests: cover the evidence-based no-progress and default-cap behavior
- docs: align backend/AGENTS.md goal paragraph with actual behavior
- e2e: prettier-format chat.spec.ts (fixes the lint-frontend CI failure)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(frontend): hide goal continuation counter until the agent continues

The goal status bar rendered a raw "0/8" before any auto-continuation, which
read as a mysterious score. Now the counter is hidden until
continuation_count > 0, then shows "Continuing N/M" with a tooltip explaining
the auto-continuation cap.

- Extract getGoalContinuationDisplay into a pure helper (hides at 0) + unit tests
- Add goalContinuing / goalContinuationTooltip i18n keys (en/zh/types)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(goal): address review findings for goal continuations

Frontend correctness
- Fix the optimistic /goal result permanently shadowing server goal state:
  the streamed continuation counter never surfaced for a goal set in-session.
  Extract a shared useActiveGoal hook (used by both chat pages) that reconciles
  the optimistic copy with server state via a goalReconciliationKey, de-duping
  the copy-pasted goal block across the two pages.
- Stop /goal status|clear failures from escaping handleSubmit as unhandled
  rejections (handleGoalCommand now returns success; the run only starts when a
  goal was actually saved).
- Use a function replacer for the goal-status toast so an objective containing
  $&/$1 isn't treated as a replacement pattern.

Backend cleanliness / correctness
- De-duplicate four byte-identical helpers (_call_checkpointer_method,
  _message_type, _additional_kwargs, _is_visible_message) by importing them
  from runtime.goal instead of re-defining them in the run worker.
- Remove the dead `checkpoint_tuple.tasks` durability guard (CheckpointTuple has
  no tasks field) and document that pending_writes is the durability signal.
- Decompose the 176-line _prepare_goal_continuation_input: extract
  _reread_goal_and_checkpoint and a _persist closure so the thread-unchanged
  guard and stand-down persistence aren't open-coded three times. Document the
  last-writer-wins write-window limitation as a follow-up.
- Add a shared parse_goal_command helper and use it from the TUI and IM-channel
  /goal handlers (one place for the status/clear/set semantics).

Tests
- Restore the 11 command-registry tests dropped by the previous goal change
  (filter_commands ranking/description, build_registry builtins/skills, resolve
  cases) alongside the new goal tests.
- Add coverage for the IM-channel _handle_goal_command, the TUI _handle_goal
  handler, parse_goal_command, and goalReconciliationKey.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix goal review feedback

* fix goal continuation checkpoint races

* prioritize goal commands while streaming

Route composer submits through a shared helper so /goal commands can be handled before the streaming stop shortcut, while ordinary streaming submits still stop the active run.

Testing: cd frontend && pnpm exec rstest run tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; cd frontend && pnpm check

* preserve goal status during clarification

Keep omitted stream goal fields distinct from explicit null clears so clarification interrupts do not hide an active thread goal that is still present in the checkpoint.

Testing: pnpm exec rstest run tests/unit/components/workspace/use-active-goal.test.ts tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; pnpm check; git diff --check

* style: format active goal hook

Run Prettier on use-active-goal.ts to satisfy the frontend lint workflow formatting gate.

Testing: pnpm format; pnpm exec rstest run tests/unit/components/workspace/use-active-goal.test.ts tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; pnpm check; git diff --check

* fix goal review followups

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-03 08:49:33 +08:00
Nan Gao
21b3510226
feat(agents): feature-gate the agents UI behind the agents_api flag (#3757) (#3769) 
* feat(gateway): add GET /api/features for frontend feature gating (#3757)

* feat(agents): add useAgentsApiEnabled feature-flag hook (#3757)

* feat(agents): gate /workspace/agents segment on agents_api flag (#3757)

* feat(sidebar): grey out Agents button with tooltip when agents_api disabled (#3757)

* fix(sidebar): show disabled Agents tooltip on hover, harden a11y + e2e (#3757)

- Wrap the disabled Agents button in a hoverable tooltip trigger so the
  'feature not enabled' hint shows in the expanded sidebar, not only when
  collapsed (the SidebarMenuButton tooltip prop is hidden unless collapsed).
- Add tabIndex={-1} and drop the redundant onClick preventDefault.
- Add e2e coverage for the disabled state + a default /api/features mock.

* style(sidebar): move cursor-not-allowed to the hoverable span (#3757)

* test(agents): anchor e2e agents-API request filter with a path regex (#3757)

* fix(agents): don't expose backend config in disabled message; tell user to contact admin (#3757)

The disabled panel previously said 'Set agents_api.enabled: true in
config.yaml', leaking backend configuration to end users. Replace with a
generic 'not enabled on this server, contact your administrator' message
(matching the existing nameStepApiDisabledError copy). e2e now asserts the
contact-admin message and that no config.yaml/agents_api text is rendered.

* make format

* fix(agents): keep agents_api flag sticky during /api/features outage (#3757)

Failing open re-mounted the agents UI and re-triggered the 403 storm when
agents_api was genuinely disabled and /api/features was down. Persist the
last definitive answer and fall back to it (sticky) before failing open,
only failing open when nothing has ever been observed. Read the cached
value after mount so the first client render matches the server (no
hydration mismatch on the non-loading-gated sidebar).

* fix(sidebar): make disabled Agents entry keyboard/SR accessible (#3757)

The disabled-state explanation was hover-only: tabIndex={-1} removed the
entry from the tab order and the reason lived only in a pointer-triggered
tooltip. Keep it in the tab order and wire aria-describedby to a
visually-hidden reason so keyboard and screen-reader users learn why it is
disabled.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-02 11:33:32 +08:00
Yi Deng
331c949b95
fix(gateway): require admin for global skills management endpoints (#3855) 
* fix(gateway): require admin for global skills management endpoints

The skills router exposed its management endpoints with no authorization
check (only Depends(get_config)), while the MCP router guards the
equivalent global extensions_config mutations with require_admin_user
(added in the #3425 security hardening). Skills storage is global/shared
across users (no per-user path), lives in the same extensions_config.json
as MCP servers, and custom skill SKILL.md content is injected into every
user's agent system prompt. Any authenticated non-admin user could mutate
or read global skills that affect all tenants:

- POST /api/skills/install: install from an arbitrary thread_id, writing
  into the global custom skills tree
- PUT /api/skills/custom/{skill_name}: rewrite a global skill, injecting
  instructions into all users' agent prompts (cross-tenant prompt injection)
- DELETE /api/skills/custom/{skill_name} and rollback: tamper with global skills
- GET /api/skills/custom, GET .../{skill_name}, GET .../history: read raw
  global custom skill bodies/history
- PUT /api/skills/{skill_name} (enable toggle): writes the shared
  extensions_config.json and refreshes the system prompt for every tenant,
  so a non-admin could enable/disable any skill globally. There is no
  per-user skill state, so this is a global mutation, not a preference.

Add require_admin_user to every endpoint above, mirroring the MCP router.
The shared read path used internally by update/rollback was extracted into
a non-auth helper (_read_custom_skill_response) so internal reuse does not
double-check auth.

Only the read-only GET /api/skills and GET /api/skills/{skill_name} stay
open to normal users: they return just name/description/enabled and back
the user-facing settings UI.

Tests:
- New tests/test_skills_router_authz.py: a non-admin user gets 403 on every
  guarded endpoint (including the enable toggle); basic listing stays open;
  admins can still toggle.
- Update tests/test_skills_custom_router.py to authenticate as admin.
- pytest tests/ -k skill -> 350 passed, 1 skipped; ruff clean.

Signed-off-by: DengY11 <151997860+DengY11@users.noreply.github.com>

* fix(frontend): gate skill enable/install UI behind admin + handle 403

Follow-up to the backend change that made the global skills mutations
admin-only. Without a matching UI change, a non-admin user would hit a
silent 403 when toggling a skill in Settings -> Skills or clicking
"Install skill" on a .skill artifact.

- core/skills/api.ts: add SkillRequestError (status + isAdminRequired),
  throw it from loadSkills/enableSkill on non-ok responses and from
  installSkill on 403 (other install errors keep the soft-failure
  contract).
- core/skills/hooks.ts: useSkills no longer retries on SkillRequestError.
- skill-settings-page.tsx: show an "admin required" message on 403, and
  disable the enable toggle for non-admins (mirrors the MCP tools page).
- artifact-file-detail.tsx / artifact-file-list.tsx: only render the
  "Install skill" action for admins, and surface an admin-required toast
  if a 403 still occurs.
- i18n: add settings.skills.adminRequired / installAdminRequired (en + zh).

Auth/no-auth and static-website modes synthesize an admin user, so these
gates do not affect single-user/local deployments.

Verified locally: pnpm check (eslint + tsc) passes with no new errors,
pnpm build succeeds, and the dev server renders / and /login (200) with
no compile/runtime errors.

Signed-off-by: DengY11 <151997860+DengY11@users.noreply.github.com>

* style(frontend): format skills hook for prettier

Keep the admin-guard skills hook aligned with Prettier output so the frontend format check passes in CI.

---------

Signed-off-by: DengY11 <151997860+DengY11@users.noreply.github.com>
 2026-07-02 11:17:23 +08:00
Ryker_Feng
ddb097a72f
feat(community): add Brave image search community tool (#3866) 
* Add Brave image search community tool

* fix(community): length-cap Brave web_search queries

Apply _clean_query in web_search_tool so over-long queries are trimmed
to Brave's 400-char limit before the API call, matching image_search_tool
and avoiding HTTP 422 from the Brave Search API.

* fix(community): harden Brave image search SSRF guard and dimension mapping

Address PR review findings:
- Catch ValueError from urlparse so a malformed bracketed-IPv6 URL skips
  one item instead of crashing the whole image_search call
- Reject IPv6 literals embedding a non-global IPv4 (IPv4-mapped, 6to4,
  NAT64, IPv4-compatible), closing the loopback/private SSRF bypass
- Report width/height from the dict of the URL actually returned, so a
  surviving thumbnail no longer reports the dropped original's dimensions

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-02 11:00:49 +08:00
Huixin615
3748344303
fix(frontend): validate attachment limits before upload (#3900) 
* fix(frontend): validate attachment limits before upload

* fix(frontend): avoid duplicate upload limit toasts
 2026-07-02 10:32:53 +08:00
Nan Gao
4fcb4bc366
feat(subagents): persist and display subagent step history (#3779) (#3845) 
* feat(subagents): persist and display subagent step history (#3779)

Capture both assistant turns and tool outputs during subagent execution,
stream them in task_running events, and persist them as subagent.* run
events so the subtask card's step timeline survives a reload.

Backend:
- step_events.py: pure layer (capture_step_message, build_subagent_step,
  subagent_run_event) shared by streaming and persistence
- executor.py: capture ToolMessage outputs, not just AIMessage turns
- worker.py: persist task_* custom events to RunEventStore (category
  "subagent" keeps them out of the thread feed; list_events backfills)

Frontend:
- core/tasks/steps.ts + api.ts: SubtaskStep model, messageToStep,
  eventsToSteps, mergeSteps, fetchSubtaskSteps
- subtask card accumulates live steps and backfills on expand
- carry run_id onto history content messages for the events endpoint

* fix(subagents): show AI turns in subtask card + paginate step backfill (#3779)

Two follow-ups to the subagent step-history feature:

Problem 1 — reload backfill could silently truncate the step timeline because
list_events capped at 500 events (seq-ASC) across the whole run. Add task_id
filtering + an after_seq forward cursor to list_events (all three stores +
abstract base + the /events route), and make fetchSubtaskSteps page through one
task's subagent.step events until a short page. No schema migration: the DB
filter rides the existing run-scoped index via event_metadata["task_id"].

Problem 2 — the card only rendered tool steps, so persisted AI turns were never
shown. Replace toolStepsForDisplay with stepsForDisplay: interleave AI reasoning
turns (with text) and tool steps by message_index, drop blank-text AI turns, and
drop the trailing final-answer AI turn when completed (already shown as result).
Card renders AI steps as muted clamped markdown with a sparkles icon.

Tests: store task_id/after_seq filtering + pagination across memory/db/jsonl,
the /events route forwarding, stepsForDisplay rules, and fetchSubtaskSteps
pagination. Docs updated in both AGENTS.md.

* make format

* fix(subagents): capture full multi-tool step tail, batch step persistence, cap tool-call args (#3779)

Address PR review findings on the subagent step-history feature:

1. executor.py streamed on stream_mode="values" and captured only
   messages[-1] per chunk, so a multi-tool-call turn (ToolNode appends
   one ToolMessage per call in a single super-step) lost all but the last
   tool output in both the live task_running stream and the persisted
   history. Replace with capture_new_step_messages, which walks the
   newly-appended tail (and still re-checks the trailing message on
   no-growth chunks so id-less in-place replacements survive).

2. worker.py persisted each step with the store's low-frequency put()
   (a per-thread advisory lock per call); a deep subagent (max_turns=150)
   emits hundreds of steps on the hot stream loop. Replace with
   _SubagentEventBuffer, which batches via put_batch (flush on terminal
   subagent.end, at FLUSH_THRESHOLD, and in the worker finally).

3. build_subagent_step capped only text; tool_calls[].args were copied
   verbatim, so a large write_file/bash payload produced an unbounded
   subagent.step row. Cap each call's serialized args at
   SUBAGENT_STEP_MAX_CHARS, flagged args_truncated.

Tests updated/added for all three; AGENTS.md refreshed.

* fix(subagents): merge backfill into latest subtask state; reuse message_content_to_text (#3779)

Address the remaining two PR review findings:

4. subtask-card's fetchSubtaskSteps().then(updateSubtask) closed over a
   stale tasks snapshot: a late-resolving backfill wrote setTasks({...stale}),
   clobbering SSE steps/status and sibling subtasks that arrived during the
   fetch. useUpdateSubtask now reads/writes through a tasksRef mirroring the
   latest state (ref-to-latest), and the pure per-subtask transition is
   extracted to core/tasks/subtask-update.ts::computeNextSubtask (unit-tested).

5. step_events._content_to_text duplicated deerflow.utils.messages.
   message_content_to_text; call the shared helper instead (guarding None
   content with 'or ""' so a tool-call-only turn still renders as "").

Tests added for computeNextSubtask and the None-content case; AGENTS.md docs updated.
 2026-07-02 07:43:09 +08:00
xiawiie
2e15e3fe0d
fix: generate fallback title for interrupted first-turn runs (#3874) 
* fix: generate title for interrupted first turn

* test(title): cover partial-exchange + dict-form messages

Harden the interrupted-run fallback path added in 19fc34fd:

- TitleMiddleware._should_generate_title now accepts a lone first-turn
  user message when allow_partial_exchange=True, so the worker can still
  derive a title if cancellation lands before any AI chunk is checkpointed.
- runtime/runs/worker._ensure_interrupted_title computes the next
  checkpoint step defensively (treat missing/non-int step as 0) and
  renames a shadowed ckpt_config local for readability.
- Add four unit tests in tests/test_title_middleware_core_logic.py:
  partial-exchange allows user-only, partial-exchange still respects an
  existing title, dict-form messages are recognized, and the sync
  fallback path derives a title from dict-form messages — matching what
  channel_values stores in the checkpoint.

Refs #3859.

* fix: persist interrupted-title via channel_versions bump

Address PR #3874 review feedback: ``_ensure_interrupted_title`` previously
called ``aput(..., new_versions={})``. LangGraph's DB-backed savers
(``PostgresSaver`` and the v4 ``SqliteSaver`` blob layout) strip inline
``channel_values`` from ``put`` and only persist blobs for channels named
in ``new_versions`` — so the fallback ``title`` channel was dropped on
read-back and ``threads_meta.display_name`` stayed ``"Untitled"`` after
refresh on those backends. The original in-memory e2e passed because
``InMemorySaver`` keeps the inline snapshot verbatim.

Fix mirrors ``_rollback_to_pre_run_checkpoint`` in the same file: bump
``channel_versions["title"]`` (via ``checkpointer.get_next_version`` when
available, else int/string fallbacks), persist the new version on the
checkpoint, and declare it in ``new_versions`` so the DB savers actually
write the blob.

Regression coverage in ``tests/test_run_worker_rollback.py``:

- ``test_ensure_interrupted_title_bumps_channel_version_and_declares_it_in_new_versions``
  — exact ``aput`` invariants: ``new_versions == {"title": 1}``, the
  written checkpoint's ``channel_versions["title"]`` is bumped, and the
  pre-existing ``messages`` version is preserved.
- ``test_ensure_interrupted_title_bumps_existing_string_version`` —
  string-shaped prior version (some savers use UUID-style versions);
  bumped value must differ from the prior, no overwrite-in-place.
- ``test_ensure_interrupted_title_skips_when_title_already_set`` — title
  short-circuit; no extra ``aput``.
- ``test_ensure_interrupted_title_returns_none_when_no_checkpoint`` —
  no checkpoint yet; returns ``None`` without writing.
- ``test_ensure_interrupted_title_round_trip_with_real_sqlite_checkpointer``
  — full round-trip against a real ``AsyncSqliteSaver`` on a disk-backed
  DB, then closes and re-opens the saver to simulate a fresh
  connection. The fallback title must still be present on the second
  ``aget_tuple``. This is the exact scenario the review flagged.

Validated locally with the full backend suite: 5195 passed, 18 skipped.

Refs #3859. Addresses review on #3874.

* test(worker, title): harden interrupted-title fallback for every saver

Defensive coverage on top of the channel_versions fix (commit 05253957),
addressing edge cases surfaced during a second-pass review of #3874.

Worker:
- Extract version bump into ``_bump_channel_version(checkpointer, current)``
  with explicit fallbacks for int / float / numeric-string / UUID-shaped
  string / None / bool, AND a wrap-around defense when the saver's
  ``get_next_version`` raises or returns an unchanged value. The
  invariant is: returned version MUST differ from the prior. Without
  this, a saver bug (or a custom backend) could leave
  ``new_versions={"title": v}`` no-op on DB savers — the very class of
  bug the original review pointed out.

Title middleware:
- Coerce ``state.get("messages")`` from ``None`` to ``[]`` on both
  ``_should_generate_title`` and ``_build_title_prompt``. A
  partially-initialized checkpoint can carry ``messages=None`` on the
  channel_values channel (the worker reads raw channel_values, not
  BaseMessages), and the default kwarg only protects against a missing
  key. Repro: ``TypeError: 'NoneType' object is not iterable`` from
  the next() generator — confirmed by reverting the fix and watching
  ``test_*_handles_none_messages_channel`` go red.

Tests (TDD-verified red→green for the new asserts):
- ``test_run_worker_rollback.py``:
  * ``_bump_channel_version`` — 8 tests covering every version type
    (int, float, numeric string, UUID-style string, None, bool) and
    every saver-side fault mode (no ``get_next_version`` / raising /
    stuck on identity).
  * ``test_ensure_interrupted_title_*`` — 5 additional helper
    boundary tests: title.enabled=false short-circuit; empty
    messages list; messages=None; aput-error propagation (helper
    contract: caller swallows, not the helper); idempotency on a
    real InMemorySaver across two invocations.
  * ``test_ensure_interrupted_title_preserves_non_title_channel_versions``
    — pins that ``new_versions`` only contains ``"title"`` and that
    other channels' versions are untouched (regression anchor for a
    sloppier draft that bumped every channel).
  * ``test_worker_finally_block_swallows_helper_exceptions`` — pins
    the integration contract: even if the helper raises, the worker's
    threads_meta status sync still runs and ``publish_end`` is still
    awaited so the SSE stream closes cleanly.

- ``test_title_middleware_core_logic.py``:
  * 4 additional tests: ``messages=None`` on both
    ``_should_generate_title`` and ``_build_title_prompt``; the
    ``role: user`` / ``role: assistant`` (OpenAI-style) dict
    normalization; partial-exchange path with a dict-form message.

Verification:
- ``PYTHONPATH=. uv run pytest tests/ -x --ignore=tests/blocking_io -q``
  → 5215 passed, 18 skipped.
- ``ruff check`` + ``ruff format --check`` clean on every touched file.
- Red/green TDD verification: temporarily reverted the
  ``new_versions={}`` fix → 4 new tests went red as expected; restored
  and the suite is green again. Same red/green dance for the
  ``messages=None`` coercion.

Refs #3859. Addresses second-pass review on #3874.

* fix(title): ignore dict context reminders in fallback

* fix(worker): link interrupted-title checkpoint to its parent

The title-bump checkpoint written by ``_ensure_interrupted_title`` was
landing without a ``parent_checkpoint_id`` — a real orphan in the
LangGraph history graph. Reproduction (disk-backed AsyncSqliteSaver):

  [seed] checkpoint_id = 1f173dbc...
  [helper] wrote title = "Why is the sky blue?"
  [issue 1] new checkpoint = 1f173dbc..., parent = None
  [issue 1] is new checkpoint orphaned? True

Root cause: ``_ensure_interrupted_title`` built ``write_config`` as
``{"thread_id": ..., "checkpoint_ns": ...}`` only. ``BaseCheckpointSaver``
implementations read ``configurable.checkpoint_id`` from that config as
the *parent* id when inserting (see ``langgraph/checkpoint/sqlite/aio.py``
``aput``: ``config["configurable"].get("checkpoint_id")`` becomes the
``parent_checkpoint_id`` column). With no value, the saver writes NULL —
the new checkpoint is a tree root.

Consequences:
- Any future LangGraph ``runs.resume_from`` / time-travel feature has no
  backward edge to walk past the title-bump.
- History-visualization UIs built on ``alist()`` render the title-bump as
  a sibling of the prior checkpoint, not its descendant.

Fix: read ``checkpoint_id`` off the tuple's own config and thread it into
``write_config["configurable"]["checkpoint_id"]`` before calling ``aput``,
the same pattern every middleware-driven write uses.

Three new regression tests against real ``AsyncSqliteSaver`` (disk-backed,
fresh connections so we exercise the on-disk read path):

- ``test_ensure_interrupted_title_links_new_checkpoint_to_its_parent`` —
  asserts ``latest.parent_config["configurable"]["checkpoint_id"]`` equals
  the seeded checkpoint id. TDD red-green verified: reverting the fix
  flips this test red with ``AssertionError: title-bump checkpoint must
  have a parent_config``.
- ``test_ensure_interrupted_title_appears_in_history_with_audit_marker``
  — pins the audit contract: the title-bump entry in ``alist()`` carries
  ``metadata.source == "update"`` and ``metadata.writes`` contains
  ``runtime_interrupt_title``. This is a deliberate design choice — we
  do NOT hide the entry from history (audit trail belongs in the saver),
  but its source and writes marker MUST be unambiguous so UIs/tools can
  identify it.
- ``test_ensure_interrupted_title_survives_immediate_next_turn`` —
  cancel → immediate user follow-up scenario. Simulates the agent's next
  turn appending a (user, ai) pair without touching the title channel,
  then opens a fresh saver and verifies the title is still present after
  the next-turn checkpoint write. Pins the channel-version-blob
  invariant established by commit 05253957 — without the
  ``new_versions={"title": v}`` declaration there, the title blob would
  vanish from the DB and this test would read back ``None``.

Verification:
- ``PYTHONPATH=. uv run pytest tests/ -x --ignore=tests/blocking_io -q``
  → 5222 passed, 15 skipped.
- ``ruff check`` + ``ruff format --check`` clean on every touched file.
- Reproduction script confirms ``parent_checkpoint_id`` is now non-null
  and the next-turn read-back preserves the fallback title.

Refs #3859.

* Revert "fix(worker): link interrupted-title checkpoint to its parent"

This reverts commit c763ed9334781db1acdce0f5f33d663d8d5f80ff.

* test: trim over-engineered test coverage

Reduce review surface area on PR #3874 by dropping defensive tests that
don't pin a real invariant. After self-review:

- ``_bump_channel_version``: 8 tests → 2 (happy path + saver-error
  fallback). Dropped float / bool / numeric-string / UUID-string /
  missing-get-next-version / stuck-get-next-version branches — those
  are speculative scaffolding for savers we don't ship.
- ``_ensure_interrupted_title``: dropped ``returns_none_when_title_disabled``,
  ``returns_none_with_no_user_message``, ``returns_none_when_no_checkpoint``
  — boundary guards already exercised by the e2e test and the
  ``handles_none_messages_channel`` regression anchor.

Net: -107 lines of test code. Remaining coverage still pins every
red-green-verified invariant (channel_versions bump, string-version
bump, idempotency, sqlite round-trip, non-title channel preservation,
aput-error contract, worker finally swallowing, partial-exchange).

Verification: 5209 passed, 15 skipped.

* fix: harden interrupted title finalization

* fix: serialize interrupted title finalization

* fix: preserve interrupt semantics during title finalization

* fix: preserve delayed interrupted title recovery

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
 2026-07-02 00:13:01 +08:00
Ryker_Feng
a8f950feb6
feat(community): add Browserless web_capture screenshot tool (#3881) 
* feat(community): add Browserless web_capture screenshot tool

Add a web_capture tool that renders a page via Browserless /screenshot and
presents it through the artifact system, alongside the existing Browserless
web_fetch provider.

Hardening:
- SSRF guard: reject URLs resolving to private/loopback/link-local (incl. the
  169.254.169.254 cloud-metadata endpoint)/reserved/multicast/unspecified
  addresses; opt out via allow_private_addresses for internal targets.
- Surface a warning when Browserless renders a target page that itself
  responded with a non-2xx/3xx status (X-Response-Code), so an error/anti-bot
  page is not mistaken for valid visual evidence.
- Dedupe colliding output filenames instead of silently overwriting prior
  captures.

Docs: comment out token: $BROWSERLESS_TOKEN in tool examples (an unset $VAR
fails AppConfig startup) and document allow_private_addresses.

* fix(community): format web_capture guard + document local Browserless startup

Address PR #3881 review: fix the lint-backend failure (ruff format on
browserless/tools.py) and add local Browserless startup instructions to
CONFIGURATION.md so reviewers can run the service to try web_fetch/web_capture.
 2026-07-01 23:41:58 +08:00
Xinmin Zeng
b431053ef0
fix(frontend): stop rendering reasoning twice for reasoning+answer messages (#3870) 
A reasoning+content AI message with no tool calls was grouped into both an
`assistant:processing` group (the ChainOfThought panel above the bubble) and an
`assistant` group (the bubble's own Reasoning collapsible), so its reasoning
rendered twice. Such a message now becomes only the assistant bubble; the
ChainOfThought panel keeps handling intermediate reasoning and tool steps.

Closes #3868
 2026-07-01 23:35:48 +08:00
AochenShen99
442248dd06
feat: preserve durable context across summarization (#3887) 
* feat: preserve durable context across summarization

* fix: harden durable context review gaps

* style: format delegation ledger live test

* chore: remove stale delegation ledger prefix

* fix: address durable context review feedback
 2026-07-01 22:49:17 +08:00