354 Commits

Author SHA1 Message Date
YxinMiracle
b6ba739297
fix(sandbox): scope restored references to authenticated threads (#5736)
* fix(sandbox): scope restored references to authenticated threads

* docs(sandbox): trim inherited guidance

* fix(sandbox): reject restored references without thread scope

---------

Co-authored-by: YxinMiracle <“939157765@qq.com”>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-23 09:53:49 +08:00
RongJie G
e01314442c
fix(mcp): scope sessions and task access by thread incarnation (#5556)
* fix(mcp): scope sessions and task access by thread incarnation

* fix(mcp): preserve thread incarnation in delegated subagents

* fix(mcp): preserve incarnation in durable batches

* fix(mcp): bind standalone graph lifecycle context

* fix(studio): preserve implicit thread creation metadata

---------

Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-23 08:33:20 +08:00
spud
74f006951c
fix(sandbox): bound AIO list_dir with a directory deadline (#5662)
* fix(sandbox): bound remaining shell session cleanup requests

release_command_scope(), close()'s scoped-session drain, and close()'s recovery-session cleanup called shell.cleanup_session() without a request budget, so a stalled cleanup could hold the thread-key serializer, scoped.lock, or the sandbox lock for the SDK's default transport budget. Pass the existing _bounded_cleanup_request_options() (5s, max_retries=0) at those three sites; _cleanup_session_best_effort() and its swallow-and-log contract are unchanged, as are create_session and list_dir lifetimes.

* fix(sandbox): bound AIO list_dir with a directory deadline

Give AioSandbox.list_dir its own 60s directory deadline with a 65s no-retry
host envelope, independent of bash_command_timeout. Preserve #5634's
shell-generation selection: list_dir runs on the current recovery session once
the implicit shell is fenced, and an ambiguous list_dir outcome - transport
timeout or an ambiguous returned status - fences whichever generation actually
executed the request, dropping local recovery ownership and attempting bounded
best-effort cleanup instead of leaving that session reusable. hard_timeout
remains definite termination and keeps the targeted session reusable. Only
completed/None results are parsed, so a partial find is never returned as a
complete listing.

Session creation RPC lifetime remains out of scope.

* fix(sandbox): recover sessions after transport failures

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-22 19:49:00 +08:00
YxinMiracle
b8ab097ca2
fix(models): support official DeepSeek managed profiles (#5718)
Co-authored-by: YxinMiracle <“939157765@qq.com”>
2026-09-22 17:53:21 +08:00
Keswyk Cheung
ef21855b8d
fix: discard stale todo reminders during context compaction (#5614)
* fix: rebuild todo reminders after context compaction

* chore: remove implementation plan from PR

* refactor: share todo reminder message name

* docs: keep agent guidance within CI size budgets

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-22 17:45:01 +08:00
Wenchao An
e2f19d8335
feat(plugins): full-stack plugin APIs and bookmarks (#5647)
* feat(plugins): add full-stack contributions and bookmarks example

* ci(plugins): provision bookmark gateway for browser tests

* fix(plugins): authenticate module downloads through configured backend

* fix(plugins): isolate contributions and localize extension UI

* fix(plugins): preserve bookmark agent routing and contain async callbacks

* fix(plugins): pin durable batch workers to app extension snapshots
2026-09-22 11:18:57 +08:00
asts
0ff2e9ddcf
fix(mcp): honor configured stdio working directories (#5643)
* fix(mcp): honor configured stdio working directories

* fix(mcp): preserve defaults for empty working directories

Treat empty stdio cwd values as omitted, including unresolved environment references. Add real subprocess regressions for discovery and pooled-call defaults, plus direct connection-dictionary coverage.
2026-09-22 11:15:06 +08:00
Totoro
9ae1e585bb
fix(sandbox): prune ignored entries before the listing limit (#5676)
* fix(sandbox): prune ignored entries before the listing limit

* docs(sandbox): clarify listing filter assumptions
2026-09-22 10:56:49 +08:00
YxinMiracle
ce50a28dfd
fix(auth): enforce write permission for Live Browser WebSockets (#5621)
* fix(auth): enforce write permission for Live Browser WebSockets

Resolve route permissions before accepting browser streams and require threads:write, matching the existing HTTP navigation endpoint.

Preserve shared authorization failure semantics and reject unexpected setup errors before acquiring a browser session.

Add authorization, frame delivery, input dispatch, cancellation, and ownership regressions. Document the admission-only permission check.

* fix(auth): improve browser authorization diagnostics

---------

Co-authored-by: YxinMiracle <“939157765@qq.com”>
2026-09-22 10:43:49 +08:00
YxinMiracle
519afe4041
fix(security): prevent external system-role message injection (#5651)
* fix(agnet): system prompt bug

* fix(security): address system-role review feedback

* fix(ci): keep agent guidance within size budget

---------

Co-authored-by: YxinMiracle <“939157765@qq.com”>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-22 10:38:16 +08:00
xbzz1018
6fae79047c
docs: clarify upgrade workflow (#5654)
* docs: clarify upgrade workflow

* docs: fix upgrade section placement
2026-09-21 22:03:47 +08:00
Weng Qiang
60d5659d1d
fix(client): honor agent MCP plugin selections (#5630)
* fix(client): honor named-agent MCP plugin selections

* fix(client): normalize MCP selection cache identity
2026-09-21 21:45:52 +08:00
lihongyuan99
1e3bfa09d4
fix(frontend): read web_fetch titles that start with blank lines or indented headings (#5560)
* fix(frontend): read web_fetch titles that start with blank lines or indented headings

* docs(frontend): describe the indented-code guard as it actually behaves

* fix(frontend): reject mixed code indentation in web-fetch titles

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 20:26:48 +08:00
yijun Lin
c52ad191f4
fix(subagents): recognize empty regular files in remote acceptance pr… (#5559)
* fix(subagents): recognize empty regular files in remote acceptance probes

* test(subagents): address empty-file acceptance review feedback

* docs(subagents): condense empty artifact guidance

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 20:03:53 +08:00
PeaceMaker-best
e4ce26f3d2
feat(memory): add DeerMem scope-isolation benchmark (#5564)
* feat(memory): add DeerMem scope-isolation benchmark

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>

* fix(benchmark): grade persisted summaries and retry failed extractions

---------

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 20:01:28 +08:00
Dan Caldr
19266a5eac
fix(middlewares): end length-capped turns cleanly, prevent todo re-engagement, annotate write_file budget (#5569)
* fix(middlewares): end length-capped turns cleanly, prevent todo re-engagement, annotate write_file budget

When a model hits its per-response output cap (finish_reason=length) while
emitting a write_file tool call, ModelLengthFinishReasonMiddleware suppresses
the truncated call and stamps model_length_termination. TodoMiddleware must
not re-engage (jump_to=model) on such a capped turn -- doing so re-emits the
same oversized call into the same cap, producing up to 3 futile responses
with junk fragments instead of a clean truncation notice.

Changes:
- TodoMiddleware.after_model: skip completion reminder jump when
  additional_kwargs.model_length_termination is present (follows the existing
  deerflow_error_fallback precedent).
- ModelLengthFinishReasonMiddleware: always append the length notice when
  tool calls were suppressed, even when partial text survived (collapses the
  visible-content ternary). Fixes a latent bug in append_visible_text that
  silently dropped string content.
- tools.get_available_tools: annotate write_file's model-visible description
  with the model's configured max_tokens output budget. Guarded extraction
  safely handles missing or non-numeric tokens, and the tool is cloned via
  model_copy to keep module-level singletons immutable across assemblies and
  prevent guidance leakage to unbudgeted models.
- release_policy_parameters() updated for both middlewares.
- AGENTS.md chain entries (#20, #35) and module docstrings updated within
  AG002 guidance limits.
- Tests: 8 new/focused unit tests + 1 updated pin + 1 real create_agent()
  integration test reproducing the incident (thread b1723286).

* fix(tools): use effective model cap for write_file guidance

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 19:28:21 +08:00
Totoro
c668716737
fix(sandbox): stop E2B reconciliation from reviving warm-pool sandboxes (#5562)
* fix(sandbox): stop E2B reconciliation from reviving warm-pool sandboxes

Periodic reconciliation probed every discovered remote sandbox with
Sandbox.connect() before the locality check, and the check itself only
consulted _sandboxes, not _warm_pool. A sandbox parked by release() was
therefore adopted back to active on the first pass, and because the SDK
normalizes connect(timeout=None) to its 300s default and the control
plane extends a running sandbox's expiry when now+timeout is later,
each 60s pass kept pushing the expiry forward — idle warm sandboxes
never hit their configured idle_timeout.

Treat _sandboxes and _warm_pool ids as locally tracked up front: skip
probing them (no timeout-mutating connect), keep them canonical, and
route only genuinely remote candidates through the duplicate-reap path.
Extend the post-probe adoption recheck to _warm_pool so a release that
lands mid-probe cannot be promoted back to active either.

Fixes #5550

* fix(sandbox): keep active E2B VMs alive and sweep expired warm entries

Address review on #5562:

- Reconciliation now refreshes the remote TTL of locally active
  sandboxes through their cached client (never connect()), restoring
  the keepalive for turns that outlive idle_timeout without reviving
  warm-pool VMs.
- Warm-pool entries parked longer than idle_timeout are dropped during
  reconciliation — their VMs are expected to be reaped by the control
  plane — releasing the ownership lease and the capacity slot they
  would otherwise pin until reclaim, eviction, or shutdown.
- Remove the now-dead thread-local canonical sort; locally tracked ids
  are skipped unconditionally, so the ordering hint had no effect.

* fix(sandbox): preserve active E2B keepalive and shared capacity

* fix(sandbox): serialize E2B reconciliation lifecycle transitions

* fix(sandbox): fence E2B ownership and timeout lifecycle writes

* fix(sandbox): isolate ownership heartbeats from E2B timeout IO

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 19:23:45 +08:00
spud
906c3d4554
fix(mcp): make durable task claims cancellation-safe (#4966)
* feat(mcp): re-scope to MCP task claim lifecycle only

Keep PR #4966 a small, closed MCP lease/cancellation state-machine change and
move RunJournal and Run lifecycle work into dedicated follow-ups. This branch
contains only the MCP task claim lifecycle:

- mcp task release/snapshot fencing by owner + per-claim lease token
- phase-level single-flight poll/cancel/notification owners with retained handoff
- routine cancellation no longer persisted as a task failure diagnostic
- bounded ordinary release ownership retention past the drain deadline
- 0018_mcp_task_lease_tokens migration + migration/bootstrap head assertions
- wait_for_task_until helper (MCP uses it); worker-specific capture helper moved
  to the run-finalization follow-up

RunJournal (journal.py + test_run_journal.py) and run lifecycle
(manager/worker/store/run sql + run tests) are preserved on
backup/cancellation-safety-full and will be raised as separate follow-ups.

* fix(mcp): unblock claims after ambiguous handoff resolves

A phase-level single-flight owner only guards an ambiguous claim outcome. Once
the claim resolves, the phase owner is released immediately; the handoff may
continue releasing returned rows as bounded, service-owned background work
(transferred to _compensation_tasks on timeout). Per-claim token fencing rejects
a late release against a newer claim generation, so a stuck release no longer
locks the whole phase until process restart.

- README: drop the stale progress-snapshot sentence from the bounded ordinary
  release description.
- service: pop the identity-checked phase owner as soon as the claim outcome is
  known, then release returned rows with the bounded path; carry the release in
  _compensation_tasks if it exceeds the drain deadline.
- mcp/AGENTS.md: document that only an unresolved claim outcome (not the handoff)
  blocks later phase scans, and that returned-row releases may continue in the
  background once the owner is released.
- tests: pin that the phase owner is released before a stuck release finishes
  while the release stays service strong-owned.

* refactor(mcp): remove unused single-record claim wrappers

_poll_one, _cancel_one, and _notify_one are unreachable in production: the
worker always processes claimed records through _run_claimed_batch, so these
wrappers preserved a second, dead single-record lifecycle (state is None)
whose only observable behavior was a wrapper-specific cancellation release.

Remove the three wrappers and migrate the regressions that guarded their
cancel/release invariants to exercise the production _run_claimed_batch path
(operation=_*_one_claimed, release=_release_*_after_cancellation). The single
wrapper-only "state is None" contract (test_poll_release_hang_without_batch)
is deleted; all 11 remaining invariants (CancelledError preservation, repeated
cancellation, poll-only token-fenced lease release, notification claimed vs
dispatched phase release, hung compensation -> service ownership, and
background compensation exactly-once observation) are now covered through the
real batch lifecycle.

* fix(mcp): fence claim-owned mutations against stale generations

The per-claim token check in the ORM release/apply paths was only in the
SELECT; the final write went out by primary key. On SQLite (where
with_for_update() is a no-op) a mutation from an older claim generation
could therefore clear a claim that a newer generation had reclaimed after lease
expiry — the exact distributed lease-fencing failure the per-claim token was
meant to prevent.

Make every claim-owned mutation a single atomic conditional UPDATE with the
owner and per-claim token in the WHERE clause (rowcount 0 => stale, return
False, no mutation):

- release_claim: atomic fence; record the poll-failure event after the fence
  wins (same transaction, holding the write lock).
- apply_snapshot / apply_cancel_snapshot: atomic fence; record the event after.
- finish_notification_run: atomic fence; use a CASE on event_version >>
  dispatch_version to keep a newer event pending for redelivery instead of
  swallowing it as delivered.

Add one regression per path: a stale generation's release/apply/finish after a
same-worker reclaim is rejected and never clears the newer claim.

* test(mcp): pin the migration chain head to the lease-token revision

0026_mcp_task_lease_tokens becomes the alembic head, so the chain-head pin in the 0025 repair test had to move on. Follow the 0023 precedent there (single head plus expected predecessor) instead of pinning a literal head, and give the new revision its own migration test, which owns the pin and covers the nullable claim-token columns on upgrade and their removal on downgrade.

* refactor(mcp): close cancellation cleanup leftovers

* fix(mcp): retain cancelled release diagnostics

* test(mcp): remove obsolete settled compensation case

* test(mcp): cover interleaved lease reclaim races

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 19:11:42 +08:00
Eilen Shin
bd995a6a26
fix(agent): align unattended prompt with tool policy (#4919)
* fix(agent): align autonomous interaction guidance

* fix(agent): harden interaction policy selection

* fix(gateway): protect legacy interaction flags

* fix(channels): honor explicit interaction mode

* docs(agent): reduce inherited guidance size

* fix(agent): honor unattended policy across approval paths

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 18:57:49 +08:00
hataa
71087f2f8e
feat(authz): gate composer send on runs:create (Phase 4 follow-up, #4063) (#5528)
* feat(authz): gate composer send on runs:create (Phase 4 follow-up, #4063)

Mirrors #5294's stop gating on the send side: both chat routes pass
canCreateRuns (from PERMISSIONS.RUNS_CREATE, lockstep with the backend
enum) into the shared composer. The gate sits at the top of
submitThreadMessage — the single choke point every composer entry
(submit button, Enter, goal-set-triggered run) funnels through — and
denies with a toast plus a rejected promise so PromptInput keeps the
text. The idle submit button is disabled and explains the boundary via
conditionally-spread aria-label/title (startTurnUnavailable, en/zh);
while streaming the button stays the runs:cancel stop affordance.

Also removes the unreachable kind === "stop" branch in handleSubmit
(flagged during #5294's review): the Enter path early-returns with the
streaming toast before the classifier runs.

* fix(frontend): reject denied goal starts before saving state

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 17:45:03 +08:00
Daoyuan Li
015ebcc88c
fix(frontend): preserve literal think tags in code (#5540)
* fix(frontend): preserve literal think tags in code

* Fix indented continuations of inline code spans

* Respect paragraph boundaries when extracting inline reasoning

* fix: respect block boundaries and escaped backtick runs

* fix(frontend): avoid quadratic reasoning delimiter backtracking

* fix(frontend): track reasoning fences inside list items

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-20 17:38:20 +08:00
Wenchao An
1f437f86c6
feat(agents): persist default knowledge scopes for custom agents (#5579)
* feat(agents): persist default knowledge scopes for custom agents

* style: format agent knowledge guidance

* fix(i18n): clarify default knowledge reset hint

* fix(knowledge): preserve retries for initially unbound agents
2026-09-20 16:31:35 +08:00
Xuehao Xu
8ef58eaa90
feat(models): manage shared models from Settings (#5596)
* feat(models): add admin UI for shared model management

* docs(gateway): keep model guidance within size budget
2026-09-20 16:13:58 +08:00
ZJPex
aa4e43a2bc
fix(ragflow): batch validation for large document selections (#5572)
* fix(ragflow): batch validation for large document selections

* docs(ragflow): align documentation language with repository conventions

* docs(ragflow): preserve spacing before validation heading

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-19 15:09:36 +08:00
Wenchao An
42334f26d7
feat(capabilities): unify catalog, plugin configuration and agent selection (#5497)
* feat(capabilities): unify catalog, plugin configuration and agent selection

* fix(capabilities): address review isolation, validation and demo issues

* fix(capabilities): preserve concurrent selections and guide launcher repair
2026-09-19 12:14:24 +08:00
xiaodu55
2ff006b0c0
feat(middleware): add deterministic PII redaction for model-bound context (#5527)
* feat(middleware): add deterministic PII redaction for model-bound context

* fix(middleware): claim national IDs before cards, redact Command results, preserve ToolMessage fields

- Reorder detectors so checksum-gated national IDs run before the credit-card
  detector; an 18-digit resident ID whose digit run also passes Luhn is no
  longer mislabeled [CREDIT_CARD_n] (review finding, reproduced at 0a2a9d0)
- Redact ToolMessages carried in Command.update.messages, mirroring
  ToolResultSanitizationMiddleware's dc_replace pattern
- Rebuild redacted ToolMessages via model_copy so artifact and
  response_metadata survive
- Extend the numbered middleware chain in agents/middlewares/AGENTS.md

* fix(middleware): span one redactor per Command result; refresh stale AGENTS.md entry range

- Placeholder numbering now continues across every ToolMessage carried in a
  single Command result (one _Redactor per _redact_result call) instead of
  restarting per message
- The renumbered AGENTS.md chain still referenced entries 9-12 in the
  ToolReceiptMiddleware entry; it now reads entries 10-13

* docs(agents): trim PiiRedactionMiddleware entry to fit the AGENTS.md chain budget

The main merge (fb36e0e) pushed the effective middlewares chain to 98341
bytes, 37 over the 98304 hard limit checked by agent-guidance (AG002).
Compress the entry while keeping the load-bearing facts: config gate, both
interception points incl. Command coverage, detector order rationale,
per-result numbering continuity, irreversibility, memory follow-up.

* fix(middleware): redact compaction input and reinjected summaries; harden detectors

Review round 3 on #5527:
- [P1] SummarizationMiddleware invokes its summary model directly from
  before_model, outside PiiRedactionMiddleware's wrap_model_call, so raw
  thread state reached the summary model and reinjected summaries carried
  raw PII into model-bound context. Add a shared redact_text() seam: the
  compaction prompt is redacted in _build_summary_prompt (app_config
  already flows into the middleware) and DurableContextMiddleware redacts
  summary_text at reinjection via a new pii_redaction_config knob wired
  at both assembly sites.
- [P2] CUIT is 2+8+1 digits, not 2+10+1.
- [P2] Digit-anchored patterns use digit-aware lookarounds instead of
  Unicode \b, which CJK characters defeat (身份证110105… / 手机号138…).
- [P2] The international-phone pattern no longer treats newlines as
  separators, so a candidate cannot swallow the following numeric field
  and then fail validation as a whole.

* fix(pii): redact title input and reserve summary placeholders

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-19 11:26:46 +08:00
JasonH
82cf57a9c3
fix(utils): return empty text for content-less messages (#5563)
* fix(utils): return empty text for content-less messages

message_content_to_text fell through to str(content), so a message whose
content is None yielded the truthy literal "None". Two call sites already
work around it with a local `or ""` and name the helper in the comment; the
subagent executor's `text if text else "No response generated"` fallback and
the archive's `if not text: continue` skip cannot work around it, so a
content-less terminal turn was reported as an answer of "None" and a
content-less LLM error fallback surfaced "None" instead of its error_detail.

* test(utils): cover non-None content compatibility and document fallbacks

* test(utils): cover contentless task history and refresh guard comments

---------

Co-authored-by: Lengshuang <90967079+Lesereingrape@users.noreply.github.com>
Co-authored-by: JasonH <4430962+yang0228@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-19 10:56:21 +08:00
alanhuangyoo
23cc4f4d00
fix(agents): close delegations a stopped run left in progress (#5507)
* fix(agents): close delegations a stopped run left in progress

Every task call is recorded in the delegation ledger as in_progress and
only moves on when its ToolMessage arrives. When the user stops a run
while a subagent is running, the task tool re-raises the cancellation
and no ToolMessage is written, so the entry stayed in_progress for the
rest of the thread and every later model call was told "already
delegated; do NOT delegate again; wait for or build on the result".

When a run starts with a new user message, mark entries that an earlier
run left in_progress and that have no ToolMessage as cancelled. Resumed
runs, which have no new user message, keep the current behaviour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(agents): share the run-opening boundary between capture and closure

Also pin that in_progress entries without a run_id are never closed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(agents): clarify legacy delegation reply handling

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-19 10:40:22 +08:00
Kris
208ba7cc65
feat(memory): opt-in tolerant MarkdownMemoryStorage (#3124) (#5545)
* feat(memory): add opt-in tolerant MarkdownMemoryStorage (fixes #3124)

User-memory summary load is now tolerant of corrupt/partially written
files: a Markdown summary (fenced `memory-json` block, best-effort
structured fallback) or JSON is accepted, and an unrecoverable file
recovers to an empty memory instead of raising MemoryStorageCorruption
and taking down the agent. On-disk JSON format and the JSON UI are
unchanged, so enabling `memory.storage_class: markdown` is fully opt-in
and cannot break existing deployments.

Co-authored-by: WorkBuddy <noreply@workbuddy.ai>

* fix(markdown-memory): address review - schema-safe loader, greedy fence, quarantine, CI tests

- drop lossy structured Markdown fallback (invalid-shape crash on load/save)
- greedy fence match: ``` inside remembered strings round-trips losslessly
- quarantine unreadable summary before returning None (no silent erase)
- remove dead _render_memory_markdown (deferred to write-path change)
- move tests to backend/tests/ so CI runs them; update storage_class docs

* fix(markdown-memory): address review - schema-safe loader, greedy fence, quarantine, CI tests

- drop lossy structured Markdown fallback (invalid-shape crash on load/save)
- greedy fence match: ``` inside remembered strings round-trips losslessly
- quarantine unreadable summary before returning None (no silent erase)
- remove dead _render_memory_markdown (deferred to write-path change)
- move tests to backend/tests/ so CI runs them; update storage_class docs

* fix(markdown-memory): address review - schema-safe loader, greedy fence, quarantine, CI tests

- drop lossy structured Markdown fallback (invalid-shape crash on load/save)
- greedy fence match: ``` inside remembered strings round-trips losslessly
- quarantine unreadable summary before returning None (no silent erase)
- remove dead _render_memory_markdown (deferred to write-path change)
- move tests to backend/tests/ so CI runs them; update storage_class docs

* fix(markdown-memory): address review - schema-safe loader, greedy fence, quarantine, CI tests

- drop lossy structured Markdown fallback (invalid-shape crash on load/save)
- greedy fence match: ``` inside remembered strings round-trips losslessly
- quarantine unreadable summary before returning None (no silent erase)
- remove dead _render_memory_markdown (deferred to write-path change)
- move tests to backend/tests/ so CI runs them; update storage_class docs

* fix(markdown-memory): address review - schema-safe loader, greedy fence, quarantine, CI tests

- drop lossy structured Markdown fallback (invalid-shape crash on load/save)
- greedy fence match: ``` inside remembered strings round-trips losslessly
- quarantine unreadable summary before returning None (no silent erase)
- remove dead _render_memory_markdown (deferred to write-path change)
- move tests to backend/tests/ so CI runs them; update storage_class docs

* fix(memory): correct tolerant Markdown parsing and regression tests

---------

Co-authored-by: WorkBuddy <noreply@workbuddy.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-19 10:38:22 +08:00
NEEDI
990c7b95aa
fix(uploads): handle UTF-8 BOM in document summaries (#5541)
Co-authored-by: NEEDI <298523066+sherxlg-gif@users.noreply.github.com>
2026-09-19 10:08:09 +08:00
Wenchao An
34bbeb1806
feat(knowledge): add verifiable RAGFlow source citations (#5551)
* feat(knowledge): add verifiable RAGFlow source citations

* docs(knowledge): scope RAGFlow guidance to its own directory

* fix(knowledge): preserve citations through rendering and budgets
2026-09-19 07:44:05 +08:00
NanPan
2bdae7518d
fix(memory): drain shutdown workers across cancellation (#5531)
* fix(memory): drain shutdown workers across cancellation

* fix(memory): contain shutdown config resolution failures

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-18 17:19:10 +08:00
zhangwei-way
b6503e9a35
feat(knowledge): add per-message RAGFlow retrieval scope (#5238)
* feat(knowledge): integrate RAGFlow retrieval and management

* test(knowledge): cover merged listing tool

* feat(knowledge): add per-message retrieval scope

* chore(docs): remove unrelated document

* docs(knowledge): add interaction screenshots

* feat(knowledge): simplify scope selector trigger

* docs(knowledge): refresh selector screenshot

* feat(knowledge): defer standalone management

* docs(knowledge): show chat-only scope UI

* fix(knowledge): honor scope on clarification replies

* fix(knowledge): harden scoped replay validation

* docs(knowledge): clarify replay scope precedence

* fix(knowledge): keep provider settings on tools

* fix(config): preserve tools-only knowledge settings

* fix(knowledge): submit custom assistant identity

* refactor(knowledge): trim PR scope changes

* fix(knowledge): sanitize document scope display

* feat(knowledge): enable scope selection in main chat

* fix(knowledge): emphasize active scope icon without button frame

* fix(knowledge): close context scrubbing and refresh e2e checks

* fix(knowledge): preserve idempotent canonical retries

* fix(knowledge): accept promptless conversation runs

* style(knowledge): format backend regression tests

* chore(knowledge): trim PR scope and fix frontend format

* fix(knowledge): remove shared-scope notice

* fix(knowledge): remove scope persistence notice

* docs(knowledge): include main chat in catalog scope

* fix(knowledge): preserve scope recovery and upgrades

* fix(config): preserve LightRAG knowledge upgrades

---------

Co-authored-by: foreleven <for-eleven@hotmail.com>
2026-09-18 16:59:31 +08:00
Xuehao Xu
114b78d7db
test(persistence): cover historical run-change repair and rollback (#5518)
* fix(persistence): repair run-change clock schema skipped by the 0023 insertion

0023_run_change_seq was chained ahead of the already-shipped
0023_user_preferences revision, so databases stamped at that revision or
later treat it as an applied ancestor and never execute it: the
run_change_clock table and runs.change_seq column are permanently missing
and the first thread deletion fails with 'no such table:
run_change_clock' (#5516). 0025_repair_run_change_seq re-applies the same
guarded DDL on upgrade and no-ops on healthy shapes. RunChangeClockRow and
UserPreferenceRow are also registered in the ORM model registry.

Fixes #5516

* fix(persistence): preserve run-change schema when rolling back repair

---------

Co-authored-by: 1553126902 <1553126902@qq.com>
2026-09-18 16:55:07 +08:00
NanPan
57d027f903
fix(subagents): drain owned batch stop across cancellation (#5525)
* fix(subagents): drain owned batch stop across cancellation

* fix(subagents): preserve cancellation across stop failures

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-18 14:27:33 +08:00
wd_pan
cc27730348
feat(memory): add opt-in relevance-aware retrieval ranking (#5251)
* feat(memory): add opt-in relevance-aware retrieval ranking

Add a deterministic, network-free lexical relevance strategy for DeerMem
(issue #4495): memory_search ranks every fact in scope by idf-weighted
token overlap combined with confidence, with optional greedy-MMR diversity
against near-duplicate facts; prompt injection ranks facts against the
current-turn query threaded from DynamicContextMiddleware through the new
optional `query` keyword on MemoryManager.get_context/aget_context.

Defaults preserve the legacy confidence-only behavior exactly; no prompt,
storage-format, or vector/embedding-dependency changes.

Refs #4495
Signed-off-by: pwd11 <fvdsrc@163.com>

* fix(memory): bound relevance retrieval and apply review feedback

Bound tokenization and index shared stems, preserve mixed CJK tokens, warm jieba, and align missing confidence with legacy injection. Cache MMR token sets and stop selection at result or injection budgets. Document retrieval-adapter precedence and add regression coverage. Refs #4495.

Signed-off-by: pwd11 <fvdsrc@163.com>

* fix(memory): preserve backend compatibility and normalize relevance

Signed-off-by: pwd11 <fvdsrc@163.com>

* fix(memory): omit absent query hints and share injection IDF

Signed-off-by: pwd11 <fvdsrc@163.com>

* test(memory): retain timeout mock until injection worker exits

Signed-off-by: pwd11 <fvdsrc@163.com>

* docs(agents): drop root guidance compaction

Signed-off-by: pwd11 <fvdsrc@163.com>

* fix(memory): validate token prefixes and preserve upload queries

---------

Signed-off-by: pwd11 <fvdsrc@163.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-18 11:32:04 +08:00
Tsai Yuan
972020cf85
fix: keep streamed answers out of thinking and improve local bash probes (#5001)
* fix: improve streaming reasoning and local bash guidance

* test: cover reasoning-only processing group

* fix(frontend): preserve streaming reasoning order

* test(frontend): cover reasoning tool-call regrouping

* fix(frontend): keep thinking-only blocks in processing

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-18 11:30:11 +08:00
Xuehao Xu
c24fd1e66f
fix(frontend): keep clarification text outside execution steps (#5508)
* fix(frontend): keep clarification text outside execution steps

* refactor(frontend): share clarification run boundary detection
2026-09-18 09:35:32 +08:00
tiammomo
d8db4e1bf4
feat(scheduled-tasks): search task titles and prompts (#5355)
* feat(scheduled-tasks): search task titles and prompts

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>

* docs(scheduled-tasks): separate search from time validation notes

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>

* fix(frontend): order scheduled-task imports

---------

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-18 09:25:32 +08:00
tiammomo
73590a626d
fix(scheduled-tasks): reject nonexistent local execution times (#5348)
* fix(scheduled-tasks): reject nonexistent local execution times

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>

* test(scheduled-tasks): align valid-time fixture with instant preservation

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>

---------

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-18 09:07:25 +08:00
tiammomo
6d725f1ccb
fix(scheduled-tasks): preserve unchanged one-time execution instants (#5330)
* fix(scheduled-tasks): preserve unchanged one-time execution instants

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>

* fix(scheduled-tasks): reset edit state before task remounts

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>

* test(scheduled-tasks): exercise timezone fallback on UTC runners

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>

---------

Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
2026-09-18 08:06:31 +08:00
Weng Qiang
a23dbdd837
feat(tavily): support configured search domain filters (#5513) 2026-09-18 07:25:49 +08:00
liunianxuxie
53f2a73d23
fix(setup): honor sandbox image in BOM-prefixed configs (#5515)
* fix(setup): honor sandbox image in BOM-prefixed configs

* fix(setup): normalize CRLF and preserve captured pull arguments
2026-09-17 22:00:22 +08:00
liunianxuxie
e19c37d813
fix(setup): detect optional extras in BOM-prefixed configs (#5504) 2026-09-17 14:53:54 +08:00
Weng Qiang
7f68fa2881
fix(tavily): use web_fetch credentials for extraction (#5496)
* fix(tavily): use fetch tool credentials for extraction

* test: register scoped Tavily agent guidance

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-17 09:13:40 +08:00
PeaceMaker-best
e831720304
feat(skills): rank deferred discovery by agent intent (#5369)
* feat(skills): rank deferred discovery by intent

* fix(skills): preserve exact selections and cache search metadata

---------

Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-17 07:59:58 +08:00
Totoro
94e69d6ff7
feat(frontend): reference conversations from the composer (#5465)
* feat(frontend): reference conversations from the composer

Adds a "Reference a conversation" button next to the attachment button,
shown only while GET /api/features reports read_conversation enabled. It
opens a picker over the recent-conversation list (current thread excluded,
capped at max_references) and shows removable chips in the composer.

On send the thread IDs ride SendMessageOptions.conversationReferences into
run context.conversation_references, which the Gateway consumes at
admission; the LangGraph SDK drops unknown top-level body fields. A
display-only copy ({thread_id, title}) on the visible human message lets
the transcript render read-only chips linking to the source.

References are per message: not persisted with the draft and cleared on
send or thread switch; regenerating or editing a turn runs without them
unless they are attached again.

Related to #5398. Depends on #5463.

* fix(frontend): pin the run-context contract and finish the picker states

Both thread.submit paths now build their run context through one exported
buildRunContext helper, tested directly: attached references travel as a
plain string[] under context.conversation_references only when the caller
passed them, a stray key in local settings is dropped instead of forwarded,
and the regenerate/edit replay path never carries references.

The picker shows a loading row while the conversation list is still in
flight instead of claiming there are no conversations, and the transcript
chip group is labelled with the previously unused referencedConversations
translation.

* fix(frontend): route conversation-reference chips to custom-agent sources

The picker offered custom-agent conversations but kept only the thread ID
and title, so transcript chips always linked to /workspace/chats/{id} and
dropped the source's custom-agent context on navigation.

Preserve the agent identity end to end: the picker now attaches
agentNameOfThread() (context first, then metadata.agent_name, mirroring
pathOfThread) to the selection, the display-only additional_kwargs metadata
round-trips it as agent_name, and the transcript chip passes it to
pathOfThread so custom-agent sources resolve to
/workspace/agents/{agent}/chats/{id}.

Tests: agent_name metadata round-trip and malformed-entry tolerance, picker
toggle carrying the metadata agent with run context winning, and a
picker-to-transcript regression pinning the /workspace/agents/writer/chats/
source-1 href.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
2026-09-16 21:19:52 +08:00
Xuehao Xu
f0cb67b223
feat(extensions): expose incremental run evidence reader (#5405)
* feat(extensions): expose incremental run evidence reader

* fix(extensions): address run evidence review feedback

* docs(extensions): clarify run deletion reconciliation

* docs(migrations): align current head documentation

* fix(extensions): isolate run evidence event reads

* test: avoid pinning run change migration to latest head

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-16 21:18:05 +08:00
Zeren Wang
a58ab484a6
feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash (#5443)
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash

Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.

Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
  admission pinning (both server-owned sets + worker hoist); latest-only
  request-scoped <project> block via DynamicContextMiddleware
  wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
  prefix + marker + provenance, never persisted); journal audit
  fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
  locked check-and-set; hash-qualified immutable shelf storage with
  Paths helpers; upload/list/content/delete-to-trash routes; project
  delete trashes the shelf in-transaction; request-scoped bounded
  <documents> index with honest count/shown + actionable overflow note;
  list_project_documents/read_project_document tools registered only on
  pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
  parity); POST from-thread with provenance; attach-to-thread with
  lock-staged copy (archived source allowed); read-only thread-files
  view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
  file moves), purge (continuous row lock across unlink/delete/commit,
  retryable on FS errors), retention sweep (lazy + startup, 24h orphan
  guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
  archived banner, content-missing rows), /workspace/trash route,
  sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
  + specs.

Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
  artifact + content responses; unified unsandboxed-iframe PDF preview
  (fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
  claims + os.link commit with suffix retry; same-name re-upload now
  unique-names instead of replacing); hidden staging only, no visible
  placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
  revalidation; drain locked filesystem work on cancellation; preserve
  bytes when an insert's commit state is uncertain (including trashed
  rows)
- original-integrity checks before serving text or cached conversions;
  content_missing surfaced in list responses (UI reads the flag, no
  409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
  declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
  rejects fractional/infinite values; composer counts staged
  attachments; pending attachments persist until submission or removal;
  in-flight instruction/rename edits survive save refetches; shelf and
  trash pagination; conversation-file and thread-files pages stay
  subscribed to refetches

Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.

Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
  bounded: the indexed expiry purge still runs on every trigger
  (GET /api/trash/documents, POST /api/trash/purge) while the
  O(all rows + all files) reconciliation is throttled to one run per
  user per 15 minutes (process-local, per-user window). The startup
  sweep now runs as a background task instead of blocking gateway
  readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
  like the render path, so a pasted, fenced <project>/<documents> snippet
  survives markdown export while real injected blocks (never fenced) are
  still removed. Fence regexes moved to a dependency-free leaf module to
  avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
  attribute (the upstream e2e contract locates it via :not([title])), and
  the upstream artifact-preview spec now pins the new contract: PDFs
  render unsandboxed, images keep sandbox="".

* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title

- Shutdown cancelled only the shield around the background startup sweep,
  so an all-users reconciliation that outlived the 5s budget kept walking
  rows and files while the document repo and DB engine were disposed
  underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
  which cancels the task and drains it before worker exit: the shield
  keeps the wait bounded, the cancel makes it final (CancelledError lands
  at the sweep's next await, and `_run_startup_trash_sweep` only catches
  `Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
  previous fix round, leaving the PDF frame without an accessible name
  while its siblings keep theirs. Restore it (WCAG frame titles), assert
  it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
  instead of `iframe:not([title])`.

* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel

`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.

* fix(projects): round-4 review — make Empty trash delete what it confirms

`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.

Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.

Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
2026-09-16 18:46:18 +08:00
Ryker_Feng
6ca12c6c8f
feat(models): add user model favorites (#5441)
* feat(models): add user model favorites

* fix(models): use anchored favorites picker

* fix(models): keep model picker compact

* fix(models): remove obsolete favorites search path

* fix(models): address picker review feedback
2026-09-16 18:42:13 +08:00