209 Commits

Author SHA1 Message Date
Aleksandr Sapronov
613b90b0e6
feat(scheduler): make scheduled-run recursion_limit configurable (#4848)
Adds scheduler.recursion_limit to config.yaml (default 1000, clamped by
max_recursion_limit) so scheduled background runs can use a different
recursion limit than the web UI. The value is read at dispatch time, so
a YAML edit applies to the next scheduled run without a Gateway restart.

Also logs a warning when the resolver falls back to the default or
clamps the configured value.
2026-08-24 07:09:08 +08:00
muguo
917fe595fc
feat(sandbox): add OpenSandbox provider (#4877) 2026-08-23 15:46:10 +08:00
Ryker_Feng
e5bf3ccf45
fix(feishu): preserve inbound attachment files (#4903)
* fix(feishu): preserve inbound attachment files

* fix(feishu): harden inbound attachment handling

* fix(dingtalk): reserve symlinked inbound filenames
2026-08-23 10:45:30 +08:00
DanielWalnut
062ba9ddfc
feat: integrate MiniMax Code as a native ACP agent (#4846)
* feat: integrate MiniMax Code as an ACP agent

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-17 08:16:14 +08:00
Aari
47b258ebd7
feat(mcp): add ordinary durable task driver (#4690)
* feat(mcp): add durable task runtime foundation

* fix(chart): sync embedded config version

* fix(mcp): isolate task polls during shutdown

* feat(mcp): track consecutive poll errors on mcp_tasks

poll_attempt_count grows on every claim (successful polls included), so it
cannot drive a failure backoff without misjudging normal long tasks. Add
consecutive_poll_error_count: incremented when a claim is released after a
poll error, reset to zero by any applied snapshot. The backoff/terminal
policy that consumes it lands with the first concrete driver.

* fix(mcp): harden durable task lifecycle

* feat(mcp): add ordinary durable task driver

* test(mcp): address durable task review feedback

* fix(mcp): preserve submit tool descriptions

* fix(mcp): bound remote task calls

* fix(mcp): bound persisted task payloads

* fix(mcp): preserve task tool error details

* fix(mcp): enforce durable task boundaries

* test(mcp): cover task config snapshot lifecycle

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-08-15 14:26:38 +08:00
rain02333z-spec
828363705a
fix(scheduler): support safe multi-instance scheduler recovery (#4713)
* fix(scheduler): reject unsafe multi-worker startup

* fix(scheduler): support safe multi-instance recovery

* fix(scheduler): make multi-instance recovery lease-safe

* fix(scheduler): address multi-instance review feedback

* docs(scheduler): add multi-instance upgrade notes

---------

Co-authored-by: rain02333z-spec <225106191+rain02333z-spec@users.noreply.github.com>
2026-08-14 23:50:40 +08:00
Eilen Shin
ce4ef1bb2f
fix(channels): bound inbound intake and worker lifecycle (#4800)
* fix(channels): bound inbound intake and worker lifecycle

* fix(channels): harden overload retry and shutdown draining

* fix(channels): retain shutdown task ownership
2026-08-14 12:35:24 +08:00
Hao Zhe
a263af2845
feat(mcp): add official OpenViking tools integration (#4745)
* feat(mcp): add OpenViking tools integration

* fix(mcp): warn on ineffective tool overrides

* docs(mcp): clarify OpenViking resource removal

* fix(mcp): expose native OpenViking forget tool

* docs(mcp): document OpenViking forget guardrail
2026-08-11 13:56:02 +08:00
AoHanBei
7910126923
fix(agents): make SQL store signatures content-sensitive (#4709) 2026-08-08 20:36:13 +08:00
lllyfff
99c926b7bb
fix(mcp): bring-up has no timeout and externalized tool outputs are counted as undelivered artifacts (#4657)
* fix: bound MCP server bring-up timeouts and exclude externalized tool outputs from delivery verification

Two related robustness fixes:

1. MCP server bring-up was unbounded. tool_call_timeout only covered
   session.call_tool(); tool discovery (subprocess spawn + initialize +
   tools/list) and persistent stdio session initialization could hang
   forever, blocking agent construction (and on the Gateway event loop,
   the whole process). Add a per-server session_init_timeout
   (default DEFAULT_MCP_SESSION_INIT_TIMEOUT = 60s, null disables) that
   bounds both discovery and pooled-session initialization. The session
   pool's existing cancellation handling tears down a session stuck
   mid-creation in its own task.

2. ToolOutputBudgetMiddleware externalizes oversized tool outputs into
   outputs/.tool-results/ (configurable tool_output.storage_subdir). The
   workspace-change scanner and run delivery verification counted those
   files as produced artifacts, so any run that externalized a tool output
   without also presenting a real artifact failed with
   "Artifact delivery incomplete". Exclude TOOL_RESULTS_DIRNAME via a
   shared constant (mirroring BROWSER_FRAMES_DIRNAME) and thread the
   configured storage_subdir through snapshot capture so both
   workspace-changes events and delivery verification stay clean.

* review: enforce single-segment tool_output.storage_subdir; document discovery-timeout cleanup

Address review feedback:

1. A custom tool_output.storage_subdir with a path separator (e.g.
   cache/tool-results) silently no-oped the workspace-scanner exclusion:
   os.walk yields one-segment dirnames, so a nested value never matched and
   its files were counted as produced artifacts again. ToolOutputConfig now
   validates storage_subdir as a single directory name (rejects separators,
   .., absolute, empty) with tests, so the exclusion is always sound.

2. The discovery-timeout path now documents why cancellation is safe, mirroring
   the session-init note: discovery runs inside the adapter's nested async
   context managers, and stdio_client's finally terminates the process tree
   (SIGTERM->SIGKILL on POSIX, process-tree on Windows), so a timed-out npx
   subprocess and its children are reaped rather than accumulating.

* review: log session-init timeouts and align API response model default with runtime config

Address second-round review feedback:

1. A session-init timeout raised TimeoutError without any log, unlike the
   discovery timeout which logs a WARNING. Wrap the bounded get_session in a
   try/except that logs the timeout (server name + seconds) and re-raises, so
   operators can diagnose tool-call failures caused by hung MCP sessions.

2. McpServerConfigResponse.session_init_timeout defaulted to None while
   McpServerConfig defaults to 60s: a server created via PUT /api/mcp/config
   without the field was persisted with null (no timeout) while the same
   server created in the config file got 60s. Align the response-model default
   to DEFAULT_MCP_SESSION_INIT_TIMEOUT so API-created and file-created servers
   behave the same; an explicit null still opts out.

* review: narrow the discovery-timeout handler to the bounded wait_for path

The except TimeoutError clause covered both the bounded wait_for branch and
the bare discovery branch. With session_init_timeout opted out (None), a
TimeoutError raised by discovery itself would hit the %.1f format with None:
logging raises TypeError internally, the WARNING is silently dropped, and a
--- Logging error --- traceback goes to stderr.

Narrow the handler to wrap only the wait_for call, where the branch condition
guarantees the timeout value is not None. A discovery-internal TimeoutError on
the opted-out path now falls through to the generic failure handler and is
reported as 'tool discovery failed' with exc_info. Covered by a regression
test that asserts the skip is reported without any broken format.
2026-08-05 08:56:18 +08:00
ajayr
d732b90dc3
feat(channels): add Buzz (Nostr) channel connector (#4649)
* feat(channels): add Buzz (Nostr) channel connector

Adds a Buzz (https://github.com/block/buzz) channel so DeerFlow can join a
Nostr-relay workspace as a member: it answers @mentions in channels, replies
to DMs, and streams answers by editing one message in place.

  * app/channels/buzz_nostr.py — pure NIP-01 helpers: canonical event ids,
    BIP-340 signing/verification, chat/edit/auth builders, relay frames.
  * app/channels/buzz.py — BuzzChannel: one NIP-42-authenticated websocket,
    channel discovery (kind 39000) with one subscription per channel, live
    membership tracking (44100/44101), per-channel replay watermarks, and
    replies posted once then edited in place (kind 40003).
  * app/channels/buzz_run_policy.py — same-thread serialization, mirroring
    the Feishu precedent.

Inbound is gated in order: signature verification, self-drop, /connect
bind-and-return, pubkey allowlist, then mention / DM / mention-free /
thread-follow. Off by default; needs the new optional `buzz` extra
(coincurve, lazily imported), which detect_uv_extras resolves from
channels.buzz.enabled the same way it already handles channels.discord.

Two relay behaviours drove the design and are worth knowing when reviewing:
a global {"kinds":[9]} subscription receives nothing from buzz-relay and a
multi-value "#h" filter receives nothing either, so one REQ per channel is
required; and a single global `since` cursor skips quiet channels, so
watermarks are per channel.

Signed-off-by: Ajay R <ajayr@formbuddy.com>

* fix(channels): only publish assistant messages from the IM stream

`_accumulate_stream_text` decided what streamed `messages-tuple` payloads
become displayable text by rejecting ONLY payloads whose `type` contained
"tool", so it published everything else. DeerFlow writes hidden model
context into the messages channel as ordinary messages -- memory recall and
the rewritten user turn as hidden HumanMessages (DynamicContextMiddleware),
the `<durable_context_data>` block as another (DurableContextMiddleware) --
and LangGraph fans those state writes out on the messages stream, so they
reached every streaming IM channel as the assistant's reply.

Proved live on a Buzz relay: the connector published a `<memory>` fact block
and, in another run, a verbatim echo of the user's own inbound message.
Affects Feishu, Telegram, WeCom and Buzz; worst on Buzz, where each update
is an immutable public Nostr event that a corrective edit cannot unpublish.

Invert the filter to an allowlist of assistant message types. Two new pure
helpers keep it testable:

- `_stream_payload_type` resolves the type from both shapes the function
  already handles: the `model_dump()` shape the gateway emits, and
  LangChain's `to_json()` constructor shape whose own `type` is the literal
  "constructor" and whose class name is the tail of the `id` path.
- `_is_assistant_stream_type` matches "ai"/"assistant" by PREFIX, not
  substring -- ordinary words contain "ai" ("chain", "domain"), and a
  substring test would admit a foreign type name by accident.

The bare-`str` branch is removed: an untyped payload cannot be attributed to
the assistant, nothing in DeerFlow produces one (serialize_messages_tuple
always emits `[message_dict, metadata]`), and a runtime that emitted raw text
deltas would emit hidden context the same way. Per-message-id buffering and
merging are unchanged.

Tests pin both directions, including multi-chunk merging across one message
id, so the allowlist cannot silently kill streaming, plus an end-to-end
`_handle_streaming_chat` test asserting the live payload never reaches an
outbound message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Ajay R <ajayr@formbuddy.com>

* chore(helm): bump config_version to 33 in chart values and README

config.example.yaml moved to 33 for the buzz channel block; the chart's
embedded config example and its README copy track it (config_version only
drives the outdated-config warning, per scripts/check_config_version.sh).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Ajay R <ajayr@formbuddy.com>

---------

Signed-off-by: Ajay R <ajayr@formbuddy.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 08:29:22 +08:00
Felix Wang
61c153ff09
feat(tui): add transparent terminal background (#4631)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-08-05 08:15:28 +08:00
Xinmin Zeng
11bb8ddcd9
fix(cli): allow overriding agent recursion limit (#4615) 2026-08-04 08:33:10 +08:00
Felix Wang
cd8825b0f0
docs(uploads): document current upload context (#4632) 2026-08-02 08:52:27 +08:00
Felix Wang
e221bddb38
feat: support per-server MCP tool name prefixes (#4624)
* feat: support per-server MCP tool name prefixes

* refactor: pass MCP connection config directly

* fix: preserve unprefixed MCP tool names in session pool
2026-08-01 22:33:11 +08:00
Huixin615
133a82c6c2
fix: isolate MCP server toggles from invalid peer configs (#4577)
* fix: isolate MCP server toggle updates

* fix: write extensions config atomically

* fix: normalize MCP transport aliases
2026-07-31 08:32:39 +08:00
阿泽
9c7cd4cad3
feat(sandbox): add thread data mount override for upload sync (#4536) 2026-07-28 23:41:14 +08:00
Aari
a5059b8284
fix(subagents): isolate callbacks and activate skills lazily (#4497) 2026-07-28 23:29:14 +08:00
阿泽
ea74367502
fix(runtime): honor LangGraph Server identity for user-scoped data (#4538)
* fix(runtime): honor LangGraph Server identity for user-scoped data

* fix(runtime): scope custom agent SOUL by resolved user
2026-07-28 22:59:14 +08:00
ShitK
b1984cf4ab
fix(security): reject legacy MCP credentials in run metadata (#4448)
* docs: design run metadata secret admission

* docs: refine run metadata secret boundaries

* docs: plan run metadata secret fix

* fix(security): centralize legacy run metadata policy

* fix(security): reject secrets at run admission

* fix(security): hide legacy secrets from history APIs

* docs(security): migrate MCP credentials to secret context

* fix(security): redact legacy runnable config metadata

* fix(security): reject legacy config metadata credentials

* fix(security): hide legacy secrets from run kwargs

* docs(security): clarify config redaction boundary

* docs: keep issue 4416 planning local
2026-07-28 21:31:23 +08:00
Vanzeren
6f53fd5e99
feat(runtime): enforce artifact delivery from workspace snapshots (#4494) 2026-07-27 22:27:16 +08:00
March-77
b22f85c686
fix(sandbox): reconcile E2B sandboxes safely (#4443)
* fix(sandbox): reconcile E2B sandboxes safely

* fix(sandbox): clear failed E2B adoption intent
2026-07-27 14:10:24 +08:00
March-77
2e5c8da257
fix(sandbox): bypass proxies for local AIO traffic (#4444)
* fix(sandbox): bypass proxies for local AIO traffic

* fix(sandbox): classify public IPv6 proxy targets
2026-07-27 07:47:39 +08:00
Huixin615
1cd5dea336
fix(streaming): signal replay history gaps (#4426)
* fix(streaming): signal replay history gaps

* fix(streaming): guard initial Redis replay window

* fix(frontend): align inactive gap recovery

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-27 07:13:06 +08:00
Vanzeren
1c7531242c
feat(runtime): record terminal artifact delivery receipts (slice 1 of #4272) (#4365)
* feat(runtime): record terminal artifact delivery receipts (#4272)

* fix(runtime): persist delivery receipts across recovery

* test(runtime): cover delivery receipt invariants

* fix(runtime): preserve terminal status on receipt outages
2026-07-26 21:45:47 +08:00
Aari
e646188ab4
fix(runtime): accept the SDK's default stream_resumable=false (#4468)
RunCreateRequest declared stream_resumable as None-only, but langgraph_sdk
defaults it to False (not None) and its payload filter only drops None, so
every SDK request carried "stream_resumable": false and was rejected with 422.
False means "no resumable stream", which is what DeerFlow already serves.

This broke every IM channel run (runs.stream and runs.create both send the
field; only runs.wait omits it) and any external langgraph_sdk client. The web
frontend was unaffected because its compatibility wrapper does not send it.

An explicit true still returns 422. The new regression test drives a real SDK
client to capture its default payload and posts that to the HTTP boundary, so a
future non-None SDK default cannot regress this silently.

Fixes #4466
2026-07-26 15:00:42 +08:00
Aari
37c343fe30
fix(summarization): summarize with the run model, fall back on summary-provider failure (#4361)
* fix(summarization): own the run model for compaction; bound failure

With summarization.model_name: null the summary model resolved to
config.models[0] while the executing model is selected per run; when they
differ and models[0]'s provider is broken (expired key, quota, outage)
compaction silently failed every triggered turn and context grew unbounded
until the main provider 400s the run (#3103's shape), even though the run's
own model was healthy.

Model ownership is now sourced from the builders, not re-derived at runtime:
- The lead, subagent, and manual /compact builders each pass the resolved run
  model into create_summarization_middleware(run_model_name=...). The middleware
  no longer reads runtime.context / get_config(), which do not carry a custom
  agent's or a subagent's resolved model, so a custom-agent lead run and a
  distinct-model subagent now summarize with their own model, not models[0] /
  the parent's. Runtime re-resolution and the per-name model cache are removed.
- model_name: null summarizes with the run's own model; an explicitly configured
  summary model generates and falls back to the run model on failure. The
  fallback is built lazily after the primary fails and its construction is
  guarded, so a broken fallback cannot skip a healthy primary or escape the
  automatic failure boundary.

Failure is bounded and side-effect-safe:
- An empty or whitespace-only response is treated as a generation failure, not a
  valid summary, so compaction never removes all history for an empty replacement.
- compact_state/acompact_state take raise_on_failure independent of force: the
  manual /compact path always surfaces a generation failure (even force=false)
  and routes it to the existing ContextCompactionFailed path (HTTP 500 ->
  frontend error toast) instead of an unconsumed response reason. The automatic
  path leaves compaction state unchanged.
- before_summarization hooks fire only after a replacement summary exists.

SummarizationConfig.model_name, config.example.yaml, and docs/summarization.md
document the final lead/subagent/manual ownership rules.

Part of RFC #4346 (section A). Evaluating fraction/triggers against the run
model's profile (profile ownership) is a separate follow-up.

* fix(summarization): manual /compact model ownership + fail-open construct/parse

Manual /compact carried only agent_name, so it derived the run model from the
custom-agent model or config.models[0] and missed the request-selected model the
run path uses (request -> custom-agent -> default). Carry model_name through
ThreadCompactRequest and the frontend compact call, resolve with the same
precedence, and move the custom-agent config read off the event loop (asyncio
.to_thread) with user_id so the strict blocking-IO gate is not bypassed by the
broad except.

Make one summary attempt own its full lifecycle so the fail-open boundary covers
construction and response parsing, not just invocation: build each candidate model
lazily and guarded (a raising constructor falls through to the healthy run model
instead of breaking agent construction), build the model_name:null primary from the
run model rather than config.models[0], and run response text extraction inside the
invocation try so a failing .text accessor falls back instead of escaping compaction.
Adds factory-level constructor-failure, response-extraction-failure (sync/async), and
route-path model-ownership tests.
2026-07-26 07:39:39 +08:00
ShitK
0f0955bf7b
fix(client): preserve ToolMessage artifacts (#4422)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-25 09:47:58 +08:00
Nan Gao
58befaf248
fix(thread-history): keep completed subtask cards stable after reload (#4432)
* fix(thread-history): hide subagent AI responses

* refactor(thread-runs): remove unused _is_middleware_message_row helper

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-25 09:29:11 +08:00
Daoyuan Li
d2b5f884e3
fix(channels): buffer GitHub follow-ups during busy runs (#4133)
Queue comments received while a run is active, then submit one deduplicated follow-up after it finishes. Failed drains are requeued and watcher tasks stop cleanly with the channel manager.
2026-07-24 22:41:07 +08:00
H Haidong
c7538cfb35
fix(runs): terminate orphaned streams after lease recovery (#4420)
* fix(runs): terminate orphaned streams after lease recovery

* fix(runs): include recovered ids in callback warnings

* fix(runs): harden orphan recovery lifecycle
2026-07-24 19:34:20 +08:00
ShitK
a4ede80deb
fix(runtime): reject unsupported run options and stream modes (#4430)
* fix(runtime): reject unsupported run options

* fix(runtime): align SDK run compatibility

* fix(frontend): avoid unsupported events stream mode

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-24 19:24:24 +08:00
Huixin615
4a2ecd430e
fix(streaming): expose custom events to astream_events (#4403)
* fix(streaming): expose custom events to astream_events

* test(streaming): validate real custom event emitters

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-23 22:56:12 +08:00
MiaoRuidx
f1632cc351
fix(run): add run event stream contract (#4342)
* docs: document run event stream contract

* fix(run): address event stream review feedback

---------

Co-authored-by: MiaoRuidx <12540796+MiaoRuidx@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-23 21:33:57 +08:00
Aari
0d4d0cb17d
feat(agents): database-backed storage for custom agent definitions (#4359)
* feat(agents): database-backed storage for custom agent definitions

Add an agent_storage.backend switch (default file, behaviour-unchanged) with a
db backend that stores each custom agent as a row in the shared SQL persistence
layer, so a multi-instance deployment sees the same agents on every node
(#4331, #4357). Introduces an AgentStore interface routing all read/write
surfaces, an agents table + migration 0006, startup validation, and a file->db
importer. Follows the thread_meta store / run_events backend-switch /
0003_scheduled_tasks migration patterns; no new dependency.

* fix(agents): make db storage path production-ready (review round 1)

Addresses review feedback on the db/sync agent-storage path:

- sql.py: mirror the async engine's per-connection SQLite PRAGMAs on the sync
  engine (busy_timeout=30000, synchronous=NORMAL, foreign_keys=ON, WAL) so both
  engines behave identically against the shared DB; guard the engine cache with
  a lock (double-checked) so concurrent first-touch cannot build duplicate
  engines or register the connect listener twice.
- routers/agents.py + routers/assistants_compat.py: offload the sync-store reads
  that ran on the event loop (list/get/check, update's pre-read + legacy guard +
  refresh, and assistants_compat's four list routes) via asyncio.to_thread — on
  db+postgres each was a network round trip stalling the loop. Writes were
  already offloaded.
- file.py: translate the create() mkdir(exist_ok=False) race FileExistsError
  into AgentExistsError (router 409, matching SqlAgentStore's IntegrityError
  path); correct the _write docstring — per-file atomic replace, two commits
  sequential not transactional.

Tests: sync-engine PRAGMA + engine-cache reuse assertions; file create-race ->
AgentExistsError; strict Blockbuster anchor over the read endpoints so a
regression back onto the loop fails CI.

* fix(agents): address round-2 review on the db store path

- update_agent tool: align the docstring/inline comment with FileAgentStore._write.
  Cross-field write atomicity is db-only; the file backend commits config then
  soul via two sequential os.replace (a crash between them can leave a fresh
  config.yaml beside a stale SOUL.md). The dropped partial-write *reporting* is
  an intentional tradeoff — the stage-then-replace safety is preserved
  (test_update_agent_soul_failure_does_not_replace_config still holds).
- SqlAgentStore.update(): true upsert. Catch IntegrityError on the
  insert-on-missing branch, re-fetch and apply, so two concurrent first-time
  writes (e.g. two setup_agent handshakes) converge instead of surfacing a raw
  UNIQUE(user_id, name) violation as a 500. Symmetric with create().
- get_agent_store(): document the graph-subprocess config-resolution invariant
  (the except->file fallback is a genuine no-config path, not a mask for a
  misconfigured graph process) and pin it with two tests driving the real
  get_app_config() file resolution: db resolves from an on-disk config.yaml,
  file fallback when config is unresolvable.

* test(agents): cover SqlAgentStore.update() write-race upsert recovery

Mandatory-TDD test for the round-2 fix in 0680340a: two concurrent first-time
update()s where the loser's insert hits UNIQUE(user_id, name). Deterministically
forces the IntegrityError recovery path by making the first _row probe miss the
committed winner, and asserts last-writer-wins instead of a surfaced 500.
2026-07-23 08:03:21 +08:00
Ryker_Feng
fa496c0c8d
feat(browser): add agentic browser control (#4187)
* feat(browser): add agentic browser control

* fix(frontend): format browser view changes

* fix(browser): keep browser optional and isolate sidecar layout

* fix(browser): address PR review security and IME findings

- Nginx: add a browser-stream WebSocket location before the generic
  /api/threads regex so Live upgrades instead of downgrading to HTTP
  (both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
  navigate, and tear down the browser session on thread deletion so a
  later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
  context-level route guard covering redirects, popups, iframes, and
  subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
  CJK candidate with Enter no longer submits the remote page form.

Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.

* fix(frontend): smooth streaming in long tool threads

* Revert "fix(frontend): smooth streaming in long tool threads"

This reverts commit f0462516eabe77f138d4027ea1c714fb226683cf.

* fix(browser): address review security and lifecycle findings

- Reject cross-origin WebSocket upgrades on the live browser stream
  (Origin allow-list reuse of CORS/same-origin helpers) to close a
  WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
  off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
  (browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
  and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
  can't permanently stall after the cumulative attempt cap.

* fix(frontend): reduce long tool thread render stalls

Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.

* fix(browser): keep live control responsive during continuous input

Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").

What:
- Frontend forwards one `click` per physical click instead of also emitting
  `down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
  start a rate-limited background refresh loop (leading frame + bounded cadence)
  that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
  inputs coalesce, and continuous input keeps refreshing before it stops.

Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.

* fix(browser): harden worker and session lifecycle

* fix(browser): address latest review feedback

* fix(frontend): preserve optimistic new-chat message

* test(e2e): preserve mocked message run ids

* fix(browser): address capability review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-21 11:46:33 +08:00
Ryker_Feng
24a45a4e68
feat(tui): add clear command (#4306) 2026-07-20 23:56:37 +08:00
Huixin615
c9b6131f8f
fix(skills): reload mounted skills without restarting Gateway (#4264)
* fix(skills): add admin-only reload endpoint

* fix(skills): preserve cache when reload fails
2026-07-17 23:22:16 +08:00
Huixin615
65afc9b1d2
fix(skills): apply allowed-tools only to active skills (#4098)
* fix(skills): scope allowed-tools to active skills

* fix(skills): tolerate stale active skill paths

* chore: retrigger CI

* fix(skills): document policy activation limits

* perf(skills): reuse per-step tool policy decisions

* fix(skills): harden runtime tool policy contracts

* fix(skills): redact cached policy decisions

* fix(skills): make slash tool policy authoritative

* fix(skills): preserve policy-safe discovery tools

* test(skills): cover explicit task delegation policy
2026-07-16 14:12:02 +08:00
yym36991
e7e9c51988
Classify four HTTP identity sources (browser session, OIDC, IM channel (#4179)
binding, Internal Auth) and group IM bindings with direct HTTP under a
shared platform-trust model. Explain Internal Auth trust boundaries:
DeerFlow validates only the platform token, does not manage end-user
accounts in users, and relies on the channel to maintain owner identity.
Cross-link AUTH_DESIGN, API.md, and docs README
2026-07-15 19:42:35 +08:00
yym36991
abd0c7a585
docs(api): document stateless /api/runs/stream endpoint (#4177)
* docs(api): document stateless /api/runs/stream endpoint

    Record that clients can start a conversation without pre-creating a thread,
    and that Gateway returns thread_id and run_id via the Content-Location header.

    Co-authored-by: yym36991@gmail.com

* docs(api): show on_run_created for stateless stream continuation

The Python SDK example now captures thread_id/run_id via on_run_created
before documenting the follow-up call with config.configurable.thread_id,
matching the TS/cURL examples and langgraph-sdk 0.3.x behavior.

Co-authored-by: yym36991@gmail.com
2026-07-14 22:33:57 +08:00
DanielWalnut
4e209827f3
feat(agent): Add subagent total delegation cap (#4115)
* fix subagent total delegation cap

* fix embedded subagent run cap context

* fix subagent cap config consistency

* fix resumed subagent run cap boundary

* fix legacy resume subagent boundary

* address subagent cap review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-13 19:26:07 +08:00
黄云龙
f85ee590e1
docs(backend): add IM channel connections and GitHub agents architecture diagrams (#4112)
Adds two focused architecture docs and references from backend/AGENTS.md:
- backend/docs/IM_CHANNEL_CONNECTIONS.md — sequence + graph diagrams for user-owned
  IM channel connections: bind-code flow (/connect/<start), single-active-owner
  transfer, provider message routing, owner-scoped file storage.
- backend/docs/GITHUB_AGENTS.md — sequence + graph diagrams for GitHub event-driven
  agents: webhook fan-out, preferred_thread_id = UUID5, GH token lifecycle, race recovery.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-12 11:42:55 +08:00
Ryker_Feng
ebc09ce130
feat(mcp): auto-promote deferred MCP tools from routing hints (#4019)
* feat(mcp): auto-promote deferred MCP tools from routing hints

When tool_search.enabled=true defers MCP tool schemas, PR1 routing hints
still require the model to spend a tool_search discovery round trip before
it can call the tool the routing metadata already points at. This adds a
McpRoutingMiddleware that matches the latest user message against PR1
routing keywords and promotes the matching deferred schemas before the
model call, removing that round trip.

Design (soft routing, opt-in, additive):
- Matches only the latest real HumanMessage (shared is_real_user_message
  helper, reused by SkillActivationMiddleware so the two cannot drift);
  case-insensitive substring match, no tokenizer dependency.
- Ordering: priority desc, then tool name asc; capped by the new global
  tool_search.auto_promote_top_k (default 3, clamped 1..5). Does not add or
  consume a per-tool auto_promote_top_k (PR1 schema unchanged); a per-tool
  value is ignored with a DEBUG note.
- Returns a plain {"promoted": ...} state update (not a Command) and relies
  on ThreadState.merge_promoted for union/dedupe, so auto-promote and a
  model-triggered tool_search converge on the same catalog hash.
- Installed before DeferredToolFilterMiddleware on every deferred-tool path
  (lead agent, subagent, embedded client, webhook via shared builders);
  a construction-time assert rejects the reversed order. catalog_hash is
  None / no routing index is a complete no-op, so bootstrap and ACP skip it.
- Privacy: never executes tools, never promotes policy-filtered tools, adds
  no routing keywords or matched tool names to trace metadata or INFO/WARN
  logs.

No behavior change when tool_search.enabled=false.

Tests: index construction, matching semantics, middleware state updates,
same-cycle deferred-filter interaction, lead/subagent/embedded-client
builder wiring + order invariant, config clamping, config.example.yaml
parseability, and privacy assertions.

* refactor(mcp): address auto-promote review nits

- executor: access app_config.tool_search.auto_promote_top_k directly to match
  the lead-agent and embedded-client paths (drop the over-defensive getattr that
  masked missing config); update the subagent test mock to carry tool_search.
- tool_search / mcp_routing_middleware: cross-reference the duplicated routing
  priority/keyword normalization between the builder and the middleware's
  defensive _normalize_index so they cannot silently drift.
- MCP_SERVER.md: document that auto-promote keyword matching is a case-insensitive
  substring test (not word-boundary), advising distinctive keywords.
2026-07-10 07:54:36 +08:00
Ryker_Feng
5ba25b06ec
feat(mcp): add MCP routing hints (#4004)
* feat: add MCP routing hints

* test: isolate mcp routing prompt config

* fix: address mcp routing review feedback
2026-07-09 16:26:31 +08:00
AochenShen99
658c39ccf7
feat(skills): Add native SkillScan phase 1 for skills (#3033)
* Add phase 1 skill static scanning

* Rework SkillScan phase 1 as native scanner

* refactor(skillscan): align phase 1 with trimmed RFC contract

- SecurityFinding: 7 fields (rule_id, severity, file, line, message,
  remediation, evidence); category/analyzer derive from the rule_id
  prefix, confidence/column/fingerprint/metadata removed
- scan_archive_preflight()/scan_skill_dir() are pure functions: no
  ScanContext, no policy schema; CRITICAL-blocks is a code constant and
  skill_scan.enabled is applied by enforce_static_scan()/callers
- secret-* evidence is redacted before findings leave the scanner
- de-dup keys on (rule_id, file, line) so repeated occurrences keep
  distinct locations for agent self-correction
- cloud-metadata detection consolidated into network-cloud-metadata
- nested zip members get a one-level stdlib magic-byte peek; an
  executable member escalates package-nested-archive to CRITICAL
- install metadata sidecar removed (Phase 7 decides if it is needed)
- rule specs moved next to their analyzers; skillscan/rules/ removed
- tests updated + new anchors: redaction, dedup lines, nested-zip
  escalation, single cloud-metadata rule, bundled-skill zero-CRITICAL

* fix(skillscan): tighten reverse-shell/secret/archive scan rules from review

Address PR #3033 review feedback on the native SkillScan analyzers:

- Reverse-shell false positives: split shell detection by signal strength
  (/dev/tcp/, nc -e stay CRITICAL; bash -i, mkfifo -> new HIGH
  shell-reverse-shell-heuristic, warn->LLM). The Python check is now
  AST-anchored on real socket.socket/os.dup2/subprocess call sites instead
  of raw-text substring matching, so prose/docstrings no longer hard-block.
- Secret evidence: _redact_secret_evidence returns [redacted] with no secret
  bytes (was value[:6], which leaked 2 real token bytes past the prefix).
- Archive DoS: cap outer archive member count (_MAX_ARCHIVE_MEMBERS=4096);
  scan_archive_preflight early-aborts with a package-too-many-members CRITICAL
  finding (routes through the existing blocked->400 fail-closed path).
- shell-destructive-command: broaden the rm -rf matcher to sensitive system
  roots (/home, /usr, /*, --no-preserve-root /) while leaving safe subpaths
  unflagged.
- Dead code: collapse _decode_text_for_analysis to a single decode path and
  drop the unused _TEXT_SUFFIXES set and _has_text_shebang helper.
- local_skill_storage: document why the host_path branch keeps app_config
  possibly-None (lazy kill-switch resolution; avoids eager get_app_config in
  config-free environments such as CI).

Tests: new negative/positive coverage in test_skillscan_native.py. Full
backend suite 6616 passed, 26 skipped.
2026-07-07 21:44:28 +08:00
sqsge
927b833ef4
fix(guardrails): propagate internal owner attribution to guardrail context (#3839)
* fix guardrail attribution for internal owner runs

* fix guardrail owner attribution mismatch
2026-07-07 08:05:26 +08:00
Vanzeren
67c8ade375
feat(sandbox): warm pool for boxlite (#3951)
* feat: add boxlite SDK as harness dependency

* test: add fake SimpleBox fixtures for BoxLite warm pool tests

* feat: add deterministic sandbox_id and warm pool fields to BoxliteProvider

* feat: pass deterministic sandbox_id to SimpleBox name

* feat: warm pool lifecycle — park on release, reclaim on acquire

- release(): parks VMs in _warm_pool with timestamp instead of closing
- _reclaim_warm_pool(): health checks warm boxes via echo ok
- acquire(): tries warm pool reclaim before creating new boxes
- Deterministic sandbox_id ensures thread isolation

* feat(boxlite): idle reaper, replica enforcement, warm-pool shutdown/reset

- Task 6: idle reaper daemon thread destroys expired warm-pool boxes
- Task 7: replica enforcement evicts oldest warm-pool box when at capacity
- Task 8: shutdown() stops idle checker first, destroys all boxes (active+warm);
  reset() clears warm pool

Tests: 17 passed (4 new: idle reaper, replica enforcement, shutdown, reset)

* fix(boxlite): harden warm pool lifecycle races

* docs(boxlite): document warm pool configuration

* fix(sandbox): log warning when evicting oldest warm box is failed

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(boxlite): stop provider lifecycle on reset

* fix(boxlite): make runtime an optional dependency

* test(boxlite): remove unused warm pool lookup

* docs(boxlite): clarify optional runtime support

* refactor: extract shared WarmPoolLifecycleMixin for sandbox warm-pool lifecycle

Introduce deerflow.community.warm_pool_lifecycle.WarmPoolLifecycleMixin
owning idle-checker loop, warm-pool expiry, oldest-warm eviction,
replica counting, and soft-cap logging. Move AioSandboxProvider and
BoxliteProvider onto the mixin; keep AIO active-idle cleanup local and
delegate only warm-pool expiry to the shared helper.

BoxliteProvider also gains:
- Prefixed box names (deer-flow-boxlite-*) for startup orphan reconciliation
- _reconcile_orphans() adopting surviving boxes from a prior process
- Pinned timeout forwarding: command timeout now bounds both BoxLite
  SDK exec(timeout=...) and the loop bridge .result(timeout)
- reset() reworked as lightweight registry clear (boxes -> warm pool,
  no close, no idle-reaper stop, no loop close) so reset_sandbox_provider()
  config switches are safe; shutdown() remains the teardown path

Backward-compatible: AIO DEFAULT_IDLE_TIMEOUT/DEFAULT_REPLICAS/
IDLE_CHECK_INTERVAL stay importable; Boxlite IDLE_CHECK_INTERVAL
stays monkeypatchable.

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-07-06 07:21:12 +08:00
Prax Lannister
f6a910dec9
fix(models): normalize api_base->base_url for ChatOpenAI + warn on unknown config keys (#3790)
ModelConfig is `extra="allow"`, so a config key like `api_base` (which
config.example.yaml uses for other model classes, e.g. PatchedChatDeepSeek and
Moonshot) gets copied onto a `langchain_openai:ChatOpenAI` model by users.
LangChain's OpenAI client does not reject the unknown kwarg — it transfers it
into `model_kwargs` (with a UserWarning), which is then spread into every
`Completions.create()` call and rejected by the OpenAI SDK at REQUEST time with
an opaque `unexpected keyword argument 'api_base'` error. The endpoint override
is also silently dropped, so the model targets the wrong base URL.

Changes in factory.py, mirroring the existing OpenAI-compatible helpers:

- `_normalize_openai_base_url`: renames `api_base` -> `base_url` for the
  OpenAI-compatible family (ChatOpenAI + PatchedChatOpenAI); when an endpoint
  key is already present, drops the alias with a warning. Runs before the
  stream_usage/stream_chunk_timeout heuristics so they see the canonical key.
- `_warn_unknown_model_settings`: scoped to the same OpenAI-compatible family
  (where the model_kwargs divert-and-crash actually happens and the field/alias
  set is accurate), logs an actionable warning for unrecognized config keys.
- The three OpenAI-compatible helpers now share `_OPENAI_COMPAT_USE_PATHS`
  instead of disagreeing on the literal class string.

Adds a note to docs/CONFIGURATION.md and 9 tests covering normalization
(ChatOpenAI + PatchedChatOpenAI, both-set precedence for base_url and
openai_api_base, non-OpenAI class untouched, no-op when unset) and the
unknown-key warning (fires on a typo, silent on a clean config and on
non-OpenAI providers like ChatAnthropic).
2026-07-05 10:17:25 +08:00
Xinmin Zeng
4fc08b4f15
feat: add scheduled tasks MVP (#3898)
* feat: add scheduled tasks MVP

* fix: harden scheduled task execution semantics

* feat(scheduled-tasks): preset-driven schedule form with timezone and live preview

Replace the raw cron input with a preset Select (hourly/daily/weekly/monthly/custom)
plus structured inputs (time picker, weekday toggles, day-of-month), datetime-local
for one-time tasks, a timezone selector defaulting to the browser timezone, and a
live human-readable preview. Reuses one ScheduledTaskScheduleInput for create and
edit; backend contract unchanged; zero new deps (pure Intl + DST-safe offset helpers).

* feat(scheduled-tasks): full-page i18n + recipe templates + E2E locale pin

Localize the rest of the scheduled-tasks page (filters, detail pane, actions,
edit form, run list, enum values) via t.scheduledTasks.* in en/zh. Add four
built-in recipe templates (GitHub Trending, news digest, issue triage, weekly
report) exposed as a chip row that pre-fills title + prompt + schedule. Pin
Playwright locale to en-US so E2E selectors stay stable against i18n. No backend
change, no new deps.

* fix(scheduled-tasks): idempotent 0003 migration, update head constants, future-date once test

Merge with main surfaced three CI failures:
- 0003_scheduled_tasks create_table collided with legacy test seeds that
  build from full metadata; guard with inspector.has_table so the revision
  no-ops when the table already exists (0004/0005 are already idempotent via
  _helpers.py).
- persistence bootstrap concurrency/regression tests pinned HEAD to main's
  0002_runs_token_usage; bump to the new head 0005_scheduled_task_thread_nullable.
- once-task router test used a fixed past run_at and tripped the
  must-be-in-the-future validation; use a future date.

* address review: ok-check, 502 for trigger failure, mock fields, migration filename, doc fences

- fetchThreadScheduledTasks now checks response.ok like the other fetchers.
- trigger endpoint returns 502 (not 409) when dispatch fails outright, so
  clients can distinguish a real conflict from a server-side failure.
- E2E mock normalizes scheduled-task objects with context_mode/last_thread_id
  and nullable thread_id, matching the backend contract the UI renders against.
- Rename 0002_scheduled_tasks.py -> 0003_scheduled_tasks.py to match its
  revision id (file was renamed in spirit already; filename now follows).
- CONFIGURATION.md: close the Tool Groups yaml fence and drop the stray fence
  after the Scheduler notes so the sections render correctly.

* fix(scheduled-tasks): harden lease, poller, config, and frontend UX after review

* fix(scheduled-tasks): harden run lifecycle, overlap skip, non_interactive gating, and DST conversion after review

- defer a once task's terminal status to the run-completion hook; the task
  stays running until the real outcome, and a startup sweep cancels once
  tasks orphaned by a crash (launch-time 'completed' could stick forever)
- record interrupted runs as a distinct 'interrupted' run status with a
  readable message; an interrupted once task ends 'cancelled', not 'failed'
- enforce overlap_policy=skip for fresh_thread_per_run via an active-run
  pre-check (same-thread ConflictError can never fire across fresh threads)
- protect terminal run statuses from the late launch-path 'running' write
- honor context.non_interactive only for internally-authenticated callers;
  arbitrary clients can no longer strip ask_clarification
- fix DST-stale timezone offset in zonedLocalToUtcIso by re-deriving the
  offset at the resolved instant (once tasks fired an hour late around
  spring-forward and the create->edit round-trip diverged)
- drop dead ScheduledTaskRunRepository.update_by_run_id; share one Gateway
  API error helper between channels and scheduled-tasks frontends

* fix(scheduled-tasks): close review round-3 gaps in guards, concurrency, and API ergonomics

- scrub internal-only context keys (non_interactive) from the assembled run
  config for non-internal callers: gating body.context alone left the same
  key smuggle-able through the free-form body.config copied verbatim by
  build_run_config
- guard update_after_launch with protect_terminal so the launch bookkeeping
  write cannot clobber a once task already finalized by a fast-failing run's
  completion hook (parent-row sibling of the run-row guard)
- reject a manual trigger while the task has an active run (409) instead of
  launching a duplicate concurrent run on fresh_thread_per_run
- re-arm a terminal once task to enabled when PATCH pushes run_at into the
  future; previously the endpoint returned 200 with a next_run_at that could
  never be claimed
- make max_concurrent_runs a real global cap: each poll claims only into the
  remaining budget of active (queued/running) scheduled runs
- paginate GET /scheduled-tasks/{id}/runs (limit<=200, offset) and push the
  thread filter of /threads/{id}/scheduled-tasks into SQL
- stamp context.user_id on scheduler-launched runs, matching IM channels, so
  user-scoped guardrail providers see the owning user

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-04 21:51:57 +08:00