40 Commits

Author SHA1 Message Date
RongfuShuiping
2aaf74b0f8
feat(memory): add OpenViking HTTP backend (#4509)
* feat(memory): add OpenViking HTTP backend

* fix(memory): harden OpenViking lifecycle

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-28 23:36:25 +08:00
hataa
964162747f
docs(harness): document subagent runaway guards + token_budget (#3875) (#4440)
Sync the harness docs with the #3875 subagent middleware changes:

- subagents.mdx (en/zh): fix built-in max_turns defaults (general-purpose
  160->150, bash 80->60) to match code and config.example.yaml; document the
  new  config (global + per-agent override) and add a
  'Runaway guards' section covering the LoopDetection / TokenBudget /
  Summarization middlewares now mirrored on the subagent chain.
- middlewares.mdx (en/zh): note that loop-detection, token-budget, and
  summarization guards are shared with the subagent chain (#3875), instead of
  implying they are Lead-Agent-only.

Code, contracts/subagent_status_contract.json (v2), and config.example.yaml
already reflect #3875; only the docs were lagging.
2026-07-24 22:36:00 +08:00
urzeye
cd86275638
fix(docs): repair broken documentation links (#4330)
* fix(docs): repair broken documentation links

* revert(docs): restore ACP protocol naming
2026-07-22 07:32:05 +08:00
qin-chenghan
ad45f59d66
feat(memory): pluggable memory abstraction with self-contained DeerMem backend (#4122)
* feat(memory): pluggable + self-contained memory system (MemoryManager plan phases 1 & 2)

Phase 1 — Pluggable (steps 0-10):
- ABC MemoryManager (9 methods) + singleton factory + drop-in backend discovery
- DeerMem default backend with core/ (storage/queue/updater/prompt/message_processing)
- NoopMemoryManager backend (proves pluggability)
- All call sites (middleware/hook/prompt/gateway/client/app) routed through manager
- hasattr capability probing for DeerMem-internal methods (no hard imports)
- MemoryConfig gains manager_class field; shared vs DeerMem-private annotated

Phase 2 — Self-contained DeerMem (steps 11-18):
- backend_config passthrough + DeerMemConfig (all DeerMem-private fields moved off MemoryConfig)
- DI: DeerMem owns storage/queue/updater/llm as instance attributes (no global singletons)
- Storage independence: core/paths.py with own root (~/.deermem or ),
  factory auto-injects deer-flow's runtime_home() as absolute base_dir (zero-config)
- LLM independence: core/llm.py via langchain init_chat_model (no create_chat_model)
- Trace independence: optional tracing_callback replaces inject_langfuse_metadata/request_trace_context
- Message processing independence: hide_from_ui default-skip + optional should_keep_hidden_message hook
- Internal imports → relative (only deer_mem.py ABC import is host-relative)
- Carrier (deer_mem.py adapter) / portable (deermem/ config+core) split
- New tests: test_deermem_self_contained + test_memory_manager_pluggable; all memory tests migrated
- Other-agent demo: samples/other_agent_demo/ + automated portability test
- config.example.yaml memory section updated to phase-2 schema

* feat(memory): port consolidation + staleness fix into self-contained DeerMem; phase-2 host hooks

Port upstream #3996 (memory consolidation) and #3993 (staleness KeyError fix)
from origin/MemoryManager into the pluggable, self-contained DeerMem structure
(backends/deermem/deermem/), adapted to the DI MemoryUpdater (config injected,
not get_memory_config globals):

- DeerMemConfig: add consolidation_enabled (opt-in, default false) /
  consolidation_min_facts / consolidation_max_groups_per_cycle /
  consolidation_max_sources
- prompt.py: factsToConsolidate JSON field + {consolidation_section} placeholder
  + CONSOLIDATION_PROMPT constant
- updater.py: _coerce_source_confidence / _select_consolidation_candidates /
  _build_consolidation_section module helpers (matching the existing
  _select_stale_candidates style); consolidation normalization in
  _normalize_memory_update_data; consolidation apply in _apply_updates (after
  max_facts trim, with apply-time guardrails mirroring staleness); staleness
  KeyError fix (f["id"] -> f.get("id") is not None) applied to both the
  staleness guardrail and the consolidation allowed_source_ids comprehension
- config.example.yaml: consolidation section under memory.backend_config
- tests/test_memory_consolidation.py: 40 DI-adapted tests (running, not skipped)
  incl. the staleness KeyError regression

Also includes in-flight phase-2 host-integration work: storage_path semantics
(any absolute/relative value = root dir) and host-default tracing_callback /
should_keep_hidden_message hooks injected into backend_config by the factory.

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(memory): add noop backend template and backends guide

- backends/noop/: complete drop-in template (config.py with zero deer-flow
  imports, noop_manager.py with a 6-step new-backend walkthrough in its
  docstring, commented optional fact-CRUD capabilities).
- backends/README.md: which files to touch when adding/swapping a backend,
  the 5-item backend contract, and common pitfalls.
- manager.py: generalize backend examples in comments (drop mem0-specific
  references).

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(frontend): guard formatTimeAgo against invalid timestamps

Return a neutral placeholder when the input date is invalid (e.g. an empty lastUpdated from a backend with no memories) instead of throwing 'Invalid time value' from date-fns.

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(memory): wire tool-driven memory mode through the MemoryManager ABC

tools.py (memory_search/add/update/delete) now calls get_memory_manager()
instead of the removed host memory module, so tool mode (memory.mode: tool)
works for any backend. DeerMem.search is implemented (case-insensitive
substring match, ranked by confidence) as a stand-in for the planned
semantic retrieval; noop.search returns [] (unchanged). Fact-CRUD tools
use getattr+callable probing -- backends lacking those ops (noop) get a
clear JSON error instead of crashing.

Tests: test_memory_tools rewired to mock the manager (handler tests) +
TestModeGating retained; test_memory_search now covers DeerMem.search;
pluggable stubs test updated (search no longer a stub).

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: resolve lint errors (import sorting, type annotation quotes, E402 in skipped tests)

* docs: restore explanatory comments in config.example.yaml memory section

* fix(security): port html-escape memory facts fix (#4097) to vendored DeerMem prompt.py

* fix(memory): address review + port dropped upstream memory fixes

Review blockers (vendored DeerMem):
- #4044 restore _escape_memory_for_prompt (current_memory blob in
  MEMORY_UPDATE_PROMPT) - prevents </current_memory> breakout
- #4028 html.escape staleness-section cat/content in _build_staleness_section
- #4119 add _escape_summary for injection-path summaries (Work/Personal/
  Current Focus/Recent/Earlier/Background)
- default-model silent no-op: factory injects host default chat model via a
  new host_llm slot (create_chat_model(name=None)); DeerMem prefers host_llm
  over build_llm(model). Zero-config extraction works out of the box again
- MemoryConfigResponse: fix stale docstring (backend-agnostic shape; DeerMem
  knobs live under backend_config, not top-level - restoring flat would
  re-couple the API to DeerMem). Frontend audited: does not read /memory/config
- _host_default_tracing_callback: restore langfuse assistant_id/environment
- search: push category onto the ABC signature; DeerMem filters BEFORE the
  top_k slice (was filtered client-side after slicing -> starved results)
- _do_update_memory_sync: split into wrapper+impl; bind trace_id into the
  request-trace ContextVar on the Timer/executor worker via a new
  trace_context_manager host hook (None trace_id left unbound - no fabrication)
- client.py fact-CRUD now passes user_id (was writing to the global bucket
  while get_memory reads per-user)
- _resolve_manager_class: fail-fast (raise ValueError) on an unresolved
  explicit manager_class instead of silently falling back to DeerMem (memory is
  persistent state - a wrong store is a silent data-integrity footgun)

Upstream memory fixes dropped by the host->vendored rename conflict, re-ported
to backends/deermem/deermem/core/ (+ deer_mem.py):
- #4073 queue busy-timer-spin -> _reprocess_pending flag (core/queue.py)
- #4074 null source.confidence in staleness -> _coerce_source_confidence
  (core/updater.py: _build_staleness_section + _apply_updates stale sort)
- #4075 factsToRemove is optional (drop from _REQUIRED_MEMORY_UPDATE_TOP_LEVEL_KEYS)
- #4076 null confidence in search ranking -> _coerce_source_confidence
  (deer_mem.py DeerMem.search)

host_llm + trace_context_manager are host-injected via backend_config (factory
in manager.py), keeping backends/deermem/ at exactly one `from deerflow` line
(the ABC contract) - portability test preserved.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: resolve lint errors (F541 f-string without placeholders, E501 line too long)

* fix(memory): restore hide_from_ui clarification preservation, expose mode

Two memory-system fixes (F541/E501 lint was already fixed on this branch):

- filter_messages_for_memory: restore default preservation of well-formed
  human_input_response clarification answers (v2 regression). The
  self-containment refactor made the bare function skip ALL hide_from_ui when
  no hook was passed, but upstream preserves well-formed clarification
  responses by default (test_hide_from_ui_human_input_response_is_preserved).
  Inline a host-agnostic _is_human_clarification_response mirror of
  read_human_input_response as the default keep-decision; the host-injected
  should_keep_hidden_message hook still overrides (production path unchanged).
  Portable package stays zero `from deerflow`.

- /memory/config: expose `mode` (middleware|tool) in MemoryConfigResponse +
  the config/status endpoints + client.get_memory_config. mode is a host-
  shared, behavior-determining field missing from the response projection.
  Sync tests (mock .mode; e2e assert mode present).

- Align manager_class field docstring with fail-fast behavior.

Tests: filter/self-contained/portability (35) + memory-config (4) pass;
ruff clean.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): resolve ruff format failures in memory module + tests

`make lint` runs `ruff format --check` in addition to `ruff check`; 8 memory
files had pending format changes -- 7 pre-existing (deer_mem, updater, tools,
test_memory_queue/router/search/tools) + message_processing from the
hide_from_ui fix. Apply `ruff format`: whitespace/wrapping only, no logic
change. 109 memory tests pass; ruff check + format --check both clean.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): address PR review - legacy field migration, fact_id contract, path/docs

Address willem-bd's review on PR head bc8bf0d4 (risk:high, persistent state):

- config: auto-migrate pre-abstraction top-level memory.* DeerMem fields
  (storage_path, max_facts, debounce_seconds, model_name, token_counting,
  staleness_*, consolidation_*) into backend_config on load + warn, so an
  upgrade does NOT silently revert customized settings (was: silent
  extra='ignore' drop). model_name -> backend_config.model.model. Unknown
  top-level keys warned.
- factory: resolve a relative backend_config.storage_path against runtime_home()
  (base_dir-relative, CWD-independent) to preserve pre-abstraction semantics;
  paths.py stays portable (no runtime_home import).
- tools: memory_add uses the fact_id returned directly by create_fact instead of
  re-deriving it via content-key matching (coupled the tool to the backend's
  content normalization; could misreport a storage cap). create_fact now returns
  (memory_data, fact_id); gateway/client/tool updated. Fix terse
  {"error":"content"} -> {"error":"empty content"}.
- app.py: update stale token_counting=="char" warm-up comment to point at
  manager.warm (DeerMem.warm re-checks char and returns early).
- router: comment explaining reload_memory silent fallback vs fact 501 asymmetry
  (read-only degrade vs write fail-loud).
- CHANGELOG: document breaking changes (/memory/config + client.get_memory_config
  shape flat->backend_config; custom storage_class path moved + __init__ must
  accept config) and the legacy-field auto-migration.
- tests: add regression test pinning the per-user memory path
  ({storage_path}/users/{safe_user_id}/memory.json == host make_safe_user_id)
  across the abstraction; update create_fact mocks for (memory_data, fact_id).

Tests: 273 passed (memory suite); ruff check + format clean.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): address PR review - storage_path, max_facts, tracing, parsing

Six review findings (willem-bd), each verified against upstream:

- storage_path semantics (file -> root dir): migration drops file-style
  (.json) legacy values with a warning; factory raises if storage_path
  resolves to an existing file (avoid silent NotADirectoryError write
  failure). CHANGELOG + config.example.yaml comment updated.
- create_memory_fact enforces max_facts again (via _trim_facts_to_max) and
  returns (memory, None) when the cap evicts the new fact; memory_add tool
  reports "not stored", client raises ValueError, POST /memory/facts -> 409.
- max_facts trim uses _coerce_source_confidence (was raw f.get("confidence",
  0) -> TypeError on non-float imported/legacy confidence, swallowed as
  silent update failure).
- memory-tracing assistant_id restored to "memory_agent" (was "lead-agent"
  copy-paste; matches upstream + DeerMem run_name).
- _is_human_clarification_response cross-checked against
  read_human_input_response (drift guard test).
- empty-string legacy values skipped silently in migration (narrow fix, not
  broad "if not value" which would skip explicit bool False).

8 new regression tests. make lint + 406 memory tests pass.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): address internal review - storage fail-fast, build_llm degrade, config warn, noop template

Addresses 4 findings from the PR #4122 internal supplemental review
(parallel to willem-bd's review, no overlap):

- create_storage fail-fast: a misspelled/unimportable storage_class now
  raises ValueError instead of silently falling back to FileMemoryStorage.
  Memory is persistent state, so a wrong store is a data-integrity footgun;
  mirrors the existing manager_class resolution policy. (storage.py)

- noop template create_fact signature: the commented template used
  keyword-only `content` and returned a bare dict, while DeerMem's actual
  create_fact takes positional `content` and returns tuple[dict, str|None]
  (the memory_add tool passes content positionally; gateway/client/tools all
  tuple-unpack). A backend copied from the template would 500 on fact-CRUD.
  Template fixed; delete_fact/update_fact templates left (callers compatible).
  (noop_manager.py)

- build_llm graceful degrade: wrap init_chat_model in try/except, degrade to
  None + WARNING on failure (mirroring _host_default_llm) so a misconfigured
  explicit model does not crash app startup -- non-LLM memory ops still work
  and an update raises at runtime with the error logged. (llm.py)

- from_backend_config unknown-key warning: log a WARNING for unknown
  backend_config keys (mirrors the host layer's load_memory_config_from_dict)
  so a typo like `storage_pat` does not silently fall back to the default and
  write memory to an unintended location. (config.py)

Tests: rewrote 3 create_storage fallback tests to expect ValueError; added 4
tests (build_llm zero-config/degrade, from_backend_config warn/silent).
make lint green; full memory suite passes.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: lllyfff <2281215061@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: lllyfff <122260771+lllyfff@users.noreply.github.com>
2026-07-15 11:21:04 +08:00
DanielWalnut
4e209827f3
feat(agent): Add subagent total delegation cap (#4115)
* fix subagent total delegation cap

* fix embedded subagent run cap context

* fix subagent cap config consistency

* fix resumed subagent run cap boundary

* fix legacy resume subagent boundary

* address subagent cap review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-13 19:26:07 +08:00
Xinmin Zeng
95a35f3725
docs(deployment-guide): fix stale userdata PVC subPath (#4139) 2026-07-13 19:08:13 +08:00
Vanzeren
c2002d9fac
feat(memory): add memory tool sets (#4023)
* feat: add memory-as-tool mode alongside existing middleware mode

- Add memory.mode config field (middleware|tool, default middleware)
- Add search_memory_facts() for case-insensitive fact lookup
- Add 4 memory tools: memory_search, memory_add, memory_update, memory_delete
- Wire mode gating in factory.py and lead_agent/agent.py
- 256 memory tests passing, zero regressions

* fix: harden tool-mode memory scoping and docs

* fix: address memory tool mode review feedback

* fix(memory): address tool mode review feedback

* fix: update config_version to 22 in values.yaml
2026-07-11 15:28:16 +08:00
Vanzeren
6060d95ee0
fix(wizard): update DeepSeek provider models to v4 (#3939)
Update DeepSeek references from deprecated model names to the V4 lineup:
- deepseek-reasoner → deepseek-v4-pro
- deepseek-chat → deepseek-v4-flash

Keep docs and frontend mocks aligned with the wizard provider list.
2026-07-04 21:44:22 +08:00
AochenShen99
66b9e7f212
feat: emit structured runtime metadata (follow-up#3887) (#3906)
* feat: emit structured runtime metadata

* fix: avoid subagent import cycle in replay gateway

* fix: preserve legacy subtask result parsing

* refactor: tighten runtime metadata contracts

* fix(middleware): keep recovery hint on task exception wrapper content

The structured-metadata stamp overwrote the wrapper text with the bare
task-failure message, dropping the model-facing 'Continue with available
context, or choose an alternative tool.' guidance that every other tool
exception keeps. Append the shared hint after the formatted message.

* fix(subagents): require lowercase hex for result_sha256 reader

Length-only validation accepted any 64-char string; a faulty serializer
or relaying wrapper could store a non-digest value in the delegation
ledger. Enforce the producer's hexdigest shape with a fullmatch.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-04 11:27:19 +08:00
ly-wang19
ddca86411b
docs: add production database guidance (#3830)
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
2026-07-03 15:41:49 +08:00
ly-wang19
c05dd46e25
docs: clarify docker sandbox mount paths (#3833)
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
2026-07-03 10:55:20 +08:00
Ryker_Feng
ddb097a72f
feat(community): add Brave image search community tool (#3866)
* Add Brave image search community tool

* fix(community): length-cap Brave web_search queries

Apply _clean_query in web_search_tool so over-long queries are trimmed
to Brave's 400-char limit before the API call, matching image_search_tool
and avoiding HTTP 422 from the Brave Search API.

* fix(community): harden Brave image search SSRF guard and dimension mapping

Address PR review findings:
- Catch ValueError from urlparse so a malformed bracketed-IPv6 URL skips
  one item instead of crashing the whole image_search call
- Reject IPv6 literals embedding a non-global IPv4 (IPv4-mapped, 6to4,
  NAT64, IPv4-compatible), closing the loopback/private SSRF bypass
- Report width/height from the dict of the URL actually returned, so a
  surviving thumbnail no longer reports the dropped original's dimensions

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-02 11:00:49 +08:00
Ryker_Feng
a8f950feb6
feat(community): add Browserless web_capture screenshot tool (#3881)
* feat(community): add Browserless web_capture screenshot tool

Add a web_capture tool that renders a page via Browserless /screenshot and
presents it through the artifact system, alongside the existing Browserless
web_fetch provider.

Hardening:
- SSRF guard: reject URLs resolving to private/loopback/link-local (incl. the
  169.254.169.254 cloud-metadata endpoint)/reserved/multicast/unspecified
  addresses; opt out via allow_private_addresses for internal targets.
- Surface a warning when Browserless renders a target page that itself
  responded with a non-2xx/3xx status (X-Response-Code), so an error/anti-bot
  page is not mistaken for valid visual evidence.
- Dedupe colliding output filenames instead of silently overwriting prior
  captures.

Docs: comment out token: $BROWSERLESS_TOKEN in tool examples (an unset $VAR
fails AppConfig startup) and document allow_private_addresses.

* fix(community): format web_capture guard + document local Browserless startup

Address PR #3881 review: fix the lint-backend failure (ruff format on
browserless/tools.py) and add local Browserless startup instructions to
CONFIGURATION.md so reviewers can run the service to try web_fetch/web_capture.
2026-07-01 23:41:58 +08:00
AochenShen99
442248dd06
feat: preserve durable context across summarization (#3887)
* feat: preserve durable context across summarization

* fix: harden durable context review gaps

* style: format delegation ledger live test

* chore: remove stale delegation ledger prefix

* fix: address durable context review feedback
2026-07-01 22:49:17 +08:00
chengjoey
e5424cbab9
feat(sandbox): add E2B sandbox provider (#3883)
Adds ``deerflow.community.e2b_sandbox.E2BSandboxProvider`` with parity
to AioSandboxProvider: metadata-keyed per-thread persistence, server-
side idle timeout, warm-pool reclaim with liveness checks, /mnt/user-data
bootstrap symlinks, dead-sandbox auto-rebuild, and release-time mirror
of agent outputs back to the host artifact directory.

Signed-off-by: joey <zchengjoey@gmail.com>
2026-07-01 11:15:27 +08:00
xiawiie
2453718acd
fix(title): avoid default LLM call before stream end (#3885)
* fix(title): avoid default LLM call before stream end

* fix(title): keep default fallback local

* fix(title): harden fallback and replay e2e

* docs(title): align fallback title behavior
2026-06-30 18:58:02 +08:00
Huixin615
0ee35ca38f
fix: preserve locale in documentation links (#3717) 2026-06-23 07:53:53 +08:00
kapil971390
d050fba07f
fix(token-budget): acquire _lock in _apply, before_agent, and _drain_pending_warnings (#3714)
_lock was defined in __init__ and used only in reset(), leaving _apply(),
before_agent(), _clear_run_state(), and _drain_pending_warnings() unprotected.
Concurrent after_model callbacks for the same run_id could race on
usage_accum.input += diff_input, causing lost-update on token counters.

Also fix stray bare 'a' character in frontend/src/content/zh/introduction/index.mdx
that rendered as visible text on the documentation page.

Co-authored-by: kapilvus <kapilvus@gmail.com>
2026-06-22 20:44:07 +08:00
Shatlyk
e418d72915
feat: add introduction documentation pages in English and Chinese and update application path references (#3690) 2026-06-22 08:24:44 +08:00
jp0xz
a6dd2876f0
feat(community): add GroundRoute web search + fetch engine (#3675)
* feat(groundroute): add GroundRoute community web_search + web_fetch tools

GroundRoute is a meta search layer over six engines (Serper, Brave, Exa,
Tavily, Firecrawl, Perplexity) with price-based routing and failover. This
adds a self-contained community engine module (httpx only, no new required
deps) mirroring community/brave + community/tavily:
- web_search: POST /v1/search, normalize to {title,url,snippet,source_engine}.
- web_fetch: fetch a URL via mode=page.
- unit tests covering normalization, auth, clamping, and graceful errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(groundroute): register GroundRoute search + fetch in wizard and config

Add GroundRoute to the setup wizard provider lists (SEARCH_PROVIDERS +
WEB_FETCH_PROVIDERS) and as commented web_search + web_fetch examples in
config.example.yaml, mirroring tavily/serper/brave so SEARCH_API can select it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(groundroute): apply repo ruff format (line-length 240)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(groundroute): add GroundRoute to tools docs and config reference

Adds GroundRoute as a web_search and web_fetch option in the en + zh
tools.mdx pages (new tab alongside Tavily/Brave/Exa/etc.) and documents
GROUNDROUTE_API_KEY in CONFIGURATION.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(groundroute): define empty groundroute extra for clean install

The docs install line 'uv add deerflow-harness[groundroute]' (mirroring the
tavily/exa/firecrawl pattern) referenced an undefined extra, which uv accepts
but warns about. GroundRoute needs no extra packages (httpx is a core dep), so
declare an empty 'groundroute' extra in deerflow-harness optional-dependencies
so the documented command resolves without a warning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(groundroute): per-tool api key + honor caller max_results (review)

Address maintainer review on PR #3675:
- _get_api_key(tool_name): web_fetch now reads the web_fetch config block's key
  instead of always web_search, so a flow that pairs GroundRoute fetch with a
  different search engine authenticates correctly. Mirrors serper/exa/firecrawl.
- web_search honors a caller-supplied max_results (sentinel default None),
  falling back to the configured value only when omitted, so the documented
  parameter is no longer silently discarded.
- warn-once is now keyed per tool. Tests cover both fixes (web_fetch key,
  agent max_results honored).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-06-21 15:55:10 +08:00
Ryker_Feng
0bbbbc06f4
feat(community): add Serper Google Images provider for image_search (#3575)
* feat(community): add Serper Google Images provider for image_search

Add a Serper-backed `image_search` tool alongside the existing Serper
`web_search` provider, so users with a SERPER_API_KEY can pull Google
Images results as reference images for downstream image generation.

- Share request/response handling between web_search and image_search
  via `_serper_post` / `_response_items`, with bounded `max_results`
  (capped at 10) and query normalization.
- Add a best-effort SSRF guard (`_safe_public_url`) that rejects
  non-http(s), localhost and private/non-global IP image URLs; filtered
  entries are dropped and never consume the result limit.
- doctor: flag literal `api_key` values in config as a warning and steer
  users toward `.env` + `$SERPER_API_KEY`.
- Docs/config: document the Serper image_search provider and SERPER_API_KEY,
  and discourage committing literal keys to config.yaml.
- Tests: cover the provider end-to-end (100% line coverage on tools.py)
  and the doctor literal-key warning path.

* fix(community): block obfuscated IPv4 literals in Serper image SSRF guard

The image_search SSRF guard only rejected dotted-decimal IP literals; encoded
forms such as decimal (http://2130706433/), hex (0x7f000001) and octal
(0177.0.0.1) raised ValueError in ip_address() and were allowed through, even
though many HTTP clients resolve them to private addresses like 127.0.0.1.

Add _decode_ipv4() to permissively decode these inet_aton-style encodings and
apply the same is_global check; hostnames that do not decode to an IP (e.g.
cafe.com) are still treated as hosts and left to fetch-time re-validation.

Addresses PR review feedback. Tests cover decimal/hex/octal loopback and
private encodings plus non-IP edge cases; tools.py stays at 100% line coverage.

* test(community): cover IPv4-mapped IPv6 URL filtering

* fix(community): address Serper image search review feedback

- Block trailing-dot hostname SSRF bypass (localhost./127.0.0.1.) in
  _safe_public_url by stripping the FQDN root label before checks.
- Keep a filtered image/thumbnail URL empty instead of collapsing onto
  its counterpart, preserving the high-res/preview contract.
- Evaluate the SSRF guard once per field rather than twice.
- Treat a null-typed organic/images field as "no results" rather than a
  malformed payload.
- doctor.py: when a config $VAR is unset, fall through to the default env
  var before reporting it as not set.
2026-06-18 07:36:35 +08:00
Ryker_Feng
6e839342a7
feat(community): add Brave Search web search tool (#3528)
* feat(community): add Brave Search web search tool

Add a community web_search provider backed by the official Brave Search
API (https://api.search.brave.com/res/v1/web/search). API key is read
from the tool config (inline api_key) or the BRAVE_SEARCH_API_KEY env
var. Output schema (title/url/content) matches existing search tools.
No new dependencies (uses the existing httpx). Also wires up the setup
wizard, doctor health check, config example, and EN/ZH docs.

* refactor(community): drop redundant [:count] slice in Brave search

The Brave API already caps results via the `count` request param, so
client-side slicing was redundant. Tests now simulate the API honoring
`count` instead of relying on the slice. Addresses PR review nit.

* style(tests): apply ruff format to test_doctor.py

Collapse multiline write_text calls onto single lines to satisfy the
CI ruff formatter (lint-backend was failing on format --check).
2026-06-13 22:47:35 +08:00
AochenShen99
0fb18e368c
refactor(lead-agent): make build_middlewares public to drop the last cross-module private import (#3458)
`client.py` imported the private `_build_middlewares` from `agent.py` across a
module boundary and called it as public API. Because the `_` name signals
"module-private, no external callers", any future rename or signature change
silently breaks the embedded `DeerFlowClient` path — and the test suite even
monkeypatched `deerflow.client._build_middlewares`, baking the leak in.

`DeerFlowClient` is a lead-agent variant that genuinely needs the lead agent's
full middleware composition, so make the dependency honest: promote the helper
to a documented public entry point `build_middlewares` and update every in-repo
caller. Found during #3341 review; #3341 already removed one such leak
(`_assemble_deferred` -> public `assemble_deferred_tools`) and left this one out
of scope on purpose.

- agent.py: rename def + both internal call sites; expand the docstring into a
  public-entry-point contract and document the previously-undocumented
  model_name / app_config / deferred_setup params
- client.py: import + call site now use the public name (removes the last
  cross-module private import)
- scripts/tool-error-degradation-detection.sh: update its import + call site
- tests (5 files): update monkeypatch/patch targets and direct calls
- docs (backend/CLAUDE.md, plan_mode_usage.md, middlewares.mdx): sync the live
  references that describe the symbol as current API

Pure mechanical rename, no behavior change. Historical design docs (rfc,
superpowers spec) intentionally keep the old name as point-in-time records.

Closes #3431
2026-06-09 11:56:28 +08:00
Nan Gao
a64a39dbc0
config: raise default summarization trigger before v2.0-m1 (#3174)
* config: update summarization configuration

* docs: sync summarization trigger guidance
2026-05-23 15:38:25 +08:00
JeffJiang
b103d1a7f5
feat(frontend): support static website demo mode (#3170)
* feat(frontend): support static website demo mode

* fix(frontend): render html artifact previews from blob content

* chore(frontend): apply pre-commit formatting

* fix(frontend): address static demo PR review comments

* Update the release information of DeerFlow

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-05-23 00:10:56 +08:00
Nan Gao
914d6a4f1c
docs: add provider safety termination post (#3167) 2026-05-22 21:33:15 +08:00
Nan Gao
253542ea0d
docs: discourage MCP filesystem workspace config (#3141) 2026-05-22 09:19:23 +08:00
Nan Gao
dcc6f1e678
feat(loop-detection): defer warning injection (#2752)
* fix(loop-detection): defer warn injection to wrap_model_call

The warn branch in LoopDetectionMiddleware injected a HumanMessage
into state from after_model. The tools node had not yet produced
ToolMessage responses to the previous AIMessage(tool_calls=...), so
the new HumanMessage landed *between* the assistant's tool_calls and
their responses. OpenAI/Moonshot reject the next request with
"tool_call_ids did not have response messages" because their
validators require tool_calls to be followed immediately by tool
messages.

Detection now runs in after_model as before, but only enqueues the
warning into a per-thread list. Injection happens in wrap_model_call,
where every prior ToolMessage is already present in request.messages.
The warning is appended at the end as HumanMessage(name="loop_warning")
— pairing intact, AIMessage semantics untouched, no SystemMessage
issues for Anthropic.

Closes #2029, addresses #2255 #2293 #2304 #2511.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(channels): remove loop warning display filter

* feat(loop-detection): scope pending warnings by run

* docs(loop-detection): update docs

* test(loop-detection): assert deferred warnings are queued

* fix(loop-detection): cap transient warning state

* docs: update docs

* add async awrap_model_call test coverage

* docs(loop-detection): document transient warnings

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 14:36:07 +08:00
Eilen Shin
84f88b6610
docs: align runtime docs with gateway mode (#2868)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-05-12 16:19:21 +08:00
YuJitang
e82b2fb4d0
docs: clarify token usage accounting semantics (#2845) 2026-05-11 07:17:49 +08:00
YuJitang
5127f08e1a
enable token usage by default (#2841) 2026-05-10 22:00:57 +08:00
yangzheli
59c4a3f0a4
feat(agent): add custom-agent self-updates with user isolation (#2713)
* feat(agent): add update_agent tool for in-chat custom-agent self-updates (#2616)

Custom agents had no built-in way to persist updates to their own SOUL.md /
config.yaml from a normal chat — `setup_agent` was only bound during the
bootstrap flow, so when the user asked the agent to refine its description
or personality, the agent would shell out via bash/write_file and the edits
landed in a temporary sandbox/tool workspace instead of
`{base_dir}/agents/{agent_name}/`.

Changes:
- New `update_agent` builtin tool with partial-update semantics (only the
  fields you pass are written) and atomic temp-file + os.replace writes so
  a failed update never corrupts existing SOUL.md / config.yaml.
- Lead agent now binds `update_agent` in the non-bootstrap path whenever
  `agent_name` is set in the runtime context. Default agent (no
  agent_name) and bootstrap flow are unchanged.
- New `<self_update>` system-prompt section is injected for custom agents,
  instructing them to use `update_agent` — and explicitly NOT bash /
  write_file — to persist self-updates.
- Tests: 11 new cases in `tests/test_update_agent_tool.py` covering
  validation (missing/invalid agent_name, unknown agent, no fields),
  partial updates (soul-only, description-only, skills=[] vs omitted),
  no-op detection, atomic-write safety, and AgentConfig round-tripping;
  plus 2 new cases in `tests/test_lead_agent_prompt.py` covering the
  self-update prompt section.
- Docs: updated backend/CLAUDE.md builtin tools list and tools.mdx
  (en/zh) with the new tool description.

* feat(agent): isolate custom agents per user

Store custom agent definitions under the effective user, keep legacy agents readable until migration, and cover API/tool/migration behavior with tests.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: consistent write/delete targets & add --user-id to migration

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-05 23:17:42 +08:00
Willem Jiang
3f88045b98 try to fix the frontend e2e test errors 2026-04-26 15:48:57 +08:00
Willem Jiang
28381e1383 fix the lint errors in frontend 2026-04-26 15:11:22 +08:00
JeffJiang
44d9953e2e feat: Add metadata and descriptions to various documentation pages in Chinese
- Added titles and descriptions to workspace usage, configuration, customization, design principles, installation, integration guide, lead agent, MCP integration, memory system, middleware, quick start, sandbox, skills, subagents, and tools documentation.
- Removed outdated API/Gateway reference and concepts glossary pages.
- Updated configuration reference to reflect current structure and removed unnecessary sections.
- Introduced new model provider documentation for Ark and updated the index page for model providers.
- Enhanced tutorials with titles and descriptions for better clarity and navigation.
2026-04-26 11:09:55 +08:00
copilot-swe-agent[bot]
88f822a8b3 docs: fill all TBD documentation pages and add new harness module pages
Agent-Logs-Url: https://github.com/bytedance/deer-flow/sessions/ff389ed8-31c9-430c-85ff-cc1b52b8239c

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-04-26 11:08:49 +08:00
copilot-swe-agent[bot]
814a488bcb docs: complete all English and Chinese documentation pages
Agent-Logs-Url: https://github.com/bytedance/deer-flow/sessions/a5f192e7-8034-4e46-af22-60b90ee27d40

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-04-26 11:08:49 +08:00
copilot-swe-agent[bot]
716cae20c6 docs: fix review feedback - source-map paths, memory API routes, supports_thinking, checkpointer callout
Agent-Logs-Url: https://github.com/bytedance/deer-flow/sessions/fb75dc8c-18a4-4a23-9229-25b3c5e545cf

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-04-26 11:08:49 +08:00
JeffJiang
7dc0c7d01f
feat(blog): implement blog structure with post listing, tagging, and layout enhancements (#1962)
* feat(blog): implement blog structure with post listing and tagging functionality

* feat(blog): enhance blog layout and post metadata display with new components

* fix(blog): address PR #1962 review feedback and fix lint issues (#14)

* fix: format

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-04-10 20:24:52 +08:00
JeffJiang
c1366cf559
Add documents site (#1767)
* feat: add docs site

- Implemented dynamic routing for MDX documentation pages with language support.
- Created layout components for documentation with a header and footer.
- Added metadata for various documentation sections in English and Chinese.
- Developed initial content for the DeerFlow App and Harness documentation.
- Introduced i18n hooks and translations for English and Chinese languages.
- Enhanced header component to include navigation links for documentation and blog.
- Established a structure for tutorials and reference materials.
- Created a new translations file to manage locale-specific strings.

* feat: enhance documentation structure and content for application and harness sections

* feat: update .gitignore to include .playwright-mcp and remove obsolete Playwright YAML file

* fix(docs): correct punctuation and formatting in documentation files

* feat(docs): remove outdated index.mdx file from documentation

* fix(docs): update documentation links and improve Chinese description in index.mdx

* fix(docs): update title in Chinese for meta information in _meta.ts
2026-04-03 07:25:40 +08:00