mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-11-05 10:10:36 +00:00
100 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
leom1919
|
58fc5bd5b8 |
docs: replace dead pull/5224 links with squash-merge commit permalinks in subagents reference (en/zh) (#7849)
* docs: replace dead pull/5224 links with squash-merge commit permalinks in subagents reference (en/zh) The "Related changes" list in harness/subagents/reference.mdx cites PR #5224 ("tool assembly off the event loop"), but the PR and its issue counterpart were deleted by repository administrators, so both URLs return 404. The PR itself was merged: squash commit 5727449 matches the cited description verbatim. Same approach as #7689 (changelog deleted-PR references): keep the [#5224] text and entry description, replace the URL with the squash-merge commit permalink. 1 line changed in each of the en and zh pages. * test(deermem): keep eviction fixtures fresh --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Wenchao An
|
7feaf051a2
|
feat(plugins): add redacted administrator diagnostics (#7612)
* feat(plugins): add redacted administrator diagnostics * ci: retrigger plugin diagnostics checks |
||
|
leom1919
|
24f15cc479
|
docs(zh): fix 5 cross-language anchor links pointing to English pages (#7603)
Translated zh pages link to the English docs routes (/docs/...) but kept localized Chinese anchor slugs, which do not exist on the English pages. Replace each Chinese anchor with the matching English heading slug (verified present in the target file): a-run-fails-after-a-config-edit, forks-are-accepted-not-refused, placements, failure-isolation, runaway-guards. Co-authored-by: leom1919 <leom1919@users.noreply.github.com> |
||
|
Agustin Torres
|
caafac0452
|
fix(wizard): replace retired Claude Sonnet 4 default with claude-sonnet-5-5 (#7585)
* fix(wizard): replace retired Claude Sonnet 4 default with claude-sonnet-5-5 The setup wizard's Anthropic default model is now `claude-sonnet-5-5` instead of the retired `claude-sonnet-4-20250514`. A new per-model `model_omitted_config` field makes the wizard leave out the extended-thinking (`budget_tokens`) and thinking-disabled templates for that model only, since Sonnet 5.5 rejects them; the other Claude models are unchanged. The Anthropic examples in `config.example.yaml` and `configuration.mdx` (which used `claude-3-5-sonnet-20241022`) now use `claude-sonnet-5-5` without those settings, and the change also adds a regression test and a CHANGELOG entry. Notice anthropic-retired-models-2026-10-27 about api.anthropic.com's API describes the change. It was read from a file and is not signed by the provider. * Update CHANGELOG with PR reference #7585 Added a reference to pull request #7585 in the CHANGELOG. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
leom1919
|
83f2dc5da4
|
docs(tools): fix dead ACP protocol link and protocol name (#7479)
agentconnectprotocol.org no longer resolves (NXDOMAIN). The integration depends on agent-client-protocol (Agent Client Protocol by Zed Industries), so point both en and zh tool guides at its canonical site https://agentclientprotocol.com and use the protocol's actual name. |
||
|
Luck
|
ce0fa4c114
|
feat(batch): add browser review for persisted results (#7006)
* feat(batch): add browser review for persisted results * fix(batch): address browser review feedback * test(batch): cover persisted result review regressions Exercise running-to-terminal detail refresh, scoped citation dialogs, omitted evidence, translated page labels, and legacy evidence bounds. Fix component lint and formatting and synchronize the review contract documentation. * docs(frontend): keep batch review guidance within budget * fix(batch): refresh retried reviews and account for omitted evidence * fix(ci): keep batch review guidance within size budget --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
lihongyuan99
|
07a4770e32
|
docs(harness): scope the memory guides per user and nest DeerMem knobs (#7463)
* test(memory): pin the memory guides to the layout DeerMem resolves
The en and zh harness guides still document the pre-abstraction storage
shape, and nothing in CI reads prose, so the drift stayed invisible.
Derive the per-user manifest and per-agent facts paths from
memory_file_path()/agent_facts_directory() and take the allowed memory:
top-level keys from the public MemoryConfig model.
* docs(harness): scope memory docs per user and nest DeerMem knobs
Both guides claimed a shared {base_dir}/memory.json plus one memory.json
per agent; DeerMem keeps one manifest per user and puts agent state in a
facts directory beside it. The zh Configuration example also still showed
DeerMem-private keys at the memory top level, which config.example.yaml
and the en twin already place under backend_config.
* docs(memory): align per-agent facts overviews in both languages
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Tsai Yuan
|
03c95f5278
|
fix(subagents): report model output length caps (#7355)
* fix(subagents): report model_length_capped when a subagent hits the output limit ModelLengthFinishReasonMiddleware only ran on the lead chain. A subagent whose response was cut at the provider output limit (finish_reason=length, stop_reason=max_tokens, MAX_TOKENS, Responses max_output_tokens) reached the lead as a clean `completed`, and a tool call truncated at that boundary still executed with partial arguments. Register the same middleware on the subagent chain, in the lead's relative position (after configured extensions, before Safety). It now also records each detected cap per context run_id and exposes `consume_stop_reason`, so the executor's existing duck-typed guard collection reports `stop_reason=model_length_capped`. Add the value to the subagent status contract (Python enum, contract fixture, frontend whitelist) with the label "model output length limit", and update the subagent docs. Tests cover the middleware channel, chain placement, the contract, and an end-to-end executor run with a fake model. Co-Authored-By: Claude Code <noreply@anthropic.com> * test(tasks): cover cap reasons on completed and failed runs Extend the existing shared-contract round-trip matrix to preserve both terminal statuses alongside every supported stop reason, including model_length_capped and loop_capped. --------- Co-authored-by: huangzz <110116@nd.com> Co-authored-by: Claude Code <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Lachlan Allen
|
7a32a5e177
|
feat(exa): support a configurable Exa-compatible base URL (#7306)
Exa web_search and web_fetch accept a per-tool base_url, falling back to EXA_BASE_URL and then the SDK default. Overrides are validated like the Serper base URL. web_fetch screens a configured backend with the shared delegated-backend guard. Co-authored-by: lachy3433 <lachlanallen.co@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Stella-xixi
|
5cef1a11ef
|
fix(checkpoints): run conservative retention after completion (#6051)
* fix(checkpoints): wire conservative retention after runs * fix(checkpoints): fence retention before checkpoint reads * fix(checkpoints): address retention review feedback * fix(checkpoints): retry delta reads after retention race * fix(checkpoints): bound admission retention to the root tail --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
trent87
|
0d73798beb
|
feature(skills): keep allowed-tools as compatibility metadata (#6935)
* refactor(skills): keep allowed-tools as compatibility metadata * docs(skills): align current guides with agent tool authority * test(skills): simplify authority regressions and sync Korean docs |
||
|
LingYi
|
ee02e51df9
|
fix(memory): preserve consecutive pre-compaction snapshots (#7152)
* fix(memory): preserve consecutive pre-compaction snapshots * fix(memory): warn on growing emergency snapshot backlogs --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Nan Gao
|
8d5f5fd0a8
|
feat(channels): share encrypted runtime channel config across Gateway instances (#6988)
* feat(channels): share encrypted runtime channel config across Gateway instances IM bot credentials entered in the web UI lived in each host's plaintext channels/runtime-config.json, and the router cached the merged config on app.state for the life of the process. With several Gateway instances a save or disconnect served by an instance that does not hold the provider's leader lease never reached the transport holder: it kept the old credentials until it next started the channel, and a disconnect could leave the old bot connected. - New shared table channel_runtime_configs (migration 0044): one row per provider with the config block encrypted under DEER_FLOW_CREDENTIALS_KEY, a plaintext disconnect tombstone, and a version bumped by every save. - SqlChannelRuntimeConfigStore replaces the JSON file for sqlite/postgres (memory keeps the file); it is synchronous on app_sync_sqlalchemy_url because its callers already run in worker threads. - The router re-reads the store on every request instead of caching the merged config, and refuses (503) to save credentials without a key. - The leader-lease supervisor compares the versions on each renewal tick and restarts or stops the holder's transport when a provider's config changed; a takeover reads a disconnect instead of reviving cached credentials. - The first start imports an existing runtime-config.json (rows already in the table win) and deletes the plaintext file. * docs(changelog): reference #6988 for the shared runtime channel config * ci: rerun checks after #6999 fixed the frontend guidance budget on main * fix(channels): start a held transport that is down when its runtime config changes When the lease holder's transport was down (a start that failed on the old credentials), applying a peer-saved config only updated the cached config, and the transport started on the next lease round: two renewal ticks instead of the one the docs promise. The apply step now starts it on the held lease right away, with the lease round's backoff bookkeeping. |
||
|
Nan Gao
|
33b8e84f11
|
fix(channels): keep IM runs alive when the creating request disconnects (#6876)
* fix(channels): keep IM runs alive when the creating request disconnects The channel manager holds a `runs.wait` / `runs.stream` request open for the whole run, and that request dies with the Gateway instance that received the IM message: a leader hand-off, a rolling restart, a crash. The Gateway cancels a run whose creating request disconnects unless the request asked for `on_disconnect: continue`, so every IM run in flight was cancelled on failover. Channel runs now ask for `continue`, like scheduled-task and MCP-task runs already do. The run finishes on the Gateway executing it and its thread state is persisted, so the answer stays readable in the web UI even though the IM reply is lost with the old transport. The key is set after the fire-and-forget branch because the SDK's `runs.create` has no `on_disconnect` parameter. Docs: CONFIGURATION.md and the Helm/deployment guides no longer list "failover cancels IM runs" as a known limit; IM_CHANNEL_CONNECTIONS.md and the channels AGENTS.md describe the policy. * docs(changelog): link #6876 * fix(channels): keep the inbound dedupe key once a run may be executing With `on_disconnect: continue` a run outlives the manager worker that requested it, but the worker loop released the inbound dedupe key on cancellation, so a provider redelivery on the replacement instance admitted the same event again and executed the turn twice (review on #6876). Each inbound message now carries an admission record: `requested` flips before the run request leaves the manager, `confirmed` once the Gateway proved it admitted the run (a run id, a stream frame, a completed response). The generic release paths (worker cancellation, worker/handler exceptions, a broken stream) keep the key when the run may continue: after confirmation, on cancellation after the request was sent, and on post-request transport failures (read timeout, read error, protocol error). Pre-request failures, Gateway rejections (busy thread, stale pin, status errors, connection failures) and the stream's terminal `error` frame still release it so the redelivery can retry. |
||
|
Nan Gao
|
e8c7df9042
|
feat(channels): arbitrate IM transports with a per-provider leader lease (#6843)
* feat(channels): arbitrate IM transports with a per-provider leader lease Every Gateway instance connected every enabled IM platform with the same bot credentials, so channels could not run on more than one instance (Telegram 409 storms, Discord answering twice, scattered per-process state). channels.leader_lease was reserved by #6746; this implements it. - channel_leases table (migration 0041): one row per provider. Acquire is one dialect-native upsert decided against the database clock and bumps a fencing epoch; renew extends only a live lease at the holder's epoch; release sets the expiry to 0 so a peer takes over on its next tick. - app/channels/leadership.py: the holder's local monotonic deadline (send time of the last confirmed renewal + lease_seconds); holds() turns false one renewal interval early, so the transport stops at least grace_seconds before a peer may connect. DB calls are bounded. - ChannelService: start fence in _start_channel (no DB call) behind every start path; fixed-rate supervisor renews/acquires and runs transport starts/stops as per-provider tasks; takeover reloads the provider's saved config; failed starts back off; shutdown releases only leases whose transport stopped. get_status reports leader/leader_here and the holder's transport state on every instance. - Notification worker leaves rows of providers another instance holds. - Gate: the enabled=false refusal now recommends auto/true; an explicit true with database.backend memory is refused when a channel is enabled. - Docs, Helm, deployment guides, AGENTS.md and CHANGELOG updated. * docs(changelog): link the channel leader lease entry to #6843 * fix(channels): see peer-held leases for providers absent from local config Review on #6843: a provider connected from the web UI on a peer never enters this instance's channel config, so the notification worker here still claimed its rows (and parked them for 900s) and GET /api/channels/ reported no leader for it. Every supervision round, and startup, now also reads the whole lease table once (bounded), and notification exclusions and status use those observations: a provider whose live lease a peer holds is left to it even without local config. Running that read concurrently with renewals exposed that, through SQLAlchemy's aiosqlite adapter, an UPDATE ... RETURNING racing a read on another connection can hand back one of its own bind values instead of the column (stored rows stay correct; plain sqlite3 and raw aiosqlite do not show it). A lease must never take such a row for a confirmed renewal, so acquire/renew/release now decide by affected-row count and acquire reads its epoch back in the same transaction. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Nan Gao
|
d18d2937e7
|
feat(channels): add channels.leader_lease and gate an explicitly disabled lease under multi-process signals (#6746)
* feat(channels): add channels.leader_lease and gate it under multi-process signals IM channels other than GitHub connect outbound to their platform. With more than one Gateway instance each one opens that connection with the same bot credentials: Telegram rejects the second poller with 409, Discord answers every message twice and per-process ChannelManager state diverges. The multi-process startup gate said nothing about channels; the only guard was a comment in the Helm values. Add the `channels.leader_lease` section (`enabled: auto|true|false`, `lease_seconds`, `grace_seconds`; startup-only, registered in the reload boundary) that the per-provider leader lease will read, validated by a Pydantic model from the non-schema `channels` mapping. The gate now refuses `enabled: false` under a multi-process signal while a non-GitHub provider is enabled under `channels.*` or connectable through `channel_connections`, naming the providers and the rollback; a malformed block is refused on every topology. `auto` follows the multi-process signal, so a single instance is unchanged. ChannelService strips the block before its per-provider loops and exposes it as `leader_lease`; `ensure_ready_channels` only visits registered providers. The two-Gateway harness overlay leaves the block alone. config_version 60 in the example, the Helm chart and its README; docs replace the comment-only "keep 1 replica" guidance with the gate. * docs(changelog): link #6746 * fix(channels): stop recommending the unimplemented leader lease as a remedy Review on #6746: the refusal told operators to switch channels.leader_lease from false to auto/true "so one instance owns each provider's connection", but the transport arbitration ships separately, so that change only bypassed the clause and every worker still started the same bot. The refusal now offers only the remedies that work today (a single instance, or disabling the channels) and states that auto/true are reserved for the lease that ships separately. A regression helper in the gate tests asserts the remedies start with the rollback and that auto/true are never offered. The config example, Helm values/README, CONFIGURATION.md, README, the deployment guide (en/zh), the model docstrings and both changelogs describe auto/true as reserved settings rather than an ownership guarantee. * fix(channels): warn on an enabled unknown channel key; reword the changelog entry Review on #6746: the registry guard in ensure_ready_channels skipped a misspelled provider key (channels.telegam) silently, where it previously reached _start_channel and logged 'Unknown channel type'. An enabled dict block that is neither a registered provider nor leader_lease now logs a warning that lists the registry. The changelog entry claimed the chart and docs drop the "keep 1 replica" guidance; they keep it (auto/true do not make several replicas safe yet) and only stop relying on the comment alone. * fix(dev): disable UI channel providers in multi-instance overlay --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Wenchao An
|
0b78051840
|
feat(extensions): add example guide and packaging checks (#6755)
* feat(extensions): add example guide and packaging checks * fix(extensions): align packaging checks with installer Reject Windows drive-qualified wheel paths before extraction and reuse the installer entry-point validator. Cover unsafe paths, dotted attributes, and valid dotted modules in regression tests. * fix(extensions): skip unused wheel extraction Only extract wheels when browser manifests need validation, with a backend-only regression test. Link the three remaining example READMEs in the developer guide. * docs(frontend): trim guidance intro below size budget --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Value Add
|
4beb6c4ea4
|
feat(community): add You.com web search provider (#6515)
* feat(community): add You.com web search provider Adds You.com as an opt-in community `web_search` engine, mirroring the existing community engines (tavily, brave, exa, firecrawl, groundroute). You.com runs its own web index and returns extracted page snippets per hit rather than a single meta description, so a research loop gets usable context without a follow-up fetch. The provider picks its endpoint by whether a key is configured, so it works out of the box and scales up when the operator adds one: - no key -> /v1/agents/search, keyless free tier - YDC_API_KEY -> /v1/search with X-API-Key, full Search API Endpoint and headers are always chosen together; the keyless endpoint rejects an auth header, so a stale key can never be sent there. Both return the same body shape, so one normalizer covers both. httpx only, no new dependency and no pyproject extra, so uv.lock is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(community/youcom): review follow-ups — drop stray .pr-body.md, guard OverflowError, add 429 hint * fix(doctor): make the You.com free-provider detail true with or without YDC_API_KEY The doctor detail returned before any key check, so operators who had set YDC_API_KEY still saw 'set YDC_API_KEY for the full Search API'. Reworded to match the sibling free-provider entries and stay accurate in both states. * fix(search): sweep youcom max_results onto the shared owner The single-owner gate that landed with #6563 flags this branch's private _coerce_max_results as a newly hand-rolled copy, so backend-unit-tests now fails after merging main. Follow the coordination note from review: delegate to deerflow.community.search_max_results.coerce_max_results, which owns the rejection profile (booleans, non-integral floats, OverflowError), and keep only the 1-100 mirror of You.com's server-side count clamp on top — clamping policy stays per-provider. The .inf regression test now goes through the public tool path and also pins the bool and non-integral-float shapes of the shared bar. * docs(tools): use the bare you.com platform link in config and docs Match every other provider entry: no other provider in config.example.yaml or the en/zh tools.mdx uses campaign-tagged URLs, so drop the utm params from the three you.com/platform links. Integration attribution stays in the identifying User-Agent sent to You.com hosts. * test(youcom): cover shared max_results ownership and bounds Register You.com among the shared-owner providers and assert default handling and the 100-result cap through outbound request parameters. --------- Co-authored-by: Brian Sparker <brainsparker@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: valueadd-lgtm <339269770+valueadd-lgtm@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Zhengxu (Joshua) Jin
|
0c5b25b4ab
|
docs: stop recommending deerflow-harness extras that do not exist (#6561)
The Tavily, Exa and Firecrawl tool pages and the AIO sandbox sections tell readers to run `uv add 'deerflow-harness[tavily|exa|firecrawl|aio-sandbox]'`. deerflow-harness has no such extras: tavily-python, exa-py, firecrawl-py and agent-sandbox are core dependencies. uv warns about the unknown extra and still writes it into backend/pyproject.toml. Say that no extra dependency is required, matching the Brave and Serply tabs and the E2B sandbox section. |
||
|
Nan Gao
|
4bab280250
|
fix(skills): share skill cache invalidation across Gateway replicas (#6495)
* fix(skills): share skill cache invalidation across Gateway replicas SkillStorage.load_skills() rescans disk and re-reads extensions_config.json on every call, but the prompt layer in deerflow/agents/lead_agent/prompt.py caches the enabled-skills list, the per-(app_config, user_id) entries and the rendered <skill_system> section per process, and every skill mutation in routers/skills.py only refreshed the process that handled it. With several uvicorn workers or several Pods sharing one home volume, a skill installed, edited, deleted, rolled back or toggled through replica A kept being offered as the old skill set by replica B until B restarted, and POST /api/skills/reload had to be sent to every Pod separately. Reuse the MCP cache-reset mechanism. A new generic helper, deerflow.config.shared_reset_marker, owns a hidden JSON marker beside the resolved extensions config (.<config name>.<suffix>.json): publish() replaces it atomically with a random generation under extensions_config_write_lock + extensions_config_file_lock, chaining previous_generation and an optional user_id; SharedResetMarkerTracker polls it at most once per second (monotonic clock, never blocking behind a concurrent poller), adopts the current state silently on its first poll, and classifies a change as user-scoped only when exactly one publication carrying that user happened since the last poll. deerflow/mcp/cache.py now builds its .<config>.mcp-cache-reset.json marker through the same helper with no behavior change. prompt.py publishes .<config>.skills-cache-reset.json via publish_skills_cache_reset(user_id=...) and checks the marker on every cache lookup (global cache, per-config cache, prompt-section LRU, user scoped entries), calling _invalidate_enabled_skills_cache() or invalidate_user_skill_cache(). The publishing process remembers its own generation so it does not retire the caches it just rebuilt. Every mutation tail in routers/skills.py (install/upload, edit, delete, rollback, enable/disable, reload) refreshes locally and then publishes the marker off the event loop; /skills/reload answers scope=shared_config, or scope=process when no extensions config path resolves, and a marker publication failure is a 500 so a 200 never means only one process was refreshed. Docs: README, backend/docs/API.md, skills/config/mcp/gateway AGENTS.md, and frontend skills.mdx (en/zh; also fixes the reference to a non-existent skills/loader.py) describe the shared invalidation and drop the per-Pod guidance. CHANGELOG entry under Unreleased / Fixed. * docs(changelog): link the skills cache reset entry to #6495 * fix(skills): drain the reload tail and keep marker failures out of 404 Two review follow-ups on the shared skills cache reset (#6495). POST /skills/reload awaited the local refresh and the marker publication unshielded. CancelledError is a BaseException, so a caller that disconnected between the two slipped past the `except Exception -> 500` mapping, surfaced no error, and left every peer replica serving the old skill set -- the one outcome the operator hook for external mount writes exists to prevent. The refresh and the publication now run as one `await_drained(_drain_skill_mutation("reload", ...))` tail, the same discipline the install/edit/delete/rollback/toggle tails already use; the exception -> 500 mapping and the scope response are unchanged. Publication errors could surface as 404. resolve_shared_config_path() only swallows the resolution-time FileNotFoundError; a config directory that vanishes between resolution and the lock file / atomic replace still raised FileNotFoundError from the marker write, and the install, edit, delete and rollback handlers mapped that to HTTP 404 with the raw OS error text for a skill that had in fact been applied. publish_skills_cache_reset() now re-raises OSError (and a misconfigured project root's ValueError) as SkillCacheResetPublishError, a RuntimeError that no handler maps to 404/400, so every handler reports it through its generic 500 branch; the message names only the exception type, never a server path. The install handler gains an explicit clause that pins that contract beside its 404 mapping. Tests: cancelling reload after the refresh still publishes the marker; a FileNotFoundError from the marker write yields 500 (not 404) for install, edit, delete and rollback, 500 for the public toggle and the generic reload error; the wrapper type is not an OSError. |
||
|
ZJPex
|
220d62d79a
|
feat(memory): add scoped fact lookup by ID (#6443)
* feat(memory): add scoped fact lookup by ID * docs(memory): synchronize tool-mode lookup guidance * docs(memory): clarify named-agent read capability requirement |
||
|
Nan Gao
|
4f64ca2ae0
|
fix(config): revalidate the extensions config singleton by file signature (#6386)
* fix(config): revalidate the extensions config singleton by file signature `get_extensions_config()` parsed `extensions_config.json` once per process and only the process that handled a write ever reloaded it. With several uvicorn workers, or several Pods sharing one volume, every other process kept its startup copy: the MCP servers the agent could use and the local-bash absolute path allowlist derived from the filesystem MCP server differed between replicas until each one restarted. The singleton now records the resolved path and the file's (mtime, size, sha256) signature and reloads when either changes, like `get_app_config()` does for `config.yaml` and `mcp/cache.py` does for the tools cache. After a successful load, a revision that cannot be loaded (file gone, truncated, invalid) keeps the last-known-good configuration and is logged once with the exception type only; the first load still raises. `set_extensions_config()` pins the injected instance until a reload or reset. The cached AppConfig's `extensions` snapshot is built from the singleton instance and refreshes when that instance changes, so it inherits the same fail-soft policy instead of parsing the file again. * docs(changelog): link the extensions config freshness entry to #6386 * fix(config): follow an explicitly reloaded extensions config path `reload_extensions_config(config_path=X)` recorded X, but the freshness probe re-resolved the default path, so the next read either flipped the cache back to the default file or pinned a stale copy of X. The cache now remembers the explicit path and both the probe and later reloads follow it until a reset or an argument-less reload. Also shortens the pointer in backend/AGENTS.md so the nested guidance chains stay under the hard size limit enforced by the agent-guidance check. * docs: keep backend guidance within the inherited size budget * fix(config): reload the probed extensions config path --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Nan Gao
|
063383d995
|
fix(deploy): wire the multi-replica prerequisites into the Helm chart and compose (#6347)
* fix(deploy): wire the multi-replica prerequisites into the Helm chart and compose The chart shipped gateway.replicas: 1 with a README citing the long-closed issue #3948, while its rendered config lacked run_ownership heartbeats and db run events: raising replicas made every starting Pod write its peers' lease-less runs off as orphans, and the Gateway's startup gate could not notice because its worker count only sees one Pod. Enable heartbeats and db run events in the default config, export DEER_FLOW_MULTI_INSTANCE from the replica count for the startup gate, generate and inject a shared AUTH_JWT_SECRET (per-Pod .jwt_secret files logged users out across replicas), declare a surge-then-drain rollout strategy, render a PodDisruptionBudget when replicas > 1, and bound uvicorn's graceful shutdown inside a 90s termination grace period. docker-compose gets the same uvicorn bound and a 60s stop_grace_period so the 30s memory flush is no longer SIGKILLed. Docs and comments now list what actually remains single-instance: IM channels, WeChat QR login and browser tools. * fix(deploy): honor existingAppSecret, declare multi-instance explicitly, cover the dev stack's shutdown Follow-up to the multi-replica prerequisites wiring, addressing review: - `deer-flow.appSecret` returns `existingAppSecret` when it is set. The gateway and frontend kept referencing the generated `<release>-app` Secret, which the chart stopped rendering, so every Pod sat in CreateContainerConfigError. The value is now declared in values.yaml. - `gateway.multiInstance` (boolean, default null) overrides the replica-derived DEER_FLOW_MULTI_INSTANCE, PodDisruptionBudget and AUTH_JWT_SECRET requirement for deployments scaled with kubectl or an HPA, which change the replica count without a Helm upgrade. - AUTH_JWT_SECRET is required, not optional, whenever the gateway is multi-instance: concurrently booting Pods otherwise race to write different `.jwt_secret` files on the home volume and sign sessions with different keys. - `gateway.uvicornGracefulShutdownSeconds: 0` is honored instead of being swallowed by `default 10`. - docker-compose-dev.yaml and dev-entrypoint.sh get the same uvicorn graceful-shutdown bound and `stop_grace_period: 60s`. - The chart README documents the one-time sign-out when upgrading a release that predates AUTH_JWT_SECRET and how to seed the previous key into the Secret first. - Tests read the shutdown budget from the Gateway through a shared helper instead of hard-coding 5/5/30, the raw-template string matches give way to `helm template` renders, and new renders cover multiInstance, existingAppSecret, a disabled PDB and a zero graceful-shutdown bound. * docs(changelog): link the multi-replica deploy entry to #6347 * fix(deploy): budget every bounded teardown hook and flag the multiInstance PDB window Review follow-ups on #6347: - The shutdown budget modeled only channel stop + run drain + memory flush (40s). The lifespan also runs the startup trash sweep, the notification delivery worker stop, browser-session close and the MCP session-pool close, each bounded at _SHUTDOWN_HOOK_TIMEOUT_SECONDS, plus a 1s retrieval-warm wait: about 61s worst case. _gateway_shutdown_budget.py now reads all of them from the Gateway, a drift guard pins the hook count against app.py, and both compose files move stop_grace_period from 60s to 90s (Helm's 90s already had room: 5 + 10 + 61). Comments and the chart README cite the full arithmetic. - gateway.multiInstance: true renders the PodDisruptionBudget immediately, which blocks drains of the single Pod until the scale-up happens; the README, values comment and both deployment guides say so and name podDisruptionBudget.enabled: false as the interim escape hatch. |
||
|
quifox
|
06e9bfd269
|
fix: propagate current run failures from wait endpoints (#6217)
* fix: propagate current run failures from wait endpoints * docs: clarify checkpoint wait errors and SDK behavior |
||
|
Hyeonsang Cho
|
7f7668d975
|
fix(community): refuse non-global addresses in the shared SSRF guard (#6202)
* fix(community): refuse non-global addresses in the shared SSRF guard is_blocked_address ORed is_private, is_loopback, is_link_local, is_reserved, is_multicast and is_unspecified. None of them covers the 100.64.0.0/10 shared address space, which holds CGNAT and Tailscale hosts and Alibaba Cloud's 100.100.100.200 instance metadata endpoint, so web_fetch, web_capture, the agentic browser and personal MCP connections could reach it, including through the IPv4-mapped ::ffff:100.100.100.200 form a DNS answer can carry. Also refuse any address that is not is_global, the classification the repo's other SSRF filters already use. Keep the existing flags alongside it: some non-public forms still report is_global, such as the NAT64 spelling of a metadata address (64:ff9b::a9fe:a9fe) and 224.0.0.1, so an is_global-only check would newly allow them. * docs(changelog): reference #6202 in the shared address space entry * test(community): pin the IPv4-mapped metadata spelling at the web_fetch and personal MCP entry points --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
lihongyuan99
|
571a7e40e4
|
docs(harness): import the Gateway app from the module that defines it (#6169)
Both the en and zh integration guides show `from deerflow.app.gateway.main import app`, but there is no deerflow.app package and no gateway/main.py in the tree: the FastAPI instance is created by create_app() and bound at module level in backend/app/gateway/app.py, which is also how backend/AGENTS.md documents it. The copy-pasteable snippet now names that module, so mounting the Gateway as a sub-application works from the documented import instead of failing at runtime. |
||
|
Anton Dziatkovskii
|
a83aebe702
|
fix(docs): harness docs document an async client API that does not exist (#5843)
* docs: replace nonexistent async client API in harness docs
DeerFlowClient has no async methods: client.astream() and client.ainvoke()
do not exist, so the "Create Your First Harness" tutorial and the harness
integration guide fail with AttributeError on their first call. Overrides
were also shown as a nested config={"configurable": {...}} dict, which
stream()/chat() swallow via **kwargs and silently ignore.
Switch the examples to the shipping API (sync stream()/chat(), flat kwarg
overrides, agent_name as a constructor arg) in the EN and ZH copies of both
pages. backend/docs/STREAMING.md is left alone: its astream mentions are
design discussion and LangGraph internals, not user-facing examples.
Assisted-by: Claude Code / claude-opus-5
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Dziatkovskii <194927794+tonydzi@users.noreply.github.com>
* docs: serialize StreamEvent before StreamingResponse, use real event names
Review follow-up on the two points raised in #5843.
P2 — the streaming examples handed a generator of `StreamEvent` dataclasses
straight to Starlette. `StreamingResponse` calls `.encode()` on any chunk that
is not `str`/`bytes`, so the first event raised
`AttributeError: 'StreamEvent' object has no attribute 'encode'`. Both
occurrences (the `run_agent` snippet and the inline FastAPI route, which shipped
the dataclass `repr()` through an f-string) now serialize to SSE frames built
from `event.type` / `event.data`.
P3 — the tutorial showed mapping literals with `messages` and `thread_state`,
neither of which exists. The real contract is
`StreamEventType = Literal["values", "messages-tuple", "custom", "end"]`
(backend/packages/harness/deerflow/client.py:128) and the stream yields
dataclasses, so the section now shows `StreamEvent(...)` with the real names
plus a consuming loop.
The zh tutorial had no equivalent section at all; added for parity.
Assisted-by: Claude Code / claude-opus-5
Machine: A-Mac16-2019-PaloAlto
Account: tonydzi
Operator: robot:pr-reply-daily
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: drop the nonexistent load_config() from every harness snippet
Review follow-up on #5843 (@willem-bd): the rewritten tutorial still opened
with `from deerflow.config import load_config`, which raises ImportError
before `client.stream(...)` is ever reached — `deerflow/config/__init__.py`
exports `get_app_config` and friends, and the string `load_config` does not
appear anywhere under `backend/packages/harness/deerflow/` (the only near
match is `load_configured_extension_middlewares`).
The call was also unnecessary: `DeerFlowClient.__init__` already resolves
configuration itself (client.py:220-222 — `reload_app_config(config_path)`
when a path is given, otherwise `get_app_config()`, which auto-loads from
the default path or DEER_FLOW_CONFIG_PATH).
Fixed the class rather than the one snippet you flagged: the same dead
import appeared 10 times across 4 pages (EN + ZH tutorial and integration
guide). Every one is gone. The "configuration in embedded mode" section now
shows the API that exists — `DeerFlowClient(config_path="...")` — instead of
`load_config(config_path="...")`.
Also from the review:
- the `end` example showed only `total_tokens`; `stream()` always yields
input/output/total (client.py:920), and the snippet right below prints the
whole usage dict, so a reader saw keys the docs omitted. Both EN and ZH now
show the full payload, and the `values` example visibly elides `artifacts`
and `summary_text` instead of pretending they are absent.
- the two SSE comments on the ZH integration page stayed English while the
example above them was translated; translated.
Verified statically, not executed: `grep` for the symbol across the harness
package (zero hits) and a read of `DeerFlowClient.__init__`. I did not stand
up a backend to run the tutorial end to end.
Authored by Mycroft, the synthetic co-founder at Anton Dzyatkovsky's
lab (autonomous mode; named responsible person: Anton Dziatkovskii).
Assisted-by: Claude Code / Opus 5 (review response, implementation)
Machine: MacBook-Anton
Account: a
Operator: mycroft (autonomous routine github-thread-watch)
Signed-off-by: tonydzi <194927794+tonydzi@users.noreply.github.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: fix native artifact SSE encoding and custom config advice
---------
Signed-off-by: Anton Dziatkovskii <194927794+tonydzi@users.noreply.github.com>
Signed-off-by: tonydzi <194927794+tonydzi@users.noreply.github.com>
Co-authored-by: Anton Dziatkovskii <194927794+tonydzi@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
NanHan Qing
|
c714629faf
|
fix(sandbox): expose DEERFLOW_USER_ID to every bash command (#6058)
* fix(sandbox): expose DEERFLOW_USER_ID to bash commands A skill's SKILL.md can trigger scripts, but those scripts run as bash subprocesses inside the sandbox and cannot reach the Gateway auth context, so get_effective_user_id() is unavailable to them and user-scoped skills are impossible to write. The maintainer settled the direction on #3919: inject the id as DEERFLOW_USER_ID. Every bash command now begins with 'export DEERFLOW_USER_ID=<resolved>; ' (or 'unset' when the resolved id is unusable), mirroring the existing DEERFLOW_CHANNEL_USER_ID channel-identity prefix (#3914). The id rides the command string rather than execute_command(env=...), because a non-empty env switches AioSandbox to the bash.exec API reserved for request-scoped secrets. The earlier attempt (#4029) was closed for making every command carry the new export without updating the exact-command assertions in tests/test_channel_user_id_env.py. Those assertions are updated here, and the composition of the two prefixes is pinned by a dedicated test. * fix(sandbox): fix the DEERFLOW_USER_ID unit test and cover review findings Follow-up to the review on this PR. Unit test (CI `backend-unit-tests (2)`): the change moved the command contract but left the exact-command assertion in tests/test_skill_request_scoped_secrets.py::test_local_bash_forwards_env_and_timeout expecting the bare `echo hi`. The local validation run used a `-k` filter that did not select that file, so the breakage was invisible there. The assertion now pins the composed command, and `_is_windows` is patched explicitly so it no longer depends on the host OS. [P2] Windows local sandbox: `export` is invalid under the PowerShell/cmd.exe fallback, but skipping the prefix outright left the promise unmet on Windows. The id is published through `execute_command(env=...)` there instead: LocalSandbox layers `env` into the per-process environment and, unlike AioSandbox, has no persistent session that could carry a stale value. It is deliberately kept out of `injected_env` so the identifier does not enter the secret-redaction set. Docstring: `_user_identity_prefix` now carries the same "informational, not authenticated identity" caveat the channel variable has in backend/app/channels/AGENTS.md. The value is authorization-grade at the Gateway boundary, but any bash command can overwrite its own environment, so skills must not treat the shell variable as proof of identity. Docs: the en/zh sandbox.mdx identity sections and the channels/AGENTS.md identity paragraph now describe the second, outermost prefix, its ordering against the channel prefix, and the Windows behaviour. |
||
|
xiaodu55
|
f67b1f574e
|
fix(tui): report headless --print/--json failures instead of leaking tracebacks (#5963)
* fix(tui): report headless --print/--json failures instead of leaking tracebacks
The headless --print and --json runners dispatched straight into the
embedded client with no error boundary, so session-creation, thread-
resolution, chat, and stream failures escaped main() and surfaced as
unhandled Python tracebacks from the console script - for example the
documented CheckpointModeMismatchError case, which the checkpoint
operations/troubleshooting docs recorded as a raw traceback.
Both runners now catch Exception (KeyboardInterrupt/SystemExit still
propagate), mirroring the TUI's stream_actions() boundary:
- --print writes one concise 'Error: ...' line to stderr and returns 1
- --json appends a terminal {"type": "error"} record to the NDJSON
stream (diagnostics stay on stderr) and returns 1; every stdout line
remains independently valid JSON, including failures mid-stream
The checkpoint operations/troubleshooting docs (en + zh) no longer
describe the headless runners as printing an uncaught traceback.
Fixes #5962
* fix(tui): close the broken-pipe edge and document the --json error contract
Review follow-ups on #5963 (both non-blocking):
- print(answer) sat outside the headless boundary, so a consumer closing
the pipe early (deerflow --print ... | head) still got a
BrokenPipeError traceback plus the interpreter's shutdown-flush noise.
The final --print write and the --json terminal error-record write now
catch BrokenPipeError, point the stdout descriptor at devnull (the
documented SIGPIPE-adjacent recipe) and exit 1 without a traceback.
- The terminal {"type": "error"} record is a real contract for --json
scripters but was only documented in the checkpoint pages; the
backend/docs/TUI.md command table and the README cheat line now state
the error record and the exit-1 semantics where CLI consumers look.
|
||
|
lekt8
|
f889ce7517
|
feat(community): add Unbrowse web fetch provider (#5981)
* feat(community): add Unbrowse web fetch provider Add a community web_fetch provider backed by Unbrowse (https://unbrowse.ai), a hosted service that returns a page as markdown, over plain HTTP when that is enough and through a cloud browser when the page needs JavaScript. The tool sends one JSON-RPC tools/call POST (unbrowse.scrape) to Unbrowse's MCP endpoint, so it needs no extra Python package and no MCP client (uses httpx, already a dependency). Unbrowse is not a general web search engine, so only web_fetch is added. Changes: - backend/packages/harness/deerflow/community/unbrowse/{__init__,tools}.py web_fetch_tool with the same output shape as the Sofya, Exa and Firecrawl fetch tools (# <title> plus markdown, 4KB cap). API key is read from the config.yaml api_key field or the UNBROWSE_API_KEY env var. Optional render: auto|never|always; unknown values log a warning and fall back to auto. JSON-RPC errors, isError tool results, HTTP errors and malformed payloads come back as "Error: ..." strings. - backend/tests/test_unbrowse_tools.py: key resolution, render handling, request shape, structuredContent vs text payloads, error paths. - scripts/doctor.py, scripts/wizard/providers.py: register the key and the web_fetch provider; backend/tests/test_doctor.py covers both doctor paths. - config.example.yaml, .env.example, backend/docs/CONFIGURATION.md, tools/AGENTS.md, EN/ZH harness/tools.mdx: document the provider. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(changelog): add Unbrowse web_fetch provider entry Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
931d2eeb81
|
fix(models): deep-merge when_thinking_enabled on the legacy path (#5894)
* fix(models): deep-merge when_thinking_enabled on the legacy path The legacy thinking path (profiles without a `reasoning:` block, i.e. every existing config.yaml) applied `when_thinking_enabled` with a shallow `dict.update`, so a template such as `extra_body.thinking.type=enabled` replaced the profile's whole `extra_body` and silently dropped sibling keys like GLM's `extra_body.tool_stream` — but only while thinking was on. The disable branch of the same function already deep-merged into `extra_body`, and the contract path added in #5780 deep-merges through `_merge_settings` for exactly this reason (its docstring names `tool_stream`). The on/off asymmetry cannot be intended: the same profile sent `tool_stream` with thinking off and lost it with thinking on. Route the enable template through `_merge_settings` as well. Precedence is unchanged (template values win on conflicts); the only observable difference is that keys present only in the profile now survive a thinking-on request. Everything else on the legacy path — the synthesized `reasoning_effort=minimal`, the disable payloads, the supports_thinking guard — is untouched. Tests pin both directions for a base extra_body beside a thinking template, recursive merging of nested chat_template_kwargs, and template precedence. * docs(changelog): reference #5894 in the when_thinking_enabled entries * fix(models): keep the template's vLLM switch spelling authoritative when merging Review follow-up. Deep-merging the thinking template into a profile keeps both spellings of vLLM's toggle when they differ — base `chat_template_kwargs.enable_thinking: false` plus a template using the legacy `thinking: true` alias — and VllmChatModel's normalizer maps the alias onto `enable_thinking` only when the latter is absent, so the profile's `false` won and thinking stayed off despite thinking_enabled=True. The previous shallow replacement never hit this because it discarded the base switch. The contract path had the same trap since it already deep-merged, and both disable branches had the mirror image (a base `enable_thinking: true` outlived a template `thinking: false`). Route every thinking on/off merge through _merge_thinking_payload: it drops the profile's *other* spelling of the switch before the deep merge, so the payload's spelling decides in both directions on both paths, and unrelated chat_template_kwargs still survive. Tests cover both spelling combinations, both directions, and both paths, asserting the switch as the provider will send it after normalization. * fix(models): deep-merge the legacy disable template and mirror the vLLM switch instead of pruning Review of the previous head found two regressions. The legacy when_thinking_disabled branch was still dict.update-ed, so a profile that declares both templates (the shape of most extra_body examples) kept tool_stream while thinking was on and lost it while off. And the merge pruned the profile's other vLLM switch spelling for every model class, although only VllmChatModel maps `thinking` onto `enable_thinking`: on a plain ChatOpenAI profile the prune removed the only key vLLM reads, so thinking-off could no longer switch reasoning off. Route when_thinking_disabled through _merge_thinking_payload too, and replace the prune with mirroring: after the deep merge, the value the payload declares for its switch spelling is copied onto the profile's other spelling when the profile carries it. VllmChatModel collapses the pair, plain OpenAI-compatible classes send both, and either way the server reads the template's intent. No spelling is invented, a payload that spells both is left alone, and non-mapping template values are forwarded unchanged instead of raising. _VLLM_THINKING_SWITCHES is now the single definition used by the disable, toggle and dialect helpers. Docs state the merge semantics (keys are never removed; a template can only add or override) and drop the "byte-for-byte" wording; tests cover the disable template on/off, nested inheritance, non-mapping values, the raw payload for a non-normalizing class, and the no-invention rule. * fix(models): copy thinking templates on merge and document the never-remove rule Review follow-up. Templates merged into the constructor settings were stored by reference wherever the profile lacked the key, so the kwargs handed to the model class aliased the cached ModelConfig; the mirroring step now writes into that mapping, so make _merge_settings and _deep_merge_dicts deep-copy every template mapping they store. Pinned at every depth by a test that also checks the profile is unchanged. Deep-merging the legacy when_thinking_disabled template means it can no longer clear a key the profile's base extra_body sets: a base extra_body.thinking {type: enabled, budget_tokens} reaches the provider as {type: disabled, budget_tokens}, which Anthropic-style APIs reject. The synthesized disable payloads and the contract path already behaved this way. State the rule and the migration (enable-only keys belong in when_thinking_enabled) in the changelog, models/AGENTS.md and configuration.mdx, and pin it with a test for both shapes. Route the legacy synthesized OpenAI disable through _merge_thinking_payload too (identical output), narrow the docstring and guidance wording about the native-Anthropic disable, and make the two vLLM switch suites resolve the real VllmChatModel / ChatOpenAI classes so a class-gated merge would fail the case it is named after. |
||
|
Totoro
|
457ac8cabd
|
docs(memory): correct backend contract details in the MemoryManager guide (#5943)
Follow-up to #5929. A config or custom backend written from a few statements on the page fails: - OpenViking's failure_policy.read takes fail_open or raise, and the backend supports middleware mode only. - from_config is abstract, so it belongs in the mandatory set. - Tool mode also needs get_memory() and fact CRUD, because it skips the per-turn add() writes. - Management methods return the DeerMem shape, and create_fact returns a (memory_data, fact_id) tuple. - staleness_review_enabled is on by default, and the governance flags live under memory.backend_config. The zh page gets the same fixes plus a typo (回记 -> 召回). Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
pclin
|
e080c194e0
|
docs(memory): explain the pluggable MemoryManager and how to implement your own backend (#5929) | ||
|
Zeren Wang
|
82671596f3
|
feat(authz): add plugin resource targets, provenance, action and management enforcement (#5842)
* feat(authz): add plugin resource targets, provenance, action and management enforcement
PR1 of the extension-tool-authorization series
(docs/plans/2026-09-24-extension-tool-authorization-spec.md): plugin
authorization plumbing, no new author-facing declaration.
- `authz/plugin_targets.py` is the single encoder/validator for composite
targets (`namespace/part`), with one validating composer behind three
kind-checked constructors; invalid input raises instead of degrading.
- `authz/plugin_authz.py` is the harness decision layer (`plugin_action`,
`plugin_management`, plus the batch filters the page slice will consume):
disabled is a no-op, an explicit deny raises, and provider errors follow
`fail_closed` like the sandbox gate.
- `POST /api/plugins/{ns}/actions/{name}` decides `plugin_action:invoke`
after the action resolves and before the body is streamed.
- `deerflow_extension_api.auth` publishes `require_plugin_management` /
`arequire_plugin_management` with two `app.state` resolver keys; the
Gateway installs both. Contract version 0.2.3 -> 0.2.4.
- `tools/tool_provenance.py` records host-written plugin provenance for
display (assembly descriptor, Layer-2 context later); no decision reads a
label.
- Plugin providers use a `(config signature, loop key)` cache so a
loop-affine provider is never handed to another loop or to a sync caller.
- RBAC gains `plugin_actions` / `plugin_management` aliases;
`config_version` 47 -> 48 across the three mirrors.
Tests: new plugin target/provenance/action/management suites plus a
blocking-IO anchor whose offloads were reverted to prove red->green.
* fix(authz): decide plugin resources from the request's config snapshot
Review round 1 on PR #5842 (willem-bd).
P1 — the decision layer no longer reloads the configuration. The six
`authz/plugin_authz.py` functions take a required `app_config` snapshot (the
same request-scoped config the Gateway used to resolve the provider), and
`resolve_/aresolve_plugin_authorization` return it as a third element, so
`enabled`/`fail_closed` can never come from a second read that drifted from
the one that produced the provider. A snapshot the caller could not read at
all is *unavailable*, not disabled: it fails closed as
`authz.config_unavailable`. The Gateway's `_plugin_app_config` now separates
an absent `config.yaml` (no gate, today's behavior) from a config that exists
but cannot be read or validated (deny) instead of swallowing both through
`safe_app_config`.
P2 — `_validated_decision` requires `type(decision.allow) is bool` on both
the sync `authorize` and async `aauthorize` paths, so a provider returning
`AuthzDecision(allow="false")` is a malformed verdict that follows
`fail_closed` instead of being read as an allow.
Both fixes are covered by regression tests that were verified red against the
pre-fix behavior (P1: 200 vs 403 and resolver True vs False; P2: 12 cases
across fail_closed/fail_open and sync/async), plus tests pinning that an
absent config and `authorization.enabled: false` keep today's behavior.
* fix(authz): treat a request-time config absence as unavailable
Review round 2 on PR #5842 (willem-bd).
P1 — `_plugin_app_config[_async]` no longer maps a missing `config.yaml` to
`None` ("no gate"). A Gateway cannot have started without a readable config
(`app/gateway/app.py` reads it through this same accessor in `lifespan`), so
absence *during a request* means the configuration this process was running on
became unavailable — a hot reload, an atomic replace, a ConfigMap remount. It is
unavailable, not disabled: the error propagates,
`resolve_/aresolve_plugin_authorization` raise
`_PluginAuthorizationUnavailable(fail_closed=True)`, the action dispatcher
returns 403 and the management resolvers answer `False`. The dead
`(None, None, None)` allow path is gone, the resolvers return a non-optional
`AppConfig` as their third element, and callers key "authorization disabled" on
`provider is None` alone. This matches the rule the decision layer already
documented (`authz.config_unavailable`) and the public docs.
P2 — `_deny_reason_code` is total: a `reasons` that is not iterable degrades the
code to `authz.denied` instead of raising `TypeError` outside the guarded
provider path, which turned a denial into an unhandled error. It is deliberately
not folded into `_validated_decision`: `allow` was a real `bool`, so the verdict
is a valid denial, and treating it as malformed would flip it to an allow under
`fail_closed: false`.
Both fixes are covered by regression tests verified red against the pre-fix
behavior (P1: resolver True vs False, async `PermissionError` not raised, route
200 vs 403; P2: `TypeError: 'NoneType' object is not iterable` at
`plugin_authz.py:113`). The action-route tests in
`tests/test_plugin_contributions.py` now install a readable
`authorization.enabled: false` config instead of relying on a config-less
process, and `docs/full-stack-plugins.md` plus the PR1 decision log record the
new rule.
* fix(authz): apply the running policy when the plugin config is unreadable
Follow-up to f03d2ea9 (review round 2 on PR #5842, willem-bd).
The first attempt read the P1 finding as "absence is never disabled" and failed
every config read closed unconditionally. That is too broad for this repo: a
config-less host is deliberately treated as having no policy to apply
(`_get_route_authorization_config` falls back to a disabled config,
`sandbox_authz.safe_app_config` documents the same no-op for "CI runners and
direct-call tests"), and the rule turned the config-less e2e plugin host
(`extension_test_fixtures.bookmark_plugin_gateway`) into a 403 — the three
`bookmark-plugin.spec.ts` cases lost the save action. Reproduced locally:
`POST /api/plugins/community.bookmarks/actions/save` against that fixture host
returned 403 before, 200 after this change.
The rule is now "has this process loaded a configuration?", not "was the last
read successful?":
- No loaded config (`peek_loaded_app_config()`, new read-only accessor on
`deerflow.config.app_config`) and no `config.yaml`: no policy to apply, so the
gate stays a no-op — unchanged for CI runners, direct-call tests and hosts
that mount only the plugins router.
- A loaded config whose file disappears or cannot be read: *unavailable*, never
*disabled*. The resolvers raise `_PluginAuthorizationUnavailable` carrying the
failure flag of the configuration this process runs on (authorization
disabled there -> no gate; enabled -> that policy's `fail_closed`, `True` by
default), so an enabled Gateway answers 403/False in the window instead of
letting protected plugin actions and management routes through.
- A config file that exists but cannot be read or validated right now: fail
closed when nothing is loaded (round 1's rule), otherwise the loaded policy
decides.
`peek_loaded_app_config()` is preferred over a Gateway-side startup snapshot: it
reflects the last successfully loaded config, so a hot-reloaded policy is what
applies during the window, and there is no lifecycle hook to keep in sync.
Evidence: the three no-config/lost-config branches in
`tests/test_plugin_management_guard.py` and
`tests/test_plugin_action_authorization.py`, round 1's unreadable-config tests
unchanged, `test_peek_loaded_app_config_survives_a_missing_file` in
`tests/test_app_config_reload.py`, and the fixture-host probe above. Docs
(`docs/full-stack-plugins.md`, `reference.mdx` en/zh) and the PR1 decision log
state the final rule; the log records the superseded variant.
* fix(authz): keep a denial whose reasons raise while being read
* fix(authz): intersect plugin batch filters with the candidates
The plugin_page/plugin_management batch helpers returned the provider's
`filter_resources` answer verbatim, so a custom provider could hand the
projections a target that was never asked about ("*", another namespace,
a phantom page). Every other filter call site re-projects from its own
candidate list; do the same here, state the intersection on the provider
contract, and pin it with a regression test that fails on the old
pass-through.
* docs(plugins): map a management denial to 403 in the route example
The contributed-route example called `arequire_plugin_management` and let
the documented `PermissionError` escape. The Gateway installs no
`PermissionError` handler, so a route copied from the example answered 500
on a normal policy denial instead of 403. Both example routes now translate
the denial, and the surrounding bullets say why the translation is the
route's job.
* docs(extensions): point the 0.2.4 contract row at the PR that ships it
Every other row in the version-history table links the pull request that
shipped the version; 0.2.4 linked the authorization tracking issue instead.
Both the English and Chinese tables now link this PR.
|
||
|
Zeren Wang
|
c943d953cd
|
docs(checkpoints): add a ten-chapter checkpoint storage manual in both languages (#5845)
* docs(checkpoints): add a bilingual checkpoint storage manual Add the missing docs-site coverage for dual-mode checkpoint storage (`database.checkpoint_channel_mode: full | delta`) as a ten-chapter manual under `harness/checkpoints/` in both languages: concepts, quick start, channel modes, snapshot cadence, history cache, resume and rollback, operating checkpoints, observability, troubleshooting by symptom, and a reference of every configuration key, error message, HTTP status mapping, public symbol, load-bearing test, and pinned upstream defect. The index page is `asIndexPage`, so `/docs/harness/checkpoints` is the section itself and its `_meta.ts` lists only the nine child chapters. Discovery: `harness/_meta.ts` registers the section, the harness configuration module table gains a `database` row, and the thread persistence section of `agents-and-threads` links to the manual. Content is grounded in the code. Only the three measured values recorded in `backend/docs/runtime-guidance-details.md` are quoted; the upstream-derived 41x storage figure in the untracked delta-channel RFC is deliberately not cited. Pinned upstream defects are documented with their issue and PR numbers and the versions they are pinned at (`langgraph 1.2.9`, `langgraph-checkpoint 4.2.0`, `langgraph-checkpoint-postgres 3.1.2`). * docs(checkpoints): correct the history-cache restart requirement The history cache is built with the checkpointer at Gateway startup (`langgraph_runtime()` -> `make_checkpointer()` -> `make_checkpoint_cache()`), so editing `database.checkpoint_cache.*` does not change a running process. The manual previously marked the whole section "not restart-required", which conflated "not frozen for correctness" with "hot-reloadable" and could mislead an operator changing capacity, credentials, or retention. Distinguish the two properties everywhere the section is described: history-cache.mdx marks every field as needing a restart and adds a callout explaining the difference, reference.mdx redefines the Restart column and fixes the five rows and the effect list (adding the `asyncprov` evidence alias), troubleshooting.mdx replaces the "two keys need no restart" claim and gains a dedicated symptom entry, and observability.mdx notes that changed counters need a restart. The Chinese text is corrected in parallel. Verified: the docs link test and the docs unit suite pass, frontmatter parses for all 20 pages, and every in-page anchor resolves on the rendered English and Chinese history-cache, troubleshooting, reference, and observability pages. * docs(checkpoints): account for periodic snapshots in the storage-growth claim Every periodic snapshot stores the entire message list as of that step (`_DeltaSnapshot(ch.get())`, persisted as a checkpoint blob), so with a fixed snapshot frequency K the retained total after N turns is O(N²/K + N), not O(N). The manual claimed unconditional linear growth, which understated capacity planning — the more so because nothing in DeerFlow prunes those snapshots yet. Distinguish per-step delta writes from total retained storage in index.mdx (prose and table), modes.mdx (step-cost row), and operations.mdx; note in cadence.mdx that K is also the capacity knob that scales the snapshot bytes. The English and Chinese chapters are corrected in parallel. The reference chapter keeps the released #4292 wording as a verbatim quotation and now names the retained-storage bound beside it. Verified: `_DeltaSnapshot(ch.get())` and its blob serialization in the pinned langgraph (channels/delta.py:196-198, checkpoint/serde/types.py:19-25) confirm the premise; the docs link test and docs unit suite pass; all ten changed pages render with resolving anchors and no stale linear-growth wording. * docs(checkpoints): correct the run-error surfaces and repair two evidence pointers Review follow-ups on the checkpoint manual, both languages. Error-surface corrections: - Run creation is not always asynchronous: when a thread's run-event feed is empty but a checkpoint head exists, ensure_checkpoint_history_seeded reads that head inside start_run, where only ConflictError (409) and UnsupportedStrategyError (501) are mapped, so a mode mismatch returns a plain 500 before admission. Troubleshooting, observability and the reference route tables now scope the "does not pre-gate" claim. - The 503 symptom is retargeted. Only Gateway startup, make_lead_agent and DeerFlowClient.__init__ call the freeze helpers, so a config.yaml edit cannot make a state/history/thread route answer 503. A mode edit is injected over by the run worker and ignored until a restart; a cadence edit fails the next run with the restart-required message. The router's 503 mapping is marked defensive in modes and the reference. - The regenerate section title no longer groups the state endpoints, which map a mismatch to 409. - operations: the Textual TUI catches the mismatch through stream_actions(); only the headless --print/--json runners print a traceback. Evidence-pointer repairs: - reference: this branch's CHANGELOG insertion shifted the #5734 entry from 3032-3039 to 3040-3048; the citation is re-pointed. - reference: the #8548 row cited an untracked worktree plan file; replaced by the in-repo test anchor, and "mirror issue #8551" reworded to "related". - reference: new `agent` alias for the agent-factory freeze sites. - history-cache: the bare `deps:`/`asyncprov:` aliases are expanded to full paths; the alias legend lives only on the reference page. * docs(checkpoints): say a delta fork is accepted, not refused "Delta mode cannot fork" overstated the constraint. Nothing in the stack rejects the fork: LangGraph stores the second child, no saver checks for an existing sibling, and ensure_checkpoint_mode_compatible returns immediately when the process mode is delta. What breaks is the read — the walk replays the abandoned sibling's writes stored under the shared parent. Reproduced on the pinned stack (langgraph 1.2.9 / langgraph-checkpoint 4.2.0, AsyncSqliteSaver) by running this repo's own #4458 fixture without the linearization call: the forked resume returns ["h1", "a1", "h2", "a2", "a2-new"] instead of ["h1", "a1", "h2", "a2-new"], i.e. the superseded answer comes back. - resume-and-rollback: title/description/heading now say "cannot fork correctly"; new "Forks are accepted, not refused" subsection covering the unguarded write side. - troubleshooting: the "old answer reappears" cause states the same split. - reference: the POST /threads/{id}/state row notes that a client-supplied checkpoint_id older than the head is applied as a delta fork with no linearization — the write shape upstream tracks as #8551 (open), which the run-path linearization does not cover. * docs(checkpoints): scope the unguarded state-update fork to what the probe showed The previous commit said a mutation against an older checkpoint stores its writes under the shared parent and implied the sibling branch suffers for it. Probing the pinned stack (AsyncSqliteSaver, delta messages channel) shows: - the write does land under the shared parent and joins the write set the parent's other children walk (raw walk for a sibling includes the abandoned head's own write); - but reducer channels are written with Overwrite, which replaces the accumulated value, so the forked branch reads clean and the abandoned branch is unchanged in that shape — #8551 did not reproduce; state the masking mechanism and the upstream tracking issue, not a corruption claim. * docs(checkpoints): fix a frontmatter description that broke the docs build `resume-and-rollback.mdx` shipped `description: Delta mode cannot fork a thread correctly: a forked delta read ...` in the previous commit. The unquoted ": " turns the value into a nested mapping, so Nextra's YAML frontmatter parse failed and the e2e lane could not start its web server (YAMLParseError at line 2, column 14) — the exact failure class this manual warns docs authors about, introduced by me two commits ago. - Replace the colon with an em dash in the English description (the Chinese one uses a full-width colon and was unaffected). - Add tests/unit/content/docs-frontmatter.test.ts: every `src/content/**/*.mdx` frontmatter block must be YAML-parseable, implemented as the one rule that actually breaks it (an unquoted scalar containing ": "), because the unit suite never compiles MDX and only a build or a dev render catches it. Verified the guard fails on the broken line and passes on the fix. Verified with a clean `BETTER_AUTH_SECRET=local-dev-secret pnpm build` on this tree: all pages prerender, resume-and-rollback EN/ZH included. |
||
|
lihongyuan99
|
2a76c9db71
|
docs(harness): make the custom memory storage guide followable (#5844)
`deerflow.agents.memory.storage` does not exist (memory/__init__.py says DeerMem-private symbols are imported from deerflow.agents.memory.backends.deermem.deermem.core.*), the documented class shape cannot be built by create_storage() (it instantiates cls(config)) or called by the memory updater (load/reload/save all pass user_id=, save also expected_revision=), and storage_class is split on the last dot, so the `module:Class` value the guide shows never resolves. memory.mdx also promised a fallback that the code explicitly refuses. |
||
|
Nan Gao
|
bcaf6db36a
|
feat(models): add declarative reasoning capability contract (#5780)
* feat(models): add declarative reasoning capability contract (#5073) Introduce an optional per-model `reasoning:` block beside the legacy `supports_thinking` / `supports_reasoning_effort` booleans. It declares thinking availability (unsupported | optional | required), the accepted effort values with aliases, default, and serialization path, the payload dialect, and the reasoning-history requirement. When present the booleans are derived from it and contradictory profiles fail at config load. `deerflow.models.reasoning` normalizes legacy and declared profiles into one immutable contract and resolves every caller's generic request against it. `create_chat_model` is the single enforcement point: required-thinking models never receive a synthesized disable payload, unsupported effort values never reach the provider, and the dialect synthesizes the on/off payload when no template exists. Legacy profiles keep the historical path byte-for-byte. The lead agent and subagent descriptor resolve the same policy first so run metadata reports effective values. `/api/models` and `DeerFlowClient` project the contract as `reasoning`; the frontend derives mode clamping and effort choices from it through `core/models/reasoning.ts`, and user preferences accept provider-specific effort tokens. The setup wizard's Z.AI GLM-5.3-Flash profile migrates to the contract, restoring its low/high/max effort control. * docs(models): move the reasoning contract guidance into models/AGENTS.md backend/AGENTS.md sits within 46 bytes of the 96 KiB effective-chain hard limit for the middlewares, runtime, sandbox, and subagents guides, so the new section pushed scripts/check_agent_guidance.py over it. The models guide is the nearest file for the factory and the contract module, and its chain has ample room. * fix(models): address review on the reasoning contract - vLLM dialect: the enable path now mirrors whichever chat_template_kwargs switch the template declares, like the disable path already did, so a template using the older `thinking` key no longer also grows `enable_thinking`. - Validate operator-supplied effort values at `effort.path` inside `when_thinking_enabled`, `when_thinking_disabled`, and the `thinking` shortcut against the contract, not only the profile-level value; they are forwarded when the caller chooses nothing, so the "unknown values never reach the provider" guarantee now holds for them too. - Constrain `effort.path` to dotted identifiers and reject single-segment paths that would replace a whole mapping (`extra_body`, `thinking`, `model_kwargs`, `default_headers`, `default_query`). - Z.AI GLM-5.3-Flash profile: `default: high` instead of `max`, because the default also governs summarization, title generation, and subagent calls that never choose an effort; `max` stays selectable. Documented in the config example, the configuration manual, and the design note. * fix(models): close reasoning contract review gaps * fix(models): preserve native Ollama reasoning config * fix(models): keep native reasoning strings and project reasoning into the descriptor - `ModelConfig.reasoning` also accepts the level string ChatOllama takes for gpt-oss style models (`low` / `medium` / `high`); the factory forwards both the boolean and the string on the legacy path, so profiles that passed `reasoning: high` through before the contract field existed keep loading and keep sending it. - The assembly descriptor no longer excludes `reasoning` from `model_parameters`: the native ChatOllama value and a declared contract's dialect/history/effort change the request payload, so they now move the assembly fingerprint. - e2e: the sidecar spec pinned the pre-contract body where a model without effort control still received `reasoning_effort: minimal`; the preset is now dropped for such models (the Gateway discarded it anyway), so the assertion expects the key to be absent. |
||
|
Zhengxu (Joshua) Jin
|
39726573aa
|
docs: fix the documented skill and MCP enable/disable endpoints (#5778)
The docs site pointed at POST /api/extensions/{skills,mcp}/{name}/enable
and /disable, and backend/docs/API.md at POST /api/skills/{name}/enable
and /disable. None of these routes exist. The gateway toggles a skill
with PUT /api/skills/{skill_name} and an MCP server with
PATCH /api/mcp/config, both admin-only, which is also what the
frontend calls.
|
||
|
kbkb628
|
ae1edfdc8b
|
docs(configuration): document the real frontend API base variables (#5829)
The frontend environment table listed NEXT_PUBLIC_API_URL, which no code reads: the validated client schema in frontend/src/env.js only accepts NEXT_PUBLIC_BACKEND_BASE_URL, NEXT_PUBLIC_LANGGRAPH_BASE_URL, and NEXT_PUBLIC_STATIC_WEBSITE_ONLY, and frontend/src/core/config/index.ts builds the API base URL from the first two. Following the old row silently did nothing. Replace it with the two variables that actually override the Gateway and LangGraph base URLs. |
||
|
spud
|
8204dcbb20
|
fix(mcp): invalidate cache only for effective MCP config changes (#5750)
* fix(mcp): avoid cache resets for unrelated config changes extensions_config.json carries MCP servers, skills and middleware settings, but the MCP cache treated any whole-file signature change as MCP state change, so toggling a skill retired every cached tool and closed all pooled sessions. Record a snapshot of the effective MCP slice (enabled servers' parsed config plus mcpInterceptors) and retire MCP state only when that slice changes. Hand the exact config revision to tool discovery so a change during discovery cannot publish tools built from a superseded revision, and add a non-initializing refresh entry point so disabling the last enabled server still converges the cache and pool on the next tool assembly. * docs(mcp): align cache guide and restore error diagnostics Review feedback on #5750 flagged three non-blocking issues: the MCP module guide still described the pre-semantic-snapshot invalidation contract; the tools.py sanitization covered every exception in the MCP block, discarding diagnostics for failures that cannot carry resolved credentials; and _is_cache_stale()'s docstring presented it as a pure predicate despite adopting a new file signature. Narrow the type-only logging to ExtensionsConfig.from_file() and restore logger.exception() for tool caching, pool cleanup and plugin filtering, with a regression test asserting the message and traceback survive. Document the effective-MCP-slice comparison, the stable config handoff to discovery, refresh_mcp_cache_if_active() and the _initialized_without_config handling in the module guide, and note the signature-adoption side effect in the docstring. * fix(mcp): unify config normalization and verify signatures Review follow-up on #5750. - Normalize mcpInterceptors (bare string == single-element list; missing key == empty list) and the type/transport alias through one shared module used by the tools cache, the interceptor builder and the durable-task config snapshot, so equivalent spellings no longer retire tools/sessions or raise McpTaskConfigurationError. - Treat a resolved-path switch as a signal to re-read the effective MCP slice instead of an invalidation reason: an identical slice adopts the new path and signature and keeps pooled sessions. - Refuse to publish or adopt a signature whose sha256 digest is missing ((mtime, size, None) from a file that could not be read). - Sync the English and Chinese MCP user docs with the effective-config comparison, and document that the Gateway admin reset stays unconditional. * docs(mcp): note the unverifiable-signature fail-closed branch A present config file whose content cannot be hashed is treated as stale, unlike a deleted file which keeps the last-known-good cache. Record that asymmetry in the module guide. * docs(mcp): scope the no-hash rationale to never-initialized processes The cost-free early return in refresh_mcp_cache_if_active() only covers a process that never published an MCP cache. A process that did publish one (including an empty cache from the zero-server or LangGraph Studio lazy-init path) still pays one stat+sha256 per tool assembly. Correct the rationale in the docstring and the matching comment in get_available_tools(). |
||
|
kuse
|
6417e802a2
|
docs(configuration): document runtime environment variables in English (#5774)
* docs(configuration): document runtime environment variables in English
* docs(configuration): correct env vars flagged in review
`LOG_LEVEL` is not read by DeerFlow: log verbosity comes from `log_level` in
`config.yaml`. `DEER_FLOW_ROOT` is not a process variable either — it is a
host-side value that only `docker/docker-compose-dev.yaml` interpolates, and it
has no fallback there, so `scripts/docker.sh` supplies one for the
`make docker-*` wrapper only.
Both rows are removed, and the table now lists only variables the process
actually reads: `DEER_FLOW_PROJECT_ROOT`, `DEER_FLOW_HOME`,
`DEER_FLOW_CONFIG_PATH`, `DEER_FLOW_SKILLS_PATH` and `AUTH_JWT_SECRET`. The
config-file home for log verbosity and the host-side home for `DEER_FLOW_ROOT`
are cross-referenced instead.
The zh page carried the same two incorrect rows; it is corrected the same way
so both locales agree.
* docs(configuration): scope the project-root rule to the variable that uses it
The blanket claim that relative path values resolve against
`DEER_FLOW_PROJECT_ROOT` is false for two rows in this table. At this head:
- `runtime_paths.runtime_home()` resolves a relative `DEER_FLOW_HOME` with
`Path(env_home).resolve()`, and `config/paths.py` resolves the state directory
the same way, so both land under the process working directory.
- `AppConfig.resolve_config_path()` builds `Path(os.getenv("DEER_FLOW_CONFIG_PATH"))`
directly, so a relative value is taken from the working directory as well.
`DEER_FLOW_SKILLS_PATH` is the only one that goes through `resolve_path()`, the
single project-root-aware helper (`base or project_root()`).
The prose now names that variable, states that the other two are used as given,
and asks operators to set them to absolute paths so writable state and the
loaded config file land where they expect. The `DEER_FLOW_HOME` row repeats the
absolute-path requirement, `DEER_FLOW_SKILLS_PATH` records the project-root
rule, and the `DEER_FLOW_PROJECT_ROOT` row no longer implies it governs relative
runtime paths in general. The zh page carries the same correction.
|
||
|
Nan Gao
|
5d8a9492ee
|
docs(extensions v0.2.3): cover full-stack plugins and request-scoped run evidence (#5775)
* docs(extensions): add an eleven-page extension developer manual
Add a harness/extensions/ section in English and Chinese documenting the
deerflow-extension-api 0.2.3 contract:
index when to write an extension vs a tool, MCP server or
skill; contribution kinds; loading; failure and
trust model; versioning
quick-start build, test, install and remove a working extension
runtime load sequence, API version rules, scopes and
ExtensionData, fail-open and cancellation, diagnostics
middleware placements and where each lands, scope, ordering,
observe-only wrap hooks, failure isolation
observers task lifecycle, system model calls, agent assembly
and fingerprint, context compaction
services-and-routes service lifecycle, router rejection rules, auth,
principals
run-evidence service and request-scoped readers (#5727), cursor
semantics, deletions, redaction, store differences
plugins experimental full-stack plugins, inline modules and
manifest-based browser assets (#5685)
operations extension manager CLI, sources, upgrade, rollback,
Docker, Helm, recovery
troubleshooting indexed by the exact log and error strings
reference every public name and the contract version history
Also fix stale descriptions: the contribution kinds listed in AGENTS.md
and deerflow/extensions/AGENTS.md (now eight, including plugins), the
run-evidence redaction claim (only auth_token is removed), and the plugin
mount return value in docs/full-stack-plugins.md ({ dispose }, not a
function). Changelog entries added in both languages.
* docs(agents): trim root AGENTS.md to keep inherited chains under the size limit
* docs(extensions): scope the manual to the 0.2.1 contract
Plugins and request-scoped run evidence are not in 2.1.x-dev yet; they
move to a stacked follow-up so this change can be cherry-picked.
* docs(extensions): cover full-stack plugins and request-scoped run evidence
Brings the extension manual up to deerflow-extension-api 0.2.3. Stacked on
the 0.2.1 manual so that part can be cherry-picked to 2.1.x-dev alone.
* docs(extensions): clarify PAT rejection on contributed routes
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Nan Gao
|
7cbcbef52c
|
docs(extensions v0.2.1): add a ten-page extension developer manual (#5769)
* docs(extensions): add an eleven-page extension developer manual
Add a harness/extensions/ section in English and Chinese documenting the
deerflow-extension-api 0.2.3 contract:
index when to write an extension vs a tool, MCP server or
skill; contribution kinds; loading; failure and
trust model; versioning
quick-start build, test, install and remove a working extension
runtime load sequence, API version rules, scopes and
ExtensionData, fail-open and cancellation, diagnostics
middleware placements and where each lands, scope, ordering,
observe-only wrap hooks, failure isolation
observers task lifecycle, system model calls, agent assembly
and fingerprint, context compaction
services-and-routes service lifecycle, router rejection rules, auth,
principals
run-evidence service and request-scoped readers (#5727), cursor
semantics, deletions, redaction, store differences
plugins experimental full-stack plugins, inline modules and
manifest-based browser assets (#5685)
operations extension manager CLI, sources, upgrade, rollback,
Docker, Helm, recovery
troubleshooting indexed by the exact log and error strings
reference every public name and the contract version history
Also fix stale descriptions: the contribution kinds listed in AGENTS.md
and deerflow/extensions/AGENTS.md (now eight, including plugins), the
run-evidence redaction claim (only auth_token is removed), and the plugin
mount return value in docs/full-stack-plugins.md ({ dispose }, not a
function). Changelog entries added in both languages.
* docs(agents): trim root AGENTS.md to keep inherited chains under the size limit
* docs(extensions): scope the manual to the 0.2.1 contract
Plugins and request-scoped run evidence are not in 2.1.x-dev yet; they
move to a stacked follow-up so this change can be cherry-picked.
* docs(extensions): clarify PAT rejection on contributed routes
|
||
|
Nan Gao
|
7d1aa00136
|
docs(subagents): restructure the subagent docs into an eleven-chapter user manual (#5761)
* docs(subagents): restructure the subagent docs into an eleven-chapter user manual
Replace the single harness/subagents.mdx page (en + zh) with a
harness/subagents/ section of eleven chapters per language:
1. index concepts, delegation flow, inheritance, terminology
2. quick-start Ultra mode, task card, results, stop behaviour
3. catalog built-ins, config.yaml / managed sources, precedence,
Custom Agent delegation scope, ACP agents
4. delegation task parameters, snapshot context, acceptance criteria,
batch_task, skills / MCP / uploads inside a subagent
5. results terminal statuses, stop_reason, report contract,
receipt verification, acceptance checklist, ledger
6. limits every limit with key / default / range / behaviour,
runaway guards, subagent_runtime capacity
7. sandbox leases, per-subagent shell sessions, MAX_SHELL_SESSIONS,
middleware chain, execution isolation
8. observability task card, SSE + persisted events, metadata keys,
token attribution, Langfuse, trace ids, batch API
9. troubleshooting symptom-indexed FAQ with the fixing PRs
10. developers create_deerflow_agent, SubagentRuntime, contracts,
extensions, security boundaries, background registry
11. reference config keys, context keys, tool signatures, enums,
events, routes, June-September 2026 change log
The old page becomes the section index (asIndexPage), so existing links to
/docs/harness/subagents keep working; the two anchor links in
middlewares.mdx now point at limits#runaway-guards. Every chapter compiles
with @mdx-js/mdx and the docs link tests pass. Changelog entries added in
both languages.
* docs(subagents): fix the docs build and align the manual with the code
Remove the `index` key from both subagents `_meta.ts` files. The index page
is marked `asIndexPage`, so Nextra treats it as the folder itself; listing it
as a child failed `_meta` validation and returned 500 for every docs page.
Correct claims that disagreed with the backend, in both languages:
- GET /api/subagents is open to all users; only writes need an admin
- subagents use their own subagents.token_budget, not the Lead Agent's
- warn_threshold injects a model-visible warning, not just a log line
- [SUBAGENT LIMIT REACHED] and subagent_limit_capped fire only when the
per-run total was already exhausted before the response
- per-response concurrency defaults to subagent_runtime.max_running
- batch tools are not registered when a supplied runtime lacks a batch
service
- ask_clarification / present_files are default denies that a config.yaml
agent can lift
- smaller fixes to result text formats, event payloads, batch item states,
MAX_SHELL_SESSIONS handling, and UI labels
The changelog entry now says only page-level links survive the split.
|
||
|
JY-M666
|
1e6fda4ec2
|
docs: add llama.cpp local model configuration (#2931) (#5688) | ||
|
xbzz1018
|
6fae79047c
|
docs: clarify upgrade workflow (#5654)
* docs: clarify upgrade workflow * docs: fix upgrade section placement |
||
|
Wenchao An
|
a246c928e9
|
feat(frontend): move capability management out of Settings (#5468)
* feat(frontend): move capability management out of settings * fix(frontend): address capability center review feedback |
||
|
wutongyuonce
|
4470932118
|
feat(extensions): allow constructor kwargs on config-declared middlewares (#5312)
* feat(extensions): allow constructor kwargs on config-declared middlewares
extensions.middlewares entries may be a class-path string or {class, kwargs}.
String entries keep the zero-argument constructor. Unknown fields and blank
class paths fail at config validation. Constructor errors still fail at
agent creation.
Fixes #5311
* fix(extensions): coerce middleware kwargs to JSON types
YAML timestamps became datetime objects while JSON kept strings, so
constructors and to_file_dict() json.dump saw different types. Validate
kwargs as JSON types at config load, stringify dates, reject NaN and
other non-JSON values, and cover the raw-dict loader branch.
* style(extensions): wrap middleware Field description for ruff E501
make lint failed: the middlewares description was 289 chars (limit 240).
Wrap it and run ruff format on the two files this PR last touched.
* docs: compact configured middleware guidance to satisfy size limit
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Yusuf Gürdoğan
|
23bd76046a
|
feat(community): add Sofya web search provider (#5239)
* feat(community): add Sofya web search provider Add a community provider backed by Sofya (https://sofya.co). Its search endpoint returns the content of the result pages, not only their snippets, and its fetch endpoint returns a page as markdown. Both are plain JSON over HTTP, so this needs no extra Python package (uses httpx, already a dependency). Changes: - backend/packages/harness/deerflow/community/sofya/__init__.py - backend/packages/harness/deerflow/community/sofya/tools.py Implements web_search_tool and web_fetch_tool using httpx. API key is read from the config.yaml `api_key` field or the SOFYA_API_KEY env var. Follows the same interface and output shape as the existing ddg_search and serper providers, including the max_results parameter with config override and the structured "No results found" error. - backend/tests/test_sofya_tools.py Unit tests covering API key resolution, config overrides, result mapping, time range, HTTP errors, empty results, and fetch failures. - config.example.yaml: add commented-out Sofya web_search and web_fetch examples alongside the other providers - .env.example: add SOFYA_API_KEY placeholder - backend/docs/CONFIGURATION.md: list Sofya under web_search, web_fetch and the environment variables * fix(sofya): honor caller max_results, validate search_depth, join time_range contract test - Caller-supplied max_results now wins; config is used only when the argument is omitted, matching GroundRoute. - search_depth is clamped to basic/snippets; an unsupported value logs a warning and falls back to basic. - Sofya added to the shared time_range schema contract test. * fix(sofya): cap per-result content so a search stays inline An unbounded search payload (up to 20 read pages) crossed the tool output budget middleware's externalize_min_chars threshold, which replaces the result list with a file reference. Cap each result's content at contents_max_characters (default 2000, 0 disables), matching Exa's config key. Five capped results stay under the 12000 char threshold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5 * fix(sofya): list Sofya in the recency contract, coerce non-string content _clip subscripted its input, so a non-string content or description from the API raised TypeError instead of degrading. Coerce to text first, the way _sofya_post and _response_results guard the shapes around it. Also add Sofya to the Web Search Recency section in backend/AGENTS.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5 * fix(sofya): coerce web_fetch content, list sofya in the tools guide, add changelog web_fetch sliced its content the same way web_search did before the last push: a truthy non-string from the API passed the falsiness guard and then raised TypeError. Reuse _clip, keeping the `or ""` so empty content still reports "No content found". Also add sofya to the community provider inventory in packages/harness/deerflow/tools/AGENTS.md and an [Unreleased] changelog entry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5 * docs(zh): add the missing InfoQuest and Firecrawl web_fetch tabs The ZH web_fetch tab list named five providers where EN names seven. Both tabs mirror their EN counterparts, so the two locales list the same web_fetch providers again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5 --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Wu Shuwen
|
364dad06aa
|
docs: update middleware contribution examples (#4945)
* docs: update middleware contribution examples * docs: clarify middleware registration paths * docs: clarify middleware injection scope * docs: clarify middleware state updates * docs: clarify middleware state updates * docs: clarify middleware pipeline placement * docs: complete middleware order guidance * docs: align middleware guard conditions * docs(middleware): name runtime sanitization order * docs: pin middleware runtime order * docs: clarify middleware assembly paths * docs: clarify middleware anchor scope |