mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-09 04:46:17 +00:00
185 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
YxinMiracle
|
b8ab097ca2
|
fix(models): support official DeepSeek managed profiles (#5718)
Co-authored-by: YxinMiracle <“939157765@qq.com”> |
||
|
spud
|
61a99a1e8a
|
fix(sandbox): enforce bash command timeouts in AIO sandbox (#5634)
* fix(sandbox): enforce AIO shell command timeouts Map Sandbox.execute_command(timeout=T) onto the AIO legacy shell as a server-side hard_timeout=T plus a bounded host request (ceil(T+5)s, max_retries=0), preserve and render the upstream status (hard_timeout -> terminated + Exit Code: 124; no_change_timeout -> may still be running), keep no_change_timeout from preempting hard_timeout, and restrict ErrorObservation replay to completed results. * fix(sandbox): contain ambiguous AIO command outcomes Treat a transport timeout and statuses terminated/no_change_timeout/unknown as ambiguous: never replay the current command, fence the scoped or recovery session generation with bounded best-effort cleanup, and never adopt a replacement session for those statuses. hard_timeout and completed keep the session reusable. Align the env-bearing bash.exec path with the same contract and make its retry result authoritative. * fix(sandbox): wire AIO command timeout from provider config Restore bash_tool's non-local behaviour (other providers keep their own defaults) and let AioSandboxProvider read sandbox.bash_command_timeout as the sandbox's default_command_timeout at every construction site, with explicit per-call timeouts still winning. Widen the config field to float with allow_inf_nan=False and read the provider value with the module fallback so partial-config providers keep working. * docs(sandbox): document AIO command timeout semantics Document the provider-scoped deadline (LocalSandbox/AioSandbox/OpenSandbox wire it; other providers keep their defaults), the semver-vs-frozen-:latest image behaviour, the no-replay guarantees, and trim the sandbox guidance back under its size budget. * docs(sandbox): scope the wedge-resistance claim to command requests willem-bd's review on #5634 reproduced that list_dir and both create_session call sites still make unbounded SDK requests while holding the same sandbox lock, so the guide must not read as if every AIO request is bounded. State that the bounded T+5s host wait applies to command requests and name session-creation and file/list RPCs as not covered; bounding those remains a separate operation-deadline/control-plane follow-up. * fix(sandbox): make list_dir honor the shell generation fence list_dir previously always targeted the implicit persistent shell, so after an ambiguous command outcome fenced that generation (#5634) a listing could re-enter it. Route default-shell target selection through _ensure_default_shell_session_id so both execute_command and list_dir reuse the explicit recovery session, creating one only when the implicit shell is fenced. No list_dir deadline, hard_timeout, request budget, retry, or status semantics are added; those remain scoped to #5644. |
||
|
Wenchao An
|
e2f19d8335
|
feat(plugins): full-stack plugin APIs and bookmarks (#5647)
* feat(plugins): add full-stack contributions and bookmarks example * ci(plugins): provision bookmark gateway for browser tests * fix(plugins): authenticate module downloads through configured backend * fix(plugins): isolate contributions and localize extension UI * fix(plugins): preserve bookmark agent routing and contain async callbacks * fix(plugins): pin durable batch workers to app extension snapshots |
||
|
asts
|
0ff2e9ddcf
|
fix(mcp): honor configured stdio working directories (#5643)
* fix(mcp): honor configured stdio working directories * fix(mcp): preserve defaults for empty working directories Treat empty stdio cwd values as omitted, including unresolved environment references. Add real subprocess regressions for discovery and pooled-call defaults, plus direct connection-dictionary coverage. |
||
|
NEEDI
|
0627de2bcc
|
fix(persistence): apply postgres schema to sync agent stores (#5678)
Co-authored-by: NEEDI <298523066+sherxlg-gif@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Wenchao An
|
1f437f86c6
|
feat(agents): persist default knowledge scopes for custom agents (#5579)
* feat(agents): persist default knowledge scopes for custom agents * style: format agent knowledge guidance * fix(i18n): clarify default knowledge reset hint * fix(knowledge): preserve retries for initially unbound agents |
||
|
Xuehao Xu
|
8ef58eaa90
|
feat(models): manage shared models from Settings (#5596)
* feat(models): add admin UI for shared model management * docs(gateway): keep model guidance within size budget |
||
|
Wenchao An
|
42334f26d7
|
feat(capabilities): unify catalog, plugin configuration and agent selection (#5497)
* feat(capabilities): unify catalog, plugin configuration and agent selection * fix(capabilities): address review isolation, validation and demo issues * fix(capabilities): preserve concurrent selections and guide launcher repair |
||
|
xiaodu55
|
2ff006b0c0
|
feat(middleware): add deterministic PII redaction for model-bound context (#5527)
* feat(middleware): add deterministic PII redaction for model-bound context * fix(middleware): claim national IDs before cards, redact Command results, preserve ToolMessage fields - Reorder detectors so checksum-gated national IDs run before the credit-card detector; an 18-digit resident ID whose digit run also passes Luhn is no longer mislabeled [CREDIT_CARD_n] (review finding, reproduced at 0a2a9d0) - Redact ToolMessages carried in Command.update.messages, mirroring ToolResultSanitizationMiddleware's dc_replace pattern - Rebuild redacted ToolMessages via model_copy so artifact and response_metadata survive - Extend the numbered middleware chain in agents/middlewares/AGENTS.md * fix(middleware): span one redactor per Command result; refresh stale AGENTS.md entry range - Placeholder numbering now continues across every ToolMessage carried in a single Command result (one _Redactor per _redact_result call) instead of restarting per message - The renumbered AGENTS.md chain still referenced entries 9-12 in the ToolReceiptMiddleware entry; it now reads entries 10-13 * docs(agents): trim PiiRedactionMiddleware entry to fit the AGENTS.md chain budget The main merge (fb36e0e) pushed the effective middlewares chain to 98341 bytes, 37 over the 98304 hard limit checked by agent-guidance (AG002). Compress the entry while keeping the load-bearing facts: config gate, both interception points incl. Command coverage, detector order rationale, per-result numbering continuity, irreversibility, memory follow-up. * fix(middleware): redact compaction input and reinjected summaries; harden detectors Review round 3 on #5527: - [P1] SummarizationMiddleware invokes its summary model directly from before_model, outside PiiRedactionMiddleware's wrap_model_call, so raw thread state reached the summary model and reinjected summaries carried raw PII into model-bound context. Add a shared redact_text() seam: the compaction prompt is redacted in _build_summary_prompt (app_config already flows into the middleware) and DurableContextMiddleware redacts summary_text at reinjection via a new pii_redaction_config knob wired at both assembly sites. - [P2] CUIT is 2+8+1 digits, not 2+10+1. - [P2] Digit-anchored patterns use digit-aware lookarounds instead of Unicode \b, which CJK characters defeat (身份证110105… / 手机号138…). - [P2] The international-phone pattern no longer treats newlines as separators, so a candidate cannot swallow the following numeric field and then fail validation as a whole. * fix(pii): redact title input and reserve summary placeholders --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
zhangwei-way
|
b6503e9a35
|
feat(knowledge): add per-message RAGFlow retrieval scope (#5238)
* feat(knowledge): integrate RAGFlow retrieval and management * test(knowledge): cover merged listing tool * feat(knowledge): add per-message retrieval scope * chore(docs): remove unrelated document * docs(knowledge): add interaction screenshots * feat(knowledge): simplify scope selector trigger * docs(knowledge): refresh selector screenshot * feat(knowledge): defer standalone management * docs(knowledge): show chat-only scope UI * fix(knowledge): honor scope on clarification replies * fix(knowledge): harden scoped replay validation * docs(knowledge): clarify replay scope precedence * fix(knowledge): keep provider settings on tools * fix(config): preserve tools-only knowledge settings * fix(knowledge): submit custom assistant identity * refactor(knowledge): trim PR scope changes * fix(knowledge): sanitize document scope display * feat(knowledge): enable scope selection in main chat * fix(knowledge): emphasize active scope icon without button frame * fix(knowledge): close context scrubbing and refresh e2e checks * fix(knowledge): preserve idempotent canonical retries * fix(knowledge): accept promptless conversation runs * style(knowledge): format backend regression tests * chore(knowledge): trim PR scope and fix frontend format * fix(knowledge): remove shared-scope notice * fix(knowledge): remove scope persistence notice * docs(knowledge): include main chat in catalog scope * fix(knowledge): preserve scope recovery and upgrades * fix(config): preserve LightRAG knowledge upgrades --------- Co-authored-by: foreleven <for-eleven@hotmail.com> |
||
|
Zeren Wang
|
a58ab484a6
|
feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash (#5443)
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md (issue #5160, tracker #5129) in the slice order of the spec's §16. Slices: - A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY admission pinning (both server-owned sets + worker hoist); latest-only request-scoped <project> block via DynamicContextMiddleware wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID prefix + marker + provenance, never persisted); journal audit fingerprints; Instructions tab. - B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with locked check-and-set; hash-qualified immutable shelf storage with Paths helpers; upload/list/content/delete-to-trash routes; project delete trashes the shelf in-transaction; request-scoped bounded <documents> index with honest count/shown + actionable overflow note; list_project_documents/read_project_document tools registered only on pinned runs; PAT allowlist + drift guards; blocking-IO anchors. - C: shared thread-upload ingestion service (uploads router refactored to parity); POST from-thread with provenance; attach-to-thread with lock-staged copy (archived source allowed); read-only thread-files view with per-group truncation reporting. - D: restore (restored/merged/not_found/no_target/content_missing; no file moves), purge (continuous row lock across unlink/delete/commit, retryable on FS errors), retention sweep (lazy + startup, 24h orphan guard, row-side reconciliation never deletes). - E: Documents tab (shelf + conversation-files browser, provenance, archived banner, content-missing rows), /workspace/trash route, sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks + specs. Review hardening folded in (10 rounds, all with tests): - force active shelf content (HTML/XML family) to download; nosniff on artifact + content responses; unified unsandboxed-iframe PDF preview (fixes the pre-existing Chromium sandbox blank in the artifact viewer) - scope document trash to the URL project under the document lock - atomic no-overwrite filename reservation for ALL ingestion (seeded claims + os.link commit with suffix retry; same-name re-upload now unique-names instead of replacing); hidden staging only, no visible placeholders; lease cleanup on setup failure - serialize conversion under the document lock with post-lock active revalidation; drain locked filesystem work on cancellation; preserve bytes when an insert's commit state is uncertain (including trashed rows) - original-integrity checks before serving text or cached conversions; content_missing surfaced in list responses (UI reads the flag, no 409-probe); downloads always serve original bytes - bounded streaming document reads with cached char counts; shelf limits declared in middleware release identity - thread-root confinement for from-thread sources; config fallback rejects fractional/infinite values; composer counts staged attachments; pending attachments persist until submission or removal; in-flight instruction/rename edits survive save refetches; shelf and trash pagination; conversation-file and thread-files pages stay subscribed to refetches Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md contracts, config.example.yaml projects block. Review follow-ups (head b4807477 → this revision): - The trash retention sweep is split so repeated lazy triggers stay bounded: the indexed expiry purge still runs on every trigger (GET /api/trash/documents, POST /api/trash/purge) while the O(all rows + all files) reconciliation is throttled to one run per user per 15 minutes (process-local, per-user window). The startup sweep now runs as a background task instead of blocking gateway readiness, and shutdown awaits it (bounded). - The export scrub (stripInternalMarkers) is fence- and indentation-aware like the render path, so a pasted, fenced <project>/<documents> snippet survives markdown export while real injected blocks (never fenced) are still removed. Fence regexes moved to a dependency-free leaf module to avoid the messages↔streamdown import cycle. - The artifact viewer's PDF iframe no longer carries an added title attribute (the upstream e2e contract locates it via :not([title])), and the upstream artifact-preview spec now pins the new contract: PDFs render unsandboxed, images keep sandbox="". * fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title - Shutdown cancelled only the shield around the background startup sweep, so an all-users reconciliation that outlived the 5s budget kept walking rows and files while the document repo and DB engine were disposed underneath it. The wait now lives in `_shutdown_startup_trash_sweep`, which cancels the task and drains it before worker exit: the shield keeps the wait bounded, the cancel makes it final (CancelledError lands at the sweep's next await, and `_run_startup_trash_sweep` only catches `Exception`, so nothing swallows it). - The browser-preview iframe lost `title={getFileName(filepath)}` in the previous fix round, leaving the PDF frame without an accessible name while its siblings keep theirs. Restore it (WCAG frame titles), assert it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]` instead of `iframe:not([title])`. * fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel `Task.cancel()` returns False when the sweep already finished inside the window between the deadline firing and the cancel, so the shutdown log claimed a cancellation that never happened. Branch on that outcome: the warning stays for a real cancel, a late finish is logged at info, and both paths still reap the task before worker exit. * fix(projects): round-4 review — make Empty trash delete what it confirms `POST /api/trash/purge` only ran the retention sweep, and the sweep's candidate selection is age-gated, so a freshly trashed document survived "Empty trash" even though the confirmation promises that every listed document is permanently deleted. With one trashed row the route answered `{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps expired rows before listing, so the visible rows were normally ineligible for the action by construction. Empty trash now drives `purge_all_trashed`: the caller's trashed rows (`list_all_trashed`, no age filter) each go through the same guarded, row-locked `purge` as the single-document delete — bytes first, then the row, in one transaction — so a row restored mid-flight is skipped instead of force-deleted, and an unlink failure rolls that row back and answers 500 with a retryable message. Retention expiry stays where it was: the sweep's `purge_candidates` is now the only age-gated selection, and the lazy retention sweep still runs on the listing and at startup. Tests: the router suite replaces the retention-gated expectation with the reviewer's repro (fresh row purged, bytes unlinked, shelf and other users' trash untouched, a failing unlink stays retryable and 500); a blocking-I/O anchor drives the new entry point through the offload; the mocked e2e covers the action end to end; a new real-backend spec performs it against the real gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE and the phase-2 design docs (en+zh) state the age-independent contract. |
||
|
yang rui
|
3dc895df4d
|
feat(models): pace shared RPM budgets before dispatch (#5432)
* feat(models): add shared RPM admission queues * fix(models): address admission pacing review feedback * docs: simplify request admission quick start guidance --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Beautyl0ve
|
80f13935c2
|
feat(agents): allow custom agents to disable memory (#5167)
* feat(agents): allow custom agents to disable memory Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> * fix(agents): honor memory opt-out during compaction Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> * fix(runtime): preserve agent binding across state rewrites * fix(client): apply named-agent memory policy * fix(agents): address memory policy review feedback Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> * fix(agents): address remaining memory opt-out reviews Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> --------- Signed-off-by: Beautyl0ve <74452755+Beautyl0ve@users.noreply.github.com> Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Nan Gao
|
cf556fa9d4
|
feat(agents): elide superseded write_file payloads from model-bound requests (#5374)
* feat(agents): elide superseded write_file payloads from model-bound requests Step 2 of #5328. After a successful write_file the file on disk is the source of truth, and the read-before-write gate forces a read_file before the next modification of that path, so once a later successful read or write of the same path exists the historical `content` argument is redundant with it. Long report-writing runs (append-in-chunks) therefore carried every section twice, once as the write argument and once as the following read output, until summarization compacted the whole turn. - ToolOutputBudgetMiddleware's model-call hooks now replace such superseded content with a short deterministic placeholder pointing at read_file, in the model-bound request only: state["messages"], checkpoints, receipts, loop detection, and the run journal keep the original arguments, and nothing is externalized to disk. The newest `keep_recent_writes` successful writes (default 1) always stay visible; str_replace payloads are never touched; a same-turn read never supersedes (parallel calls run in no fixed order); only results stamped deerflow_tool_meta.status == "success" count, so failed, gate-blocked, partial, or unstamped writes are never candidates. - New `tool_output.elide_superseded_writes` (default on), `tool_output.superseded_write_min_chars` (default 2000), and `tool_output.keep_recent_writes` (default 1); config_version 41 -> 42 in config.example.yaml and the Helm chart. - The per-occurrence call/result pairing the gate introduced in #5329 moves into the shared `tool_call_args.pair_tool_call_results` helper so both policies pair the same way; the gate now uses it. * fix(agents): scope tool-call result pairing to the issuing turn Review finding on #5374 (P2): pair_tool_call_results consumed results from a history-wide per-id queue, so an interrupted write_file with no result whose tool-call id a later turn reused inherited that later call's success. With the default elision the unconfirmed draft was then replaced by a placeholder claiming the write succeeded, and the gate's blocked-call pairing had the mirror-image hole. Pair results the way DanglingToolCallMiddleware does: walk in document order, open each AIMessage's calls, and let a ToolMessage answer only a still-open call of the most recent preceding AIMessage. A result never answers a call from an earlier turn, so the interrupted call stays unanswered (never a candidate, never labeled blocked) and stray or duplicate results are ignored. Regressions cover the helper, the superseded-write policy, and the gate. * fix(agents): never rewrite tool-call ids duplicated within one AIMessage Review finding on #5374 (P2): the policies select calls per occurrence, but every provider surface is addressed by tool-call id, so when a malformed provider payload repeats an id inside one assistant turn the rewriter could only replace all of its occurrences at once. A failed write_file sibling then took on the superseded successful call's path and elided content and was presented as a success; the gate's blocked-call elision had the mirror-image hole (a successful sibling rewritten into the blocked call). rewrite_messages_tool_call_args now never offers an id that repeats within its message to the selector and leaves those calls untouched on every surface. Both policies are covered by the shared helper; regressions cover the helper, the superseded-write policy, and the gate. * fix(agents): skip unhashable tool-call ids in the duplicate-id guard Review finding on #5374 (round 3): _duplicated_call_ids fed every id into a Counter before the string guard, so a list or dict id from a malformed provider payload raised TypeError out of wrap_model_call and failed the whole model call whenever the history also held a rewrite candidate. The pre-PR loop and pair_tool_call_results skip such ids; only this helper regressed. Count non-empty string ids only, and pin it with regressions for the helper, the superseded-write policy, and the gate. * docs(agents): keep middleware guidance within size limit --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Xuehao Xu
|
56540fab01
|
fix(gateway): make recursion limit configurable (#5390)
* fix(gateway): make recursion limit configurable * docs: keep backend guidance within inherited size budget * fix(gateway): address recursion limit review feedback * docs(gateway): clarify recursion default scope --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Wenchao An
|
1b76ab9060
|
feat: add opt-in task notes and compacted history recall (#5382)
* feat: add opt-in task notes and compacted history recall * fix: validate task continuity state and preserve user answers Honor explicit opt-out, preserve clarification replies and capture failure statuses, validate notebook writes, and clear branch archive references. Update the config version and audit optional LLM credentials, with regression and integration evidence. * fix: align Helm config version with task continuity schema * fix: preserve mixed task history and declare continuity policies * fix: recover malformed history and evict archives atomically |
||
|
Hyeonsang Cho
|
bc4a33aba7
|
fix(skills): stop persisting resolved secrets when toggling skills (#5357)
Toggling a skill wrote resolved secrets into extensions_config.json. The Gateway skill toggle and DeerFlowClient.update_skill loaded the file with ExtensionsConfig.from_file(), which replaces every "$VAR" string with the environment value (and an unset variable with ""), then serialized that model back through to_file_dict(). A "$GITHUB_TOKEN" reference was persisted as the plaintext token, and an unset reference was erased for good. DeerFlowClient.update_mcp_config had the same flaw for every key other than mcpServers. Every writer now does a raw read-modify-write, the way the MCP router already did: read_raw_extensions_config reads the on-disk JSON, set_raw_skill_enabled changes only the target entry, and validate_raw_extensions_config checks the candidate the way the runtime will load it before the atomic write. When the file does not exist yet, the Gateway seeds it with the cached skill states only, never the resolved cached model. The MCP router's raw loader and candidate validation delegate to the same helpers, so the rule lives in one place, and to_file_dict() is removed so the unsafe serialization has no entry point left. Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
yang rui
|
ac2b6415ea
|
feat(agents): support Unicode display names for custom agents (#5324)
* feat(agents): support Unicode display names for custom agents * fix(agents): preserve and validate Unicode display names * fix(agents): tolerate invalid stored labels and reject invisible names * fix(agents): identify agent in invalid display name warning * style(frontend): format agent display name fallback --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
wutongyuonce
|
4470932118
|
feat(extensions): allow constructor kwargs on config-declared middlewares (#5312)
* feat(extensions): allow constructor kwargs on config-declared middlewares
extensions.middlewares entries may be a class-path string or {class, kwargs}.
String entries keep the zero-argument constructor. Unknown fields and blank
class paths fail at config validation. Constructor errors still fail at
agent creation.
Fixes #5311
* fix(extensions): coerce middleware kwargs to JSON types
YAML timestamps became datetime objects while JSON kept strings, so
constructors and to_file_dict() json.dump saw different types. Validate
kwargs as JSON types at config load, stringify dates, reject NaN and
other non-JSON values, and cover the raw-dict loader branch.
* style(extensions): wrap middleware Field description for ruff E501
make lint failed: the middlewares description was 289 chars (limit 240).
Wrap it and run ruff format on the two files this PR last touched.
* docs: compact configured middleware guidance to satisfy size limit
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Nan Gao
|
3f0b6ecc81
|
feat(agents): elide blocked write payloads from model-bound requests (#5329)
* feat(agents): elide blocked write payloads from model-bound requests A write_file / str_replace call rejected by the read-before-write gate never runs, yet its payload (up to 80 KB for a non-append write, unbounded for append) stayed verbatim in every later model request: nothing in the chain rewrites AIMessage tool-call arguments, and ToolOutputBudgetMiddleware only budgets ToolMessage output. The gate demands a re-read plus a fresh call, so the model re-emits the content anyway and the original is pure dead weight. - ReadBeforeWriteMiddleware stamps `deerflow_write_block` on the blocked ToolMessage and, in wrap_model_call, replaces the paired call's payload fields (content / old_str / new_str) with a short deterministic placeholder in the model-bound request only. state["messages"], receipts, loop detection, and the run journal keep the original arguments; nothing is externalized to disk, since a file reference to content the model must re-derive after reading the target would only invite bypassing the gate. - New `tool_call_args` helper rewrites every provider surface together (structured tool_calls, raw additional_kwargs.tool_calls, tool_use content blocks, tool_call_chunks) so strict providers never see them disagree; the gate only supplies the policy (which calls, what placeholder). - `read_before_write.elide_blocked_payloads` (default on) and `read_before_write.elide_min_chars` (default 2000) configure it; the runtime builder passes the config through and the middleware declares it via release_policy_parameters. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(agents): condense middleware guide entry 11 to fit the guidance budget The agent-guidance CI check failed: the effective AGENTS.md chain for agents/middlewares was 99673 bytes against a 98304-byte hard limit. The chain already sat at 98459 on main, so the ReadBeforeWrite entry could not grow. Rewrite entry 11 so it states the same facts (gate, lock scope, fail-open, authorization scope, blocked-payload elision, shared tool_call_args helper) in 1229 bytes instead of 2640; the chain is now 98262 bytes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(config): bump config_version for the read_before_write elision keys Review follow-ups on #5329: - `read_before_write.elide_blocked_payloads` / `elide_min_chars` are new user-settable YAML keys, i.e. a config schema change, so bump `config_version` 40 -> 41 in config.example.yaml; without it an existing config.yaml gets no outdated-config warning and `make config-upgrade` has nothing to signal. - Say in the `elide_min_chars` description (and the example comment) that the threshold and the placeholder's size figure are Python character counts, not tokens: the same value spans roughly 3-4x in real context cost between ASCII and CJK text. - The builder wiring test now asserts only the wired `elide_min_chars` value instead of the whole `ReadBeforeWriteConfig` dump, so future knobs do not have to edit an unrelated test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(helm): bump chart config_version to 41 validate-chart's config_version drift check failed after config.example.yaml moved to 41 in ef9ee267. Bare bump of the chart's embedded `config:` block and the README example; the chart does not mirror the read_before_write section, so no field changes are needed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(agents): rewrite Responses and v1 content-block arguments too Review finding on #5329 (P2): the content rewriter only handled Anthropic `tool_use` blocks. With `use_responses_api=true` and `output_version='responses/v1'`, AIMessage.content carries `function_call` blocks whose `arguments` still hold the full write payload, and langchain_openai's Responses input builder emits that block instead of the rewritten structured call whose `call_id` it already carries. Standard `v1` `tool_call` blocks likewise keep `extras.arguments`, which the v1->Responses translator prefers over the structured args. So the blocked payload was still sent on every later Responses API request. `tool_call_args` now rewrites every content dialect that carries its own copy of the arguments: Anthropic `tool_use` (input, drop partial_json), Responses `function_call` (arguments, matched by call_id, `fc_...` item id and status preserved), and v1 `tool_call` / `tool_call_chunk` (args plus `extras.arguments`). Tests assert against the real adapter serializers: `_construct_responses_api_input` for responses/v1, v1, and v0 messages, `_convert_message_to_dict` for chat completions, and Anthropic `_format_messages` for native and v1 content, plus an end-to-end probe through the gate's wrap_model_call. * fix(agents): pair blocked writes per call occurrence and defeat Responses chaining Two review findings on #5329: - Tool-call ids may repeat across assistant turns (DanglingToolCallMiddleware pairs them with per-id queues). The gate matched blocked results against a history-wide id set, so a successful write sharing an id with a later (or earlier) blocked one also lost its payload and was labelled as blocked. `_blocked_call_occurrences` now pairs ToolMessages with call occurrences the same FIFO-per-id way and the selector keys on (message, call id). - With `use_previous_response_id`, the OpenAI adapter sends only the messages after the last AIMessage carrying a `resp_` response id and lets the server rebuild the rest from its stored copy, which still holds the original arguments and cannot be edited; every later response chains back to it. `rewrite_messages_tool_call_args` now drops every `resp_` id from the model-bound copy whenever it rewrote anything, so the adapter replays the full rewritten history (the `use_previous_response_id=False` request shape). OpenAI bills chained input tokens as input either way, so replay costs no more; the state keeps its ids. Tests cover success-before-block and block-before-success histories through the Chat Completions serializer, and chaining through `ChatOpenAI._get_request_payload` with `use_previous_response_id=True`: unrewritten history chains and omits the call, rewritten history is replayed with the placeholder and no `previous_response_id`. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
PeaceMaker-best
|
36ce7590b7
|
fix(agents): isolate loop detection state by run (#5344)
* fix(agents): scope loop detection state per run * fix(agents): harden loop scope fallback * docs: move loop lifecycle detail out of inherited guidance --------- Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
wutongyuonce
|
48a8978b7b
|
feat(scheduler): add interval schedule type (#5291)
* feat(scheduler): add interval schedule type Allow scheduled tasks to fire every N seconds from last dispatch, not only wall-clock cron or a single run_at. Cadence is UTC now+N with no missed-beat catch-up, bounded by min_once_delay_seconds and 30 days. * fix(scheduler): let interval tasks create, edit, and keep next run Create/edit now keep every_seconds. Unchanged interval spec no longer resets next_run_at, including timezone-only PATCH. * fix(scheduler): keep non-minute intervals on edit Stop rounding every_seconds to whole minutes in the form. Values that are not whole minutes or hours now use a seconds unit so edit/duplicate round-trips the stored cadence instead of rewriting it and resetting next_run_at. Document that min_once_delay_seconds is also the interval floor. * fix(scheduler): clamp interval seconds to the default 60s floor The new seconds unit allowed 1–59, which the API rejects under the default min_once_delay_seconds. Clamp the form to >= 60 and show the floor next to the preview. Also mention interval in the scheduler field_doc, matching config.example.yaml. * fix(scheduler): do not clamp interval amount while typing Keystroke clamp made 90 become 9 -> 60, then 600, and backspace could not leave 60. Keep the raw field text and apply the 60s floor on blur and emit only. * test(scheduler): cover interval input editing * fix(frontend): preserve saved interval cadence until edited * style(tests): format scheduled task router tests --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Wu Shuwen
|
364dad06aa
|
docs: update middleware contribution examples (#4945)
* docs: update middleware contribution examples * docs: clarify middleware registration paths * docs: clarify middleware injection scope * docs: clarify middleware state updates * docs: clarify middleware state updates * docs: clarify middleware pipeline placement * docs: complete middleware order guidance * docs: align middleware guard conditions * docs(middleware): name runtime sanitization order * docs: pin middleware runtime order * docs: clarify middleware assembly paths * docs: clarify middleware anchor scope |
||
|
Aari
|
0f7d8709d3
|
feat(sandbox): add controlled egress with approvals (#5152)
* feat(sandbox): add controlled egress approvals * Apply batched suggestions from code review * fix(sandbox): harden restricted network policy * fix(sandbox): harden denied egress handling * fix(sandbox): isolate network proxy sidecar * chore: retry sandbox image smoke * fix(sandbox): close remaining network policy gaps * fix(sandbox): harden relay token rejection * fix(sandbox): fence incompatible policy replacement * fix(sandbox): replace containers across network modes * fix(sandbox): close remaining lifecycle gaps --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Tu Naichao
|
ae82f426bf
|
fix(summarization): stop fraction triggers from crashing the agent build (#4901)
* fix(summarization): resolve fraction triggers from declared context_window, degrade instead of crashing the agent build A fraction trigger/keep clause requires profile["max_input_tokens"], which any third-party OpenAI-compatible model lacks, so SummarizationMiddleware construction raised ValueError out of create_summarization_middleware and failed the whole agent build (#3103). - factory: translate a declared model context_window into the langchain profile (metadata-only, never reaches the provider payload); explicit caller/override profiles win - summarization factory: drop unusable fraction trigger clauses (absolute clauses survive), fall a fraction keep back to the messages default, and disable compaction with an actionable warning only when no usable trigger clause remains — the agent build never dies from summarization config - docs: config.example.yaml, ModelConfig.context_window, summarization.md * refactor(summarization): share the default keep constant with the fraction fallback The fraction-keep degradation fallback hardcoded ("messages", 20), duplicating SummarizationConfig.keep's default_factory literal. Move the value to a shared DEFAULT_KEEP constant so the two cannot drift apart. * fix(summarization): keep trigger-null + fraction-keep constructing after degradation A trigger of None with a fraction keep hit the all-clauses-dropped branch (has_usable_trigger=False) and disabled compaction, and the accompanying warning claimed configured triggers were all fraction-based when none were configured. Only report nothing-usable when trigger clauses actually existed; trigger:null keeps constructing the never-firing middleware with the degraded keep, matching its behavior outside the degradation path. * fix(summarization): address review — keep manual compaction, validate ContextSize, pin wiring Review follow-ups on #4901: - When every configured trigger is a dropped fraction clause, keep constructing the never-firing middleware (trigger=None) instead of returning None: manual /compact runs with force=True and never consults trigger clauses, so it must keep working for a profile-less model rather than reporting 'compaction is disabled'. The warning now says auto-compaction will not fire while manual compaction remains. - ContextSize gains a config-load validator: fraction values must be in (0,1] (a percent-style 80 instead of 0.8 previously produced a threshold the context could never reach — a silently inert trigger), absolute values must be positive. - New un-monkeypatched integration test pins the shipped wiring (context_window declared -> real factory attaches profile -> fraction clause survives -> middleware constructs), which the stubbed middleware-side tests and kwarg-capturing factory-side tests each stopped short of. - Docs (summarization.md + config.example.yaml) clarify that the fraction resolves against the summary/anchor model's context_window (summarization.model_name when set, else the run model), including the mismatch caveat for a larger-window summary model. * fix(summarization): reject non-finite ContextSize values at config load YAML .nan / .inf pass pydantic's float parsing, and nan <= 0 is False, so the positivity check alone let them through as dead thresholds (count >= nan is always False) — the same silent-inert-trigger class the range validator was added to close. Guard with math.isfinite first, consistent with the existing non-finite guards on mem0 timeout_seconds and poll_after_seconds. * fix(summarization): merge context_window into inferred profile, require whole message counts - construct the model first, then merge max_input_tokens into the provider-inferred langchain profile: passing profile= to the constructor replaced the whole inferred metadata (tool_calling, structured_output, io capabilities, output limits) with the single key. An explicitly configured profile is still never clobbered. - reject non-integral ContextSize values for type=messages at config load: langchain slices the message list with them, so a float index raised TypeError mid-compaction. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
hataa
|
08b27aef73
|
feat(auth): make login rate-limit parameters configurable, fixes #5108 (#5110)
* feat(auth): make login rate-limit parameters configurable, fixes #5108 Add auth.local.max_login_attempts (default 5) and auth.local.lockout_seconds (default 300) so operators can tune the per-IP login throttle: raise the ceiling for shared-egress-IP offices behind proxies/NAT, or tighten it for stricter posture. Policy is live-read per call (matching the _local_registration_enabled precedent), so a config reload applies without a Gateway restart; raising the threshold mid-lockout immediately unblocks affected IPs. Review feedback addressed (willem-bd): - Only FileNotFoundError falls back to the hardcoded defaults; a malformed config propagates, mirroring _local_registration_enabled, so an operator who tightened the policy never silently gets the more permissive defaults. - _check_rate_limit looks up the record before resolving the policy, so a clean IP pays zero config reads (get_app_config re-hashes config.yaml per call and login_local is an unauthenticated async endpoint). Bumps config_version to 39 in config.example.yaml and the Helm chart (values.yaml + README example) so the chart drift check stays green. * fix(auth): reject max_login_attempts=1 and honor live lockout_seconds for active lockouts * fix(auth): close live-policy state gaps in login throttle (resurrection, count reset, broken-config verification) * fix(auth): commit evaluated lockout duration on decreases too, preventing raise-resurrection * test(auth): pin broken-config fail-closed sequence through the login route * fix(auth): sweep expired locks by stored sentence and keep policy reads off the event loop * fix(auth): re-read throttle record after the policy-resolution yield point --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
9146bfa03d
|
feature(gateway): issue request trace ids unconditionally (#5119)
* refactor(gateway): issue request trace ids unconditionally The request trace id was gated behind logging.enhance.enabled at every entry point, so downstream code had to keep asking whether one existed: a header-provenance flag in its own ContextVar, a precedence resolver, and three-level carrier fallbacks at each consumer. Bind one unconditionally instead. TraceMiddleware covers Gateway HTTP; ensure_trace_context covers the entry points that never touch ASGI -- scheduled occurrences, MCP task notification runs, IM channel messages, and the embedded client -- each scoped to one unit of work so a long-lived worker task cannot leak one occurrence's id into the next. The ContextVar becomes the only source; the response header, runtime context, run metadata and log records are derived outputs. Consumers now use ensure_trace_id() or resolve_trace_id(*carriers) and drop their presence guards. Removed: resolve_deerflow_trace_id, the header-provenance flag and its three helpers, set/reset_current_trace_id, is_trace_correlation_enabled and its gateway alias. BREAKING CHANGE: every Gateway HTTP response now carries X-Trace-Id and it cannot be turned off; logging.enhance.enabled controls log output only. Installations on the default enabled: false will start seeing the header. No config keys were added or removed. * fix(gateway): stop persisting a caller-supplied trace id on the run record body.metadata forks two ways: through build_run_config into the live run config, which the run worker restamps, and through create_or_reject into the run record that the runs API echoes verbatim. Only the first was covered, so a client sending metadata.deerflow_trace_id made the most durable and most visible surface of a run disagree with the X-Trace-Id and the log lines the same request produced -- a correlation id that does not match the logs is worse than none. Stamp the server-issued id once at the trust boundary so both forks receive it, preserving the caller's own metadata keys. Close the same gap on config.context, which reaches the runtime context by a separate path: _build_runtime_context no longer merges server-owned keys from the caller, and _install_runtime_context assigns rather than setdefaults. A thread's metadata is no longer seeded with the run-scoped id of whichever run created it -- one thread spans many runs and as many trace ids. Found by driving a real run through the Gateway and reading the run back from the runs API; every unit test built its metadata by hand and so could not see it. * fix(gateway): expose X-Trace-Id to split-origin browser clients X-Trace-Id is not on the CORS safelist, so a browser client served from a separate origin could not read it -- and those are exactly the clients that cannot read the Gateway's logs either, leaving them with nothing to quote in a bug report. Same-origin nginx deployments were unaffected, which is why this stayed hidden. Add it to CORS_EXPOSED_HEADERS beside Content-Location, referencing TRACE_ID_HEADER rather than repeating the literal. * fix(gateway): keep X-Trace-Id on unhandled-exception 500s Starlette's ServerErrorMiddleware sits outside every user middleware and emits unhandled-exception 500s through the raw send, so those responses never pass TraceMiddleware's header-writing wrapper. The 500 for a server bug is exactly the response a user most needs to correlate with a log line, and it was the one response that shipped without the id. TraceMiddleware now tracks whether http.response.start has been sent. On an exception with no response started it emits its own plain 500 carrying the header, then re-raises: the outer ServerErrorMiddleware sees the response already started and only re-raises too, so the server's exception logging is untouched. An exception mid-stream keeps propagating unchanged — a second response start cannot be sent, and the already-written header stands. The trace id is printable ASCII by construction (normalize_trace_id / generate_trace_id), which is what makes the raw latin-1 header encoding safe. * fix(gateway): strip the forged trace id from the persisted request echo The run-record fix stopped a forged metadata.deerflow_trace_id on the authoritative metadata surface, but the raw request echo still carried one: create_or_reject persists body.config verbatim as runs.kwargs_json, which the runs API serves back. A client posting config.context.deerflow_trace_id therefore still got its forged value stored and echoed on one API surface while the header, logs, run metadata, and checkpoint all carried the real id — the id is ignored as input there, so echoing it back only manufactures disagreement. Two changes close it. redact_config_secrets — already the shared scrub for that echo, applied at admission and again at serve time, so historical records are covered too — now also drops deerflow_trace_id from config.metadata and config.context. And build_run_config now merges run metadata onto a copy of the caller's config["metadata"] instead of updating it in place: the nested values of the request config are reference copies, so the in-place merge was writing the server-stamped key through into body.config, contaminating the "what the client sent" record before it was persisted (and incidentally masking the forged-value echo on the metadata container). The regression test posts a forged id through body.metadata, config.metadata, and config.context at once and reads the kwargs echo back off the run record, failing if either leak returns. * docs(harness): record the trace-echo scrub, 500 fallback, and accepted retry divergence The trace section of the harness AGENTS.md now covers the two fixes that close the derived-output rule (the kwargs-echo scrub in redact_config_secrets plus build_run_config's copy merge, and TraceMiddleware's own 500 for unhandled exceptions), and CHANGELOG gains their Fixed entries. It also writes down the one accepted divergence: a crash-recovered scheduled launch reuses the durable run through its idempotency key, and start_run returns early on idempotency_reused without restamping — so the run record keeps the first attempt's deerflow_trace_id while the retry's own log lines carry the freshly minted id of its ensure_trace_context binding. The divergence is confined to the crash-recovery window and is accepted rather than fixed: restamping on reuse would rewrite a persisted record for a run that already exists, which is worse than two ids that each correlate their own attempt's logs. Written down so the next reader of the scheduler recovery path does not diagnose it as a bug. * docs(config): align the logging.enhance schema note with the unconditional trace id The config-module AGENTS.md still described logging.enhance as the gate for the Gateway X-Trace-Id header and Langfuse deerflow_trace_id. That model is gone: ids are issued unconditionally and this block decides log output only. Left as-is, the stale wording invites an agent to "restore" a header gate it believes was lost. Reworded to match the sibling AGENTS.md files and config.example.yaml, with a pointer to the Request Trace Context section that owns the full model. * docs(changelog): link the trace entries to #5119 The five new entries pointed at the ([#XXXX]) placeholder with no reference definition, rendering as literal text instead of a link — and RELEASING.md step 2 relies on those references when the section becomes release notes. All five now point at #5119, with the definition appended to the reference block. * refactor(harness): rename _stream_without_trace_context to _stream_turn The name asserted the opposite of what the method now does. It was accurate while logging.enhance.enabled could route stream() around the trace scope; with the gate gone it is the only stream implementation left, and it binds the id itself via ensure_trace_id(). Private, so the rename touches only the definition and the one stream() call site. * docs(harness): fit the trace-context guidance inside the AGENTS.md chain budget The expanded Request Trace Context section pushed the effective AGENTS.md chain for agents/middlewares to 99,815 bytes, past the 98,304 hard limit scripts/check_agent_guidance.py enforces in CI (AG002). Compressed the section from 7,359 to 4592 bytes with no facts removed: the entry-point table, the derived-output rule and its enforcement points, the accepted scheduled-retry divergence, the two resolution helpers, the stream() binding rationale, the log-output-only gate, the CORS listing, the 500 fallback, and the test map all remain. Sized against the merge, not just the branch: current main grew the same chain by ~724 bytes, so the check was verified on the merged tree as well (97,772 bytes; branch tree 97,048). * fix(gateway): declare content-length on the fallback 500 The pre-response 500 declared content-type but no content-length, leaving the framing to the ASGI server: chunked on HTTP/1.1, close-delimited on HTTP/1.0 — the one wire difference from the ServerErrorMiddleware response it replaces, which sends content-length: 21. The explicit header keeps the fallback byte-identical to what clients saw before. * docs(readme): drop the trace-correlation condition from the translations The zh/ja/fr/ru Langfuse sections still said metadata.deerflow_trace_id matches X-Trace-Id "when request trace correlation is enabled". The id now always matches and that condition no longer exists, so each bullet states the unconditional match and that logging.enhance.enabled only controls whether the id is printed into logs — the one piece of the feature a user can still configure. * test(gateway): pin TraceMiddleware wiring through create_app() Every X-Trace-Id test exercised a hand-built four-route app, so the real stack's add_middleware(TraceMiddleware) line was pinned by nothing: deleting it — or short-circuiting above it — passed CI while silently dropping both the response header and the ambient id the run-record stamp and enhanced log records derive from. One case now drives /health through create_app() and asserts the inbound id round-trips; mutation-checked by removing the wiring line, which fails exactly this test. * docs(gateway): note the fallback 500 is CORS-opaque The pre-response 500 is emitted outside CORSMiddleware — the exception has already unwound past it — so it carries no Access-Control-Allow-Origin and a split-origin browser client cannot read the id on this one response, unchanged from the ServerErrorMiddleware 500 it replaces. Documented on the class and in the CHANGELOG entry rather than fixed: replicating the origin allowlist outside CORSMiddleware would let the two policies drift. * fix(harness): keep abandoned-stream cleanup inside the trace binding stream() binds the turn's id around each next(inner) and resets it before yielding, but the finally's inner.close() ran after that binding was gone. Abandoning the stream therefore drove the inner LangGraph generator's GeneratorExit/finally path with no trace id — or an unrelated ambient one from whichever context ran the close — so cancellation and finalization logs and callbacks did not correlate with the turn they belong to. inner.close() is now wrapped in a local bind/reset of the same turn id. The token is set and reset in the same frame, never across a yield, so the per-step cross-context safety is preserved even when GC closes the generator from another Context — pinned by the existing copy_context close test, which now exercises this path. The regression test records the id from the inner generator's finally and fails without the binding. * test(harness): teach the worker-trace fake about RunManager.cleanup Upstream #5112 (bound gateway memory after terminal runs) added a run_manager.cleanup(run_id) call to run_agent's finalization, so the merge-commit CI run failed all five worker-trace-binding tests with AttributeError on this PR's _FakeRunManager. The fake gains the same no-op shape as its other methods. * docs(gateway): bring the gateway AGENTS.md back under its soft budget Upstream #5092 grew backend/app/gateway/AGENTS.md to 40,966 bytes, 6 over the 40,960 soft budget that test_agent_guidance_check.py::test_repository_guidance_stays_below_soft_budgets_and_avoids_doc_indexes enforces — its Unit Tests run on main was cancelled by push concurrency, so main is currently red on that test and every PR merge-run inherits the failure. Two whitespace/wording trims in the row #5092 touched (a doubled space, and "its configured `context_window`" → "its `context_window`") bring the file to 40,953 with no content change. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Aari
|
317577e285
|
fix: enforce custom agent skill allowlists in sandboxes (#5077)
* fix: enforce agent skill allowlists in sandboxes * fix: guard E2B skill projection resets * fix: preserve agent skill isolation across delegation * fix: close sandbox skill isolation bypasses * fix(sandbox): close skill isolation review gaps * fix(sandbox): harden skill isolation lifecycle |
||
|
Wuong
|
e12925458a
|
feat(streaming): make heartbeat interval configurable (#5017)
Co-authored-by: Wuong <26929475+Wuong@users.noreply.github.com> |
||
|
Nefelibata
|
c6f6a01f56
|
fix(runtime): prevent IndexError in MemoryStreamBridge._make_gap on empty events buffer (#5047)
* fix(runtime): prevent IndexError in MemoryStreamBridge._make_gap on empty events buffer * fix(runtime): handle empty stream replay gap bounds across backend and frontend - Clamp MemoryStreamBridge queue_maxsize at 1 and validate StreamBridgeConfig.queue_maxsize >= 1 - Update StreamGap docstring to clarify None retained bounds - Allow StreamReplayGapData and parseStreamReplayGap in frontend to accept string | null bounds, safely resuming when bounds are null - Add backend and frontend regression unit tests for queue clamping and null bounds replay gap * docs(stream-bridge): bump config_version and document empty buffer replay gap behavior * docs: document nullable gap bounds and sync helm config_version to 37 |
||
|
青榆牧
|
a94b2d8897
|
feat(mcp): map request-scoped secrets to MCP HTTP/SSE headers (#5010)
* feat(mcp): map request-scoped secrets to HTTP/SSE headers `user_auth` binds a credential to a configured DeerFlow user, so a caller that picks the credential per request — a multi-tenant gateway, a per-run API key, one shared MCP server fronting several environments — had to register one MCP server entry per credential. Add a declarative `mcpServers.<server>.headers_from_context` block mapping HTTP header names to keys of the run request's `config.context.secrets` carrier. A new built-in interceptor resolves the mapping on every tool call and rewrites those headers, mirroring `user_scoped_auth`. The config file stores names only, never a credential, so the Gateway returns the block unmasked. Registered after OAuth and `user_auth` in the interceptor chain: the later interceptor runs closer to the transport, and the value chosen for this one request is the most specific, so it wins. Fail-closed by default — a mapped key missing from the request raises a `ToolException` naming only that key, because falling back to the server's discovery credential would send one tenant's call under another tenant's authority. `on_missing: "passthrough"` opts out. Durable background tasks are excluded: `McpTaskToolCaller` drives status and cancel polls after the Agent run ends, where no run context exists, so the fail-closed interceptor would deny every poll. Those calls keep using server-level credentials, and a server declaring both `headers_from_context` and `task_toolsets` now logs a warning. Also corrects the custom-interceptor example in docs/MCP_SERVER.md (and the matching claim in skills/AGENTS.md), which read request secrets from `langgraph.config.get_config()["context"]`. That key is `None` inside a tool call — the run context rides the LangGraph runtime, not the RunnableConfig propagated to child runnables — so interceptors written from that example never saw a value. The example now reads `request.runtime`, and tests/test_mcp_context_headers.py pins LangGraph's runtime-injection rule by driving a real langchain-mcp-adapters tool through a real graph with the ambient-runtime fallback disabled. Closes #5005 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): resolve credential headers case-insensitively, carry them on durable submit Review follow-ups on `headers_from_context`. HTTP field names are case-insensitive, but every dict on the path to the wire is not: `build_server_params` copies the operator's static `headers` spelling verbatim, and langchain-mcp-adapters merges interceptor overrides into the connection with a plain `{**connection_headers, **override_headers}` splat. A static `authorization` and an injected `Authorization` therefore both reached httpx as separate field lines, and a server reading the field with a single-value accessor got the static discovery credential — inverting the documented `headers` < `oauth` < `user_auth` < `headers_from_context` precedence and running a per-request call under the shared credential. Normalizing inside the interceptor cannot fix that on its own: the adapter builds the request with `headers=None`, so an interceptor never sees the connection's static headers and cannot displace them however it spells its own key. A new `mcp/headers.py::apply_header_overrides` therefore drops any key differing only in case and emits the spelling the connection already uses. Applied to `headers_from_context`, `user_auth`, the OAuth interceptor, the OAuth discovery-header write, and the durable-task connection merge, which all carried the same collision. `headers_from_context.headers` now also rejects one header mapped under two spellings at config load, in both the harness model and the Gateway mirror. Durable submit now carries the mapped headers, as docs/MCP_SERVER.md already promised. `McpTaskToolCaller` disabled the interceptor for the whole caller, but that caller serves submit as well as the polls, and submit is awaited inline inside the Agent's tool call — where the run's LangGraph runtime is still the ambient contextvar, so no secret has to be threaded through `TaskSubmitRequest` or reach durable storage. The caller builds one chain and keeps a second view of it without the context-headers interceptor; `call_tool` takes `request_scoped_headers`, set only by `OrdinaryMcpTaskDriver.submit`. Status and cancel keep server-level credentials, so background polls still cannot fail closed, and the startup warning now describes the half it actually covers. `_merge_preserving_secrets` restores masked extras inside `headers_from_context` instead of writing the `***` sentinel back over the stored value, matching the treatment `user_auth` extras and server-level extras already get; extras a PUT omits carry over as well, while the declared mapping still replaces verbatim so a round trip can remove an entry. `extra="allow"` plus name-based sensitivity detection means the usual casualty is a name-valued key such as `tokenHeader`, not only a credential. The existing override test seeded the static header onto `request.headers`, which production never does, so it modelled a merge that really happens one layer down; the new tests drive a real adapter tool through a real connection and assert on the headers the session is opened with, and the durable-submit test runs through a real tool node with no runtime patching. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): reject case-insensitive duplicate static header names * fix(mcp): preserve omitted headers_from_context fields on partial updates --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Aari
|
ff0a6768c2
|
feat(subagents): add unified capacity and durable batch execution (#4998)
* feat(subagents): add capacity controls and durable batches * fix(helm): sync subagent config schema version * fix(subagents): preserve batch history without worker * fix(subagents): support explicit factory runtimes * fix: address durable batch review findings |
||
|
Aari
|
645ca08f16
|
fix(scheduler): enqueue busy scheduled task runs (#4918)
* fix(frontend): clarify reuse-thread scheduling behavior * fix(scheduler): enqueue overlapping scheduled runs * fix(scheduler): preserve queue lease fencing * fix(scheduler): close queue concurrency races * fix(scheduler): harden queue timeout bookkeeping * fix(scheduler): preserve manual failure schedule --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Tsai Yuan
|
336cd3acc4
|
fix(sandbox): reject non-finite ownership timings (#4960) | ||
|
Aari
|
1aa813ddb3
|
feat: add managed subagents and delegation scopes (#4887)
* feat: manage and scope subagents * fix: address subagent review feedback * fix: address managed subagent review feedback * fix: harden subagent settings semantics * fix: harden managed subagent cache invalidation * fix: reuse assembled lead agent inputs * fix: migrate managed subagent definitions --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Aleksandr Sapronov
|
613b90b0e6
|
feat(scheduler): make scheduled-run recursion_limit configurable (#4848)
Adds scheduler.recursion_limit to config.yaml (default 1000, clamped by max_recursion_limit) so scheduled background runs can use a different recursion limit than the web UI. The value is read at dispatch time, so a YAML edit applies to the next scheduled run without a Gateway restart. Also logs a warning when the resolver falls back to the default or clamps the configured value. |
||
|
muguo
|
917fe595fc
|
feat(sandbox): add OpenSandbox provider (#4877) | ||
|
Zeren Wang
|
4e35f0d1d4
|
feat(harness): deterministic tool receipts with model-visible ledger (RFC #4651, layer 1) (#4659)
* feat(harness): add deterministic tool receipts with model-visible ledger Stamp an immutable per-call fact record (tool name, status, args/output hashes, byte count, timestamp) onto every tool result via a new ToolReceiptMiddleware, and inject the derived receipt ledger (r1..rN) into the model context so subagent reports can cite executed actions. - tool_receipt.py: receipt core (make/extract/render), newest-first budget eviction, ids derived from the append-only message stream - ToolReceiptMiddleware: stamps ToolMessages directly or inside Command-wrapped results; hidden ledger injection mirrors DurableContextMiddleware; sits between ToolProgress and ToolErrorHandling with a build-time ordering guard - config: new verification section (receipts on, judge off), config version 32 -> 33 with example/helm/docs updates * feat(harness): split receipt rendering from stamping; address PR review Review fixes (PR #4659): - output_sha256 now uses sort_keys=True for structured content, matching the order-invariant args fingerprint - stamping failures log at warning (silent ledger gaps would corrupt citations); tool execution remains never blocked - _insert_after_leading_system_messages extracted to shared public message_utils.insert_after_leading_system_messages; both middlewares depend on it instead of a private cross-module helper - code comments in English RFC #4651 revision-2 alignment: - receipts_render_mode config ('always' | 'delegation_only'): subagent chains always render the ledger (citations are produced there); the lead chain renders only while processing subagent results, removing the always-on token tax from ordinary turns - receipts gain bounded args_preview/output_preview (<=200 chars, tail for output) so later typed claim bindings (tests_passed) can anchor to a specific recorded execution * docs(harness): state receipt freshness caveat and vocabulary layering in module docstring * merge: upstream/main — resolve AGENTS.md split, bump config_version to 34, drop unused receipt previews - backend/AGENTS.md: take upstream's slimmed root guidance (#4799); move the ToolReceiptMiddleware chain entry into agents/middlewares/AGENTS.md and the verification.* hot-reload mention into config/AGENTS.md - config.example.yaml + helm values/README: config_version 33 -> 34 so existing v33 configs get the outdated-config prompt (review: willem-bd) - tool_receipt.py: drop args_preview/output_preview — no Layer 1 consumer reads them; re-add with the Layer 2 claim-binding consumer (review: willem-bd) * docs(harness): cover receipt id renumbering after compaction in module docstring Positional display ids are stable only while history is append-only; compaction drops ToolMessages and the survivors renumber, so Layer 2 citation verification must resolve [rN] against the ledger as of the citing turn (review: willem-bd, doc-only). * chore(config): bump config_version to 35 main reached 34 via #4780 without the verification section; publishing the new schema at the same number would silently skip the outdated-config prompt for configs synced from main in that window (review: willem-bd). * fix(skills): restore errno import dropped upstream in #4830 upstream/main adf6c422 uses errno.ENOTDIR in the drift guard but removed the import, so the PR merge ref fails lint-backend (F821). * fix(harness): harden tool receipts against forgery and turn-scope delegation_only Address willem-bd's pre-merge review on #4659: 1. Untrusted receipt metadata: the gateway now strips the server-owned deerflow_tool_receipt key from external input messages; stamping always overwrites any tool-supplied value instead of preserving it; and extract_tool_receipts validates persisted receipt shapes (required typed fields, unknown keys ignored) so malformed entries are skipped instead of crashing render or passing as runtime-stamped evidence. 2. delegation_only no longer sticks on: _should_render now scopes the subagent_status scan to the current turn (messages after the latest genuine user message), so an old completed delegation stops rendering the ledger on later ordinary turns. The genuine-user predicate moves to message_utils.is_genuine_user_message, shared with input sanitization. * fix(harness): stamp receipts outside short-circuiting tool middlewares Address willem-bd's review on #4659: ToolReceiptMiddleware was registered inside Guardrail/SandboxAudit/ReadBeforeWrite/ToolProgress, each of which can return a ToolMessage without invoking its handler — blocked calls (e.g. a read-before-write-denied write_file) never got a receipt, silently gapping the ledger on a default-enabled path. SandboxAudit additionally rebuilds medium-risk results, dropping an inner stamp. ToolReceiptMiddleware is now the outermost wrap_tool_call layer in the runtime tail. Normal results still carry deerflow_tool_meta (stamped by ToolErrorHandling on the inner return path); short-circuit messages self-stamp meta or fall back to message.status. The new invariant is declared as ordering constraints in deerflow.extensions.ordering, with composed-chain regression tests for a blocked write and a warn-rebuilt bash result. |
||
|
ajayr
|
7e95bef2e7
|
feat(mcp): per-user credential injection for shared MCP servers (#4868)
* feat(mcp): per-user credential injection for shared MCP servers
A single HTTP/SSE MCP server entry can now serve several users, each
authenticated to the remote service with their own credential. A server
opts in with a user_auth block mapping user ids to credential header
values ($ENV_VAR references supported):
"user_auth": {
"header": "Authorization",
"users": { "<user-id>": "$SERVICE_TOKEN_ALICE" }
}
The built-in user-scoped auth interceptor resolves the authenticated
runtime user on every tool call (request runtime -> ambient LangGraph
runtime -> auth config -> request-scoped user ContextVar) and rewrites
the configured header via request.override(), the same per-call
mechanism as the OAuth interceptor. It registers after OAuth in the
shared assembly so its per-user value wins the header when a server
declares both. The entry's static headers are used only for startup
tool discovery.
Fail-closed by default: an unmapped user - including the anonymous
default-user fallback - or a credential whose env reference resolved
empty gets an actionable ToolException instead of another user's
credential; on_missing: "passthrough" opts out per server. Combined
with the existing per-(user, thread) MCP session scoping this gives
credential isolation on shared servers.
Gateway API: user_auth.users values are masked in GET responses, and
PUT round-trips preserve stored credentials for masked values (same
contract as env/headers/oauth secrets); a masked value for a user id
not already stored is rejected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(mcp): address review — preserve stored user_auth sub-fields on partial PUT, warn-and-skip stdio, allow extras
Review findings from #4868:
1. A partial user_auth payload (e.g. {"enabled": false}) merged to
users={} and default on_missing, irreversibly wiping stored
credentials on PUT. The merge is now sub-field-aware via
model_fields_set — omitted sub-fields carry the stored values, an
explicitly sent users map still replaces (so full-round-trip removal
works), masked values still swap back for stored credentials.
2. user_auth on a stdio server was a silent no-op: rewritten headers go
to call meta, never a transport header, while deny errors still fired.
The interceptor builder now warns and skips non-sse/http servers,
matching the tool_call_timeout transport-mismatch convention.
3. McpUserScopedAuthConfigResponse now allows extra keys like the
harness-side model, and extras survive masking and merge, matching
the server-level model_extra handling.
Adds four regression tests (partial-PUT preservation, explicit-map
replacement, extras round-trip, stdio warn-and-skip).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(mcp): reject blank user_auth.header at the gateway
A blank header passed the gateway response model, was persisted, then
failed the harness-side ExtensionsConfig validator on reload — the PUT
returned 500 after the write and every later config load/startup failed
until the file was hand-edited. Mirror the harness non-blank validator
on McpUserScopedAuthConfigResponse so the PUT fails with 422 before
anything is written.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style: ruff format extensions_config.py
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(mcp): harden the trust chain and masking around user-scoped credential selection
Address review round 4:
- Scrub client-supplied user_id from run context/configurable for
external callers in inject_authenticated_user_context, before every
early return, and restamp only from request.state.user. Now that
user_id selects which user's credential user-scoped MCP auth injects,
a forged value must not survive any future path that skips the
restamp. Internal callers (IM channels, scheduler) keep supplying
end-user identity as before (PR #3294 contract). Regression tests pin
both the scrub and that a forged body.context.user_id can never
resolve as another user through merge + inject ordering.
- Include the caller's own resolved user id in the fail-closed deny
message so operators can copy the exact users key (it differs by
deployment path), and document the key formats in the mcp.mdx doc.
- Mask sensitive extra keys inside user_auth on GET like server-level
extras, and swap masked sentinels back for stored values on PUT via
_merge_extra_value_preserving_masked.
- Extract the interceptor wrap loop into compose_tool_interceptors and
pin the security property functionally: an OAuth interceptor that
actually sets Authorization loses the final header to the per-user
credential through the same composition the session-pool path uses.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Aari
|
236a068e77
|
fix(docker): let the Gateway write extensions_config.json in production (#4852)
* fix(docker): let the Gateway write extensions_config.json in production AGENTS.md states extensions_config.json may be edited at runtime through the Gateway API, and the Gateway implements that for the MCP enable switch, PUT/PATCH /api/mcp/config and the skill update route. Two properties of the production compose stack made every one of those writes fail: - the file was mounted read-only, and - Docker mounts it as its own mount point, so the temp-file-plus-rename in atomic_write_extensions_config hit EBUSY. Linux refuses rename() over a mount point whether or not the mount is writable, so making the mount read-write alone is not enough. Mount it read-write and fall back to an in-place overwrite on EBUSY only. The fallback is deliberately non-atomic and says so in a warning; it is reached only where the atomic route cannot work, and any other errno still propagates. config.yaml stays read-only: no API writes it. docker-compose-dev.yaml mounts the whole project directory, so the destination is an ordinary file there and this never surfaced in development. * test(docker): parse mount options instead of matching a :ro suffix Docker's short-syntax options segment is comma-separated, so a read-only mount can legally be spelled ":ro,z" or ":z,ro" — common with SELinux relabelling. Matching the raw string for a ":ro" suffix reads those as writable, which silently defeats the guard: the writability assertion would pass on a read-only mount, and the config.yaml assertion would fail on a correctly read-only one. Parse the options segment and test membership instead, and cover the parser with the spellings that broke the suffix check. * fix(config): harden mutable extensions config |
||
|
Aari
|
47b258ebd7
|
feat(mcp): add ordinary durable task driver (#4690)
* feat(mcp): add durable task runtime foundation * fix(chart): sync embedded config version * fix(mcp): isolate task polls during shutdown * feat(mcp): track consecutive poll errors on mcp_tasks poll_attempt_count grows on every claim (successful polls included), so it cannot drive a failure backoff without misjudging normal long tasks. Add consecutive_poll_error_count: incremented when a claim is released after a poll error, reset to zero by any applied snapshot. The backoff/terminal policy that consumes it lands with the first concrete driver. * fix(mcp): harden durable task lifecycle * feat(mcp): add ordinary durable task driver * test(mcp): address durable task review feedback * fix(mcp): preserve submit tool descriptions * fix(mcp): bound remote task calls * fix(mcp): bound persisted task payloads * fix(mcp): preserve task tool error details * fix(mcp): enforce durable task boundaries * test(mcp): cover task config snapshot lifecycle --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
rain02333z-spec
|
828363705a
|
fix(scheduler): support safe multi-instance scheduler recovery (#4713)
* fix(scheduler): reject unsafe multi-worker startup * fix(scheduler): support safe multi-instance recovery * fix(scheduler): make multi-instance recovery lease-safe * fix(scheduler): address multi-instance review feedback * docs(scheduler): add multi-instance upgrade notes --------- Co-authored-by: rain02333z-spec <225106191+rain02333z-spec@users.noreply.github.com> |
||
|
Ryker_Feng
|
ccff5f5ce7
|
docs: govern agent guidance size (#4799)
* docs: govern agent guidance size * refactor: split agent guidance by code scope * Clarify virtual path handling in AGENTS.md Updated the translation section to clarify the role of `LocalSandboxProvider` and the handling of virtual paths in the tool layer. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Aari
|
e9387394bc
|
feat(mcp): add durable task runtime foundation (#4665)
* feat(mcp): add durable task runtime foundation * fix(chart): sync embedded config version * fix(mcp): isolate task polls during shutdown * feat(mcp): track consecutive poll errors on mcp_tasks poll_attempt_count grows on every claim (successful polls included), so it cannot drive a failure backoff without misjudging normal long tasks. Add consecutive_poll_error_count: incremented when a claim is released after a poll error, reset to zero by any applied snapshot. The backoff/terminal policy that consumes it lands with the first concrete driver. * fix(mcp): harden durable task lifecycle * fix(mcp): preserve tracked task on dedup conflict --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
lllyfff
|
99c926b7bb
|
fix(mcp): bring-up has no timeout and externalized tool outputs are counted as undelivered artifacts (#4657)
* fix: bound MCP server bring-up timeouts and exclude externalized tool outputs from delivery verification Two related robustness fixes: 1. MCP server bring-up was unbounded. tool_call_timeout only covered session.call_tool(); tool discovery (subprocess spawn + initialize + tools/list) and persistent stdio session initialization could hang forever, blocking agent construction (and on the Gateway event loop, the whole process). Add a per-server session_init_timeout (default DEFAULT_MCP_SESSION_INIT_TIMEOUT = 60s, null disables) that bounds both discovery and pooled-session initialization. The session pool's existing cancellation handling tears down a session stuck mid-creation in its own task. 2. ToolOutputBudgetMiddleware externalizes oversized tool outputs into outputs/.tool-results/ (configurable tool_output.storage_subdir). The workspace-change scanner and run delivery verification counted those files as produced artifacts, so any run that externalized a tool output without also presenting a real artifact failed with "Artifact delivery incomplete". Exclude TOOL_RESULTS_DIRNAME via a shared constant (mirroring BROWSER_FRAMES_DIRNAME) and thread the configured storage_subdir through snapshot capture so both workspace-changes events and delivery verification stay clean. * review: enforce single-segment tool_output.storage_subdir; document discovery-timeout cleanup Address review feedback: 1. A custom tool_output.storage_subdir with a path separator (e.g. cache/tool-results) silently no-oped the workspace-scanner exclusion: os.walk yields one-segment dirnames, so a nested value never matched and its files were counted as produced artifacts again. ToolOutputConfig now validates storage_subdir as a single directory name (rejects separators, .., absolute, empty) with tests, so the exclusion is always sound. 2. The discovery-timeout path now documents why cancellation is safe, mirroring the session-init note: discovery runs inside the adapter's nested async context managers, and stdio_client's finally terminates the process tree (SIGTERM->SIGKILL on POSIX, process-tree on Windows), so a timed-out npx subprocess and its children are reaped rather than accumulating. * review: log session-init timeouts and align API response model default with runtime config Address second-round review feedback: 1. A session-init timeout raised TimeoutError without any log, unlike the discovery timeout which logs a WARNING. Wrap the bounded get_session in a try/except that logs the timeout (server name + seconds) and re-raises, so operators can diagnose tool-call failures caused by hung MCP sessions. 2. McpServerConfigResponse.session_init_timeout defaulted to None while McpServerConfig defaults to 60s: a server created via PUT /api/mcp/config without the field was persisted with null (no timeout) while the same server created in the config file got 60s. Align the response-model default to DEFAULT_MCP_SESSION_INIT_TIMEOUT so API-created and file-created servers behave the same; an explicit null still opts out. * review: narrow the discovery-timeout handler to the bounded wait_for path The except TimeoutError clause covered both the bounded wait_for branch and the bare discovery branch. With session_init_timeout opted out (None), a TimeoutError raised by discovery itself would hit the %.1f format with None: logging raises TypeError internally, the WARNING is silently dropped, and a --- Logging error --- traceback goes to stderr. Narrow the handler to wrap only the wait_for call, where the branch condition guarantees the timeout value is not None. A discovery-internal TimeoutError on the opted-out path now falls through to the generic failure handler and is reported as 'tool discovery failed' with exc_info. Covered by a regression test that asserts the skip is reported without any broken format. |
||
|
ajayr
|
d732b90dc3
|
feat(channels): add Buzz (Nostr) channel connector (#4649)
* feat(channels): add Buzz (Nostr) channel connector Adds a Buzz (https://github.com/block/buzz) channel so DeerFlow can join a Nostr-relay workspace as a member: it answers @mentions in channels, replies to DMs, and streams answers by editing one message in place. * app/channels/buzz_nostr.py — pure NIP-01 helpers: canonical event ids, BIP-340 signing/verification, chat/edit/auth builders, relay frames. * app/channels/buzz.py — BuzzChannel: one NIP-42-authenticated websocket, channel discovery (kind 39000) with one subscription per channel, live membership tracking (44100/44101), per-channel replay watermarks, and replies posted once then edited in place (kind 40003). * app/channels/buzz_run_policy.py — same-thread serialization, mirroring the Feishu precedent. Inbound is gated in order: signature verification, self-drop, /connect bind-and-return, pubkey allowlist, then mention / DM / mention-free / thread-follow. Off by default; needs the new optional `buzz` extra (coincurve, lazily imported), which detect_uv_extras resolves from channels.buzz.enabled the same way it already handles channels.discord. Two relay behaviours drove the design and are worth knowing when reviewing: a global {"kinds":[9]} subscription receives nothing from buzz-relay and a multi-value "#h" filter receives nothing either, so one REQ per channel is required; and a single global `since` cursor skips quiet channels, so watermarks are per channel. Signed-off-by: Ajay R <ajayr@formbuddy.com> * fix(channels): only publish assistant messages from the IM stream `_accumulate_stream_text` decided what streamed `messages-tuple` payloads become displayable text by rejecting ONLY payloads whose `type` contained "tool", so it published everything else. DeerFlow writes hidden model context into the messages channel as ordinary messages -- memory recall and the rewritten user turn as hidden HumanMessages (DynamicContextMiddleware), the `<durable_context_data>` block as another (DurableContextMiddleware) -- and LangGraph fans those state writes out on the messages stream, so they reached every streaming IM channel as the assistant's reply. Proved live on a Buzz relay: the connector published a `<memory>` fact block and, in another run, a verbatim echo of the user's own inbound message. Affects Feishu, Telegram, WeCom and Buzz; worst on Buzz, where each update is an immutable public Nostr event that a corrective edit cannot unpublish. Invert the filter to an allowlist of assistant message types. Two new pure helpers keep it testable: - `_stream_payload_type` resolves the type from both shapes the function already handles: the `model_dump()` shape the gateway emits, and LangChain's `to_json()` constructor shape whose own `type` is the literal "constructor" and whose class name is the tail of the `id` path. - `_is_assistant_stream_type` matches "ai"/"assistant" by PREFIX, not substring -- ordinary words contain "ai" ("chain", "domain"), and a substring test would admit a foreign type name by accident. The bare-`str` branch is removed: an untyped payload cannot be attributed to the assistant, nothing in DeerFlow produces one (serialize_messages_tuple always emits `[message_dict, metadata]`), and a runtime that emitted raw text deltas would emit hidden context the same way. Per-message-id buffering and merging are unchanged. Tests pin both directions, including multi-chunk merging across one message id, so the allowlist cannot silently kill streaming, plus an end-to-end `_handle_streaming_chat` test asserting the live payload never reaches an outbound message. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Ajay R <ajayr@formbuddy.com> * chore(helm): bump config_version to 33 in chart values and README config.example.yaml moved to 33 for the buzz channel block; the chart's embedded config example and its README copy track it (config_version only drives the outdated-config warning, per scripts/check_config_version.sh). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ajay R <ajayr@formbuddy.com> --------- Signed-off-by: Ajay R <ajayr@formbuddy.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Nan Gao
|
1f792d0f4b
|
feat(extensions): add middleware plugin foundation (#4636)
* feat(extensions): add middleware plugin foundation * fix(extensions): stop config resolution from masking extension loading `create_app()` resolved the configured plugin list inside the fail-open guard around `load_extensions()`. CI has no `config.yaml` (gitignored and never generated by the workflow), so `get_app_config()` raised `FileNotFoundError` there and was swallowed as an extension failure -- `load_extensions()` never ran at all, and the four `create_app()` tests in `test_extension_app_loading.py` passed locally but failed on every runner. Resolve the plugin list before the guard. Only an absent `config.yaml` is tolerated, mirroring `_resolve_trace_enabled_for_app_construction()`: `create_app()` runs at import time, and lifespan still performs strict config loading before serving. A `config.yaml` that exists but fails to parse or validate now propagates instead of being reported as an extension failure -- reporting it as the latter silently dropped a `required: true` extension rather than failing the boot. Make the tests config-independent with an autouse `stub_app_config` fixture, following the existing pattern in `test_gateway_lifespan_shutdown.py`, and cover both new branches of the config-resolution boundary. * fix(extensions): bind the run's extension snapshot through subagent delegation The lead-agent path resolves one immutable loaded-extension snapshot per run and binds it through task-store allocation and graph construction, but the subagent path re-read the process-wide singleton at execution time. In production both are the same object, yet a `set_loaded_extensions()` between the lead run's start and a subagent's execution (test teardown, a future hot-reload path) would let one run mix two extension generations — exactly what the documented invariant exists to prevent. The graph-build binding is a ContextVar scoped to synchronous construction, so it has already exited by the time a tool delegates; the snapshot has to travel through runtime context instead. The run worker publishes it under the host-internal `EXTENSION_SNAPSHOT_CONTEXT_KEY` (written after the caller merge, popped when the run has none, so a caller-supplied value is never authoritative), `task_tool` reads it back through the type-checking `resolve_run_extensions()`, and `SubagentExecutor` binds it at construction. Callers outside the Gateway run path — embedded `DeerFlowClient`, standalone LangGraph Server — install no snapshot and keep the existing `get_loaded_extensions()` fallback. * refactor(extensions): defer the ordering table by call, not by a lying tuple `CORE_ORDERING_CONSTRAINTS` was a `tuple` subclass that overrode only `__iter__` and resolved into a class-level `_resolved` side channel. A tuple cannot populate its own storage after construction, so the instance stayed the empty tuple it was built as: `len()` was 0, `bool()` was False, `in` was always False, indexing raised, slicing and `reversed()` came back empty, and it compared unequal to the plain tuples tests substitute for it — all while iteration yielded the real constraints. Only `assert_ordering` consumed it, and only by iterating, so the split went unnoticed. The sibling `_AnchorTable(dict)` uses the same idea soundly because dict is mutable: `self.update()` fills the real storage, making every inherited operation correct. That trick does not survive the port to an immutable type. Replace it with `core_ordering_constraints()`, matching how `stack.py` defers the same kind of table via `_anchors()`. The deferral is kept — it is about dependency direction, not just cycles: `extensions/` is the layer the middleware layer calls into, so a module-scope `agents.middlewares` import here points the dependency backwards and closes a cycle as soon as any middleware imports something under `extensions/` at module level. Resolution stays at `assert_ordering` time, which already runs inside the middleware builder. Tests pin both halves: the returned value is a plain tuple whose len/bool/ membership/indexing/reversal/equality agree with iteration, and a subprocess probe asserts importing `extensions.ordering` does not load the middleware layer while calling the function does. |
||
|
Vanzeren
|
c8cf1bf2fb
|
feat(checkpoint): checkpoint history cache (#4638)
* feat(checkpoint-cache): delta-mode checkpoint history cache with recursive compose
Read-only, invalidation-free cache for LangGraph delta-channel history
({writes, seed}) at the get_delta_channel_history choke point:
- database.checkpoint_cache config (memory|redis; max_entries 0=disabled;
redis bounded by TTL, Gateway/async only)
- memory LRU backend (copy-on-read, zero-serde hit path) and redis backend
(lazy import, degrades to all-miss on outage)
- CachedHistorySaver: recursive composition from the nearest warm ancestor
(depth budget 8), caching each level; depth-0 cold chains delegate one
inner fast-path walk. Entries keyed by immutable
(db, thread, ns, checkpoint_id, channel) — no invalidation, coherent
across workers
- provider wiring: wraps in delta mode only (async + sync), full mode
untouched; sync path is memory-only
- bench opt-in: DEERFLOW_CHECKPOINT_BENCH_HISTORY_CACHE=1
sqlite bench (500 updates, payload 2KB): write phase 2.28x at f=250,
1.32x at f=10; one delegated walk per thread cold start.
* chore(config): bump config_version to 32 for database.checkpoint_cache
The checkpoint history cache feature added the database.checkpoint_cache
section to config.example.yaml; bump the schema version so existing
deployments get the outdated-config warning and can run make config-upgrade.
* chore(helm): bump config_version to 32 in chart values and README
* fix(checkpoint-cache): purge thread history entries on delete paths
Addresses review on #4638: delete_thread/prune removed source-of-truth
checkpoints but left the thread's materialized history payloads in the
cache (memory: until LRU eviction; redis: until TTL, default 1 day) — a
data-lifecycle gap for tenant offboarding / GDPR-style erasure.
- Cache contract gains thread-scoped adelete_thread/delete_thread
(lifecycle purge, not invalidation; entries remain immutable)
- Memory backend: stem scan over the LRU map; redis: SCAN MATCH + UNLINK,
outage degrades to TTL-bounded retention without raising
- CachedHistorySaver purges on delete_thread/adelete_thread and
prune/aprune (prune rewrites chains, so pre-prune histories must go);
delete_for_runs stays delegation-only (run->thread mapping unavailable,
no in-tree callers), documented in code
- ttl_seconds description documents the residual-retention window
- Tests: thread-scoped purge on both backends, saver-level delete/prune
purge, prefix-safety (t1 vs t10), redis outage degradation, and the
pinned no-purge behavior of delete_for_runs
* fix(checkpoint-cache): stable db identity, prefix-aware sync singleton, explicit zero TTL
Addresses Copilot review on #4638:
- checkpoint_cache_db_hash now hashes the credential-free postgres
identity (host:port/database + schema): credential rotation no longer
changes the cache namespace (cold cache + orphaned keys until TTL).
Unparseable URLs fall back to the raw string.
- The sync-path memory cache singleton is also keyed by its key_prefix:
a namespace change (db identity change or operator override) recreates
the cache instead of leaving stale-prefix entries unreachable and
unpurgeable.
- ttl_seconds=0 is now an explicit, documented opt-out of redis expiry
(SET without EX; redis maxmemory policy only) instead of a silent
'ttl_seconds or None' coercion.
Tests: credential-rotation hash stability, unparseable-URL fallback,
prefix-change singleton recreation, same-prefix singleton reuse, and
zero-TTL wire behavior (ex=None).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Felix Wang
|
e221bddb38
|
feat: support per-server MCP tool name prefixes (#4624)
* feat: support per-server MCP tool name prefixes * refactor: pass MCP connection config directly * fix: preserve unprefixed MCP tool names in session pool |
||
|
Vanzeren
|
095092418c
|
fix(gateway):unify thread id validation (#4589)
* fix(gateway): unify thread ID validation at the API boundary
Thread ID entry points accepted arbitrary strings while downstream
consumers (filesystem paths, Kubernetes Provisioner, JSONL event store)
each enforced different character restrictions, so invalid IDs were
persisted first and only failed later during sandbox/workspace init.
Centralize validation in deerflow.utils.thread_id (pattern
^[A-Za-z0-9_-]{1,64}$): validate at routers, RunCreateRequest,
scheduler dispatch, paths.py, JSONL store, embedded client, and align
the Provisioner pattern (pinned by a parity test). UUIDs are still
generated only when no ID is supplied; caller-supplied opaque IDs stay
supported.
Deliberate exceptions: DELETE /threads/{id} keeps str as the legacy
cleanup escape hatch (filesystem cleanup guarded), read-only
client.get_thread stays unvalidated, and scheduler rows with legacy
invalid IDs record a failed dispatch instead of raising out of the
poll loop.
* docs: document canonical thread ID contract
README: caller-supplied thread IDs need not be UUIDs; the canonical
pattern and per-endpoint behavior. AGENTS.md: the shared
deerflow.utils.thread_id contract, its enforcement boundaries, and the
legacy-ID escape hatches.
* fix(gateway): close thread ID validation gaps at remaining entry points
Follow-up to the canonical thread ID contract: a full audit found the
uniform-422 coverage only reached about half of the thread_id surfaces.
- routers: 18 routes still took a bare thread_id: str — 13 in
thread_runs.py (including the five messages/events/workspace-changes
reads that returned 500 on the JSONL event store vs 404/empty on the
DB store), 4 read routes in threads.py, and the suggestions route
flagged in review. DELETE /api/threads/{id} keeps str as the declared
legacy-cleanup escape hatch.
- client: upload_files/delete_upload/list_uploads/get_artifact now
validate up front, fulfilling the RFC's 'all mutating entry points'
clause (get_thread stays unvalidated as the declared legacy read path).
- tui: the /resume literal-ref fallback validates against the canonical
contract and reports a descriptive error instead of failing deep in
the client.
- scripts/support_bundle.py: replace the drifted dot-allowing pattern
with a byte-identical copy of THREAD_ID_PATTERN (kept local so the
script still runs with a broken venv).
* test(gateway): guard the canonical thread ID contract against regressions
- test_thread_id_route_contract.py: static AST sweep asserting every
route handler with a thread_id parameter annotates ThreadId
(whitelist: the DELETE escape hatch), plus a runtime sweep hitting
all 44 thread_id routes with a non-canonical ID and asserting a 422
that names thread_id, plus a websocket upgrade-rejection case.
- test_thread_id_validation.py: client entry-point validation,
support_bundle pattern parity, and TUI literal-ref fallback tests.
- Align two tests that encoded the old contract (dotted IDs).
|