mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-11-07 11:10:35 +00:00
4401 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Hyeonsang Cho
|
59aa91d622
|
fix(projects): label untitled threads in the conversation-files view (#5879)
* fix(projects): label untitled threads in the conversation-files view
GET /api/projects/{id}/threads/files returns each member thread's
display_name as `str | None`: the thread meta row is created with NULL and
only filled once title generation has run after the first reply, or never
if that step fails. The frontend's ProjectThreadFileGroup declared the field
as a required string and ConversationFileGroup rendered it verbatim, so a
thread whose files were uploaded before its first reply showed them under
an empty heading. The sibling ProjectThread type already models the field
as nullable and every consumer of it falls back to t.projects.untitled.
Make the group type nullable to match the wire, and render the heading
through a shared threadTitle() helper that both the thread-name map and the
file group now use, so the "Untitled" rule lives in one place.
The DOM test covers null, missing and whitespace-only titles.
* docs(changelog): reference #5879 in the conversation-files entries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Hyeonsang Cho
|
9c53ebb06f
|
fix(skills): unlink a symlinked custom skill package on delete (#5881)
* fix(skills): unlink a symlinked custom skill package on delete
The storage contract (_is_external_skill_directory_symlink) accepts a
one-level symlink under custom/ that points at an operator-managed skill
package, and every read path honours it: the skill loads, lists, validates
and can be enabled. delete_custom_skill, however, removed the package with
shutil.rmtree, which refuses symlinks. The delete raised OSError after the
history record had already been appended, skill_projection_mutation cleared
the user's projected custom view on the way out (rebuilt on the next
acquire), and DELETE /api/skills/custom/{name} answered 500 — for both
LocalSkillStorage and UserScopedSkillStorage, which inherits the method.
Unlink the entry when it is a symlink and keep rmtree for real directories.
The external tree is not ours to delete: removing the skill means removing
the link. Checking is_symlink() first also covers a dangling link, which
exists() would report as absent.
Tests cover both storages: the link is gone, the skill no longer exists,
the external package (SKILL.md and a support file) is untouched, the
history record was written, and the sibling skill's projected view
survives the delete.
* docs(changelog): reference #5881 in the symlinked-skill delete entries
|
||
|
yc2bgr8
|
17f4722d2b
|
feat(community): migrate tavily tools to AsyncTavilyClient (#5784)
* feat(community): migrate tavily tools to AsyncTavilyClient
Replace the synchronous TavilyClient with AsyncTavilyClient in the tavily
community tools, as tracked in backend/docs/TODO.md planned features
('Replace sync requests with httpx.AsyncClient in community tools').
web_search_tool and web_fetch_tool become async functions and await
client.search()/client.extract(). External behavior (return values,
error strings, credentials handling) is unchanged; the client is still
constructed with api_key=None when the tool config has no key, letting
the SDK read TAVILY_API_KEY from the environment.
Tests are updated to invoke the async tools via ainvoke under
@pytest.mark.anyio, and the credential assertions now verify the client
is constructed with the right api_key (AsyncTavilyClient keeps the key
inside its httpx client and exposes no public api_key attribute).
* fix(tavily): close async clients after each tool request
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
NanPan
|
f2dd189cf9
|
fix(sandbox): retain AIO flock across cancellation (#5820)
* fix(sandbox): retain AIO flock across cancellation * fix(sandbox): drain async create before flock release * test(sandbox): cover cancellation during AIO create * docs(sandbox): pin create drain ownership * fix(sandbox): keep flock waits cancellable * test(sandbox): pin AIO flock lifecycle drains * docs(sandbox): document cancellable flock wait * test(sandbox): follow nonblocking flock helper * test(sandbox): update create cancellation lock mock * style(sandbox): sort lifecycle imports * test(sandbox): assert cancelled flock wait closes file * docs(sandbox): trim cancellation guidance below chain limit --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
haochen
|
fc9fb2dea2
|
test(lark): make blocking I/O CLI fixture work on Windows (#5826)
* test(lark): make blocking I/O CLI fixture work on Windows * docs(tests): document portable Lark CLI fixtures --------- Co-authored-by: Zhangchenhao-111 <285911435+Zhangchenhao-111@users.noreply.github.com> |
||
|
Hyeonsang Cho
|
74f6dce489
|
fix(dev): keep whole worktree paths when serve.sh builds DEERFLOW_ROOTS (#5856)
* fix(dev): keep whole worktree paths when serve.sh builds DEERFLOW_ROOTS
serve.sh collects every deer-flow worktree root so `make stop` / `make dev`
can reclaim the shared dev ports (8001/3000/2026) from a sibling checkout.
It parsed `git worktree list --porcelain` with `awk '{print $2}'`, but that
output does not quote paths, so a worktree at `.../deer flow two` was
recorded as `.../deer`. `_is_deerflow_pid` matches open files against
"<root>/", which the truncated root never satisfies, so a Gateway or
frontend started from that worktree was treated as a foreign process and
the start aborted with "port already in use".
Take the remainder of the line after the `worktree ` key instead. The
dedupe and longest-first ordering that follow are unchanged.
The new test builds a scratch repo with a spaced linked worktree and runs
the DEERFLOW_ROOTS block from serve.sh against it, from both the main
checkout and the spaced one.
* docs(changelog): reference #5856 in the worktree-roots entries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Hyeonsang Cho
|
bbab1e84d0
|
fix(uploads): never let a malformed files[*].size abort the run (#5855)
* fix(uploads): never let a malformed files[*].size abort the run
UploadsMiddleware._files_from_kwargs validates every field of a
client-supplied file entry fail-soft — non-dict entries, non-basename or
staging filenames, and files missing from the uploads dir are all skipped —
except `size`, which went straight to int(). A value int() cannot convert
("abc", a list, inf) raised ValueError/TypeError/OverflowError out of
before_agent (and the executor-wrapped abefore_agent), failing the whole run
before the model was called. The Gateway strips only server-owned metadata
keys and edit/regenerate deep-copies `files` verbatim, so the same entry
failed again on every retry of that message.
The size only feeds the human-readable "(N KB)" line in <current_uploads>,
so route it through _coerce_file_size: a non-negative finite number (or a
numeric string such as "2048") is kept as int; None, bools, negatives,
non-finite floats and anything unconvertible fall back to 0, the value a
missing size already gets.
Tests pin the fallback for each unusable shape, the kept values, and that
before_agent still injects an existing upload whose size is malformed.
* docs(changelog): reference #5855 in the uploads size entries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Jholly
|
8121dbcb57
|
fix(frontend): prevent run creation retries (#4725)
* fix(frontend): prevent run creation retries * test(frontend): cover safe run stream reconnection --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Grapette.L
|
63401d79fd
|
fix(community): reject bool and fractional web-search max_results like image search does (#5852) | ||
|
哈基米
|
5242fcfb63
|
fix(tools): route bound task and batch tools' sync path through the bound runtime (#5861)
bind_task_tool and _bind_batch_tool copied their process-wide singletons with only the coroutine rebound. Once get_available_tools had wrapped a singleton in place with a sync func, the copy inherited that wrapper around the unbound coroutine, so synchronous invocation bypassed the explicit SDK submitter/capacity and fell through to the process-global one -- or raised NotImplementedError in a fresh process where the singleton was never wrapped. Rebind the copy's func to a sync wrapper around the bound coroutine so both invocation paths use the runtime's own submitter and capacity. |
||
|
哈基米
|
f92118e1e9
|
fix(sandbox): list the skills root when only category mounts exist (#5857)
* fix(sandbox): list the skills root when only category mounts exist With the default (non-policy-scoped) LocalSandboxProvider layout, the four skill categories (/mnt/skills/public, custom, legacy, integrations) are mounted at separate host directories and there is no mapping for the aggregate /mnt/skills root itself. LocalSandbox.list_dir resolved the root to the unmapped host path, and the host list_dir raised FileNotFoundError before the virtual sub-directory overlay -- which exists exactly so the agent can discover the categories via "ls /mnt/skills" -- could run, so the ls tool reported "Directory not found: /mnt/skills" even with skills installed. Treat a FileNotFoundError from the host scan as an empty listing when mappings are mounted inside the requested container path, so the overlay can surface those children. Paths with nothing mounted inside them keep raising FileNotFoundError as before. * fix(sandbox): preserve missing virtual parent errors * test(screening): align PII fixture with keyed placeholders * fix(sandbox): preserve non-directory errors for virtual parents --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
yc2bgr8
|
3d921afa5f
|
feat(community): migrate infoquest client to httpx.AsyncClient (#5782)
* feat(community): migrate infoquest client to httpx.AsyncClient Replace blocking sync requests.post calls with httpx.AsyncClient in the InfoQuest community tool, as tracked in backend/docs/TODO.md planned features. fetch/web_search/image_search and their raw-results helpers are now async, keeping the 30s transport timeout and external behavior (return values, error strings) unchanged. The langchain tool wrappers in infoquest/tools.py become async functions; web_fetch offloads readability extraction to a thread (same pattern as the already-migrated jina_ai tool). Tests are updated to mock httpx.AsyncClient instead of requests.post. * test(community): use AsyncMock for infoquest fetch in relative-links test The infoquest web_fetch_tool now awaits client.fetch, so the test stub must return an awaitable instead of a sync lambda. * fix(community): follow InfoQuest API redirects like the sync client did Address the review note on #5782: requests.post follows redirects by default while httpx.AsyncClient defaults to follow_redirects=False, so a 3xx answer from the reader/search endpoints would have surfaced as 'Error: fetch API returned status 30x' (or fallen into the generic error path after raise_for_status) instead of the final content. All three AsyncClient call sites now pass follow_redirects=True. Adds regression tests asserting each client is constructed with follow_redirects=True, and documents the parity in the test contract. * test(community): fix InfoQuest mock endpoint typo |
||
|
IronMurphy
|
65ed502e54
|
fix(release): refresh backend/uv.lock when bumping the version (#5859)
* fix(release): refresh backend/uv.lock when bumping the version `scripts/bump_version.sh` rewrote backend/pyproject.toml, frontend/package.json and the Helm chart's `version`/`appVersion`, but `backend/uv.lock` also records the root package version (uv keeps its PEP 440 form, so `2.1.0-rc0` is stored as `2.1.0rc0`). The documented release step left the lock pinning the previous version, and that is not cosmetic drift: `uv lock --check` (lint CI) fails on a stale lock and `uv sync --locked` (e2e / plugin workflows) refuses the tree, so the release commit could not pass CI. RELEASING.md only mentioned the manual `cd backend && uv lock` step for release candidates. The script now refreshes the lock via `uv lock` and refuses to start without `uv` — it exits before editing anything rather than leaving a half-bumped tree whose lock CI rejects. `uv lock` stays the authority for the normalized prerelease form; the lock diff for a bump is a single line. RELEASING.md and AGENTS.md now describe the lock as part of the bump. Tests: backend/tests/test_bump_version_lockfile_sync.py (red on main, green here) pins both halves — the lock moves with the sources, and a missing uv is an error instead of a silent no-op. * docs(changelog): record the release bump lockfile fix (#5859) --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
xiaodu55
|
64e499be98
|
feat(memory): redact queued memory payloads (pii_redaction follow-up) (#5577)
* feat(memory): redact queued memory payloads (pii_redaction follow-up) Closes the remaining vector from #3190 (memory storage): request-scoped redaction leaves originals in thread state, and MemoryMiddleware queued that raw conversation into the durable extraction buffer — so the extraction model's input and the persisted facts carried raw PII even with pii_redaction.enabled. - MemoryMiddleware gains pii_redaction_config (same pattern as summarization / durable-context / title); the payload returned by _resolve_add_args() is redacted at the enqueue boundary before add()/aadd(), covering both after_agent and aafter_agent - Reuses the shared redact_text() seam from #5527 via active_pii_detectors/_Redactor/_redact_content; the buffered payload itself becomes clean, not just the extraction model's view - Wired at the lead-agent assembly site - Tests: queue-payload redaction enabled/disabled, detector toggles, message immutability, token stability across turns * fix(memory): reserve existing placeholder indices across the queued batch Review round 1 on #5577: thread state mixes raw user turns with already-redacted tool results and summaries, so a token in a later message must not collide with a new value in an earlier one. Pre-scan the whole queued batch with redactor.reserve() before allocating. Also extend the two MemoryMiddleware stubs in test_lead_agent_model_resolution.py to accept the new pii_redaction_config kwarg (CI backend-unit-tests red). * fix(memory): redact the compaction-triggered memory_flush_hook queue too Review round 2 on #5577: with memory + summarization enabled, memory_flush_hook() queues list(event.messages_to_summarize) directly into MemoryManager.add_nowait(), bypassing MemoryMiddleware entirely — and compaction removes those messages from state right after, so the after-agent redaction can never repair a raw batch queued there. Extract the batch redaction (reserve pre-scan + rebuild) into a shared redact_queued_messages() used by both MemoryMiddleware's enqueue boundary and the hook; the hook gains an optional pii_redaction_config wired via functools.partial at the summarization factory (subagents use skip_memory_flush=True and are unaffected). Regression covers the hook path enabled/disabled/absent. * test(summarization): unwrap the partial in the factory hook identity check edd916e wrapped memory_flush_hook in functools.partial to carry the pii_redaction config, but missed committing the matching update to test_factory_attaches_memory_flush_hook_by_default — a functools.partial is not equal to its wrapped function, so the identity assertion failed deterministically (caught by static review on #5577). * fix(memory): cover tool-call arguments and lenient text blocks in queued payloads Review round 4 on #5577 — two reproduced gaps still allowed raw PII into memory payloads with pii_redaction.enabled=true: - OpenViking retains the full message object, so parsed tool_calls args and provider-format arguments in additional_kwargs bypassed the content-only redaction; queued payloads now redact tool-call args recursively (and reserve them in the pre-scan). - DeerMem's format_conversation_for_update reads p.get("text") of any dict block, not only type=="text" — _redact_content now redacts every string-valued text field so a lenient consumer cannot bypass it. Both repro shapes are pinned as regressions. * fix(pii): value-derived placeholders for stable capture identity Review round 5 on #5577: sequential allocation is order-dependent — the same unchanged message re-redacted in a later batch can get a different token, which flips OpenViking's content-signature and replays already recorded turns into persistent memory sessions. Placeholders are now a pure function of the matched value (sha256-6 per category): identities stay stable across batches, enqueues, seams, and backends with no shared state and no stored mapping. - _Redactor becomes stateless; the reserve machinery (index pre-scans, _PLACEHOLDER_PATTERN) is removed along with its call sites - redact_texts keeps its signature, now a simple per-text redaction - reserve-semantics tests are superseded by cross-enqueue identity tests (two successive captures where the second batch carries higher existing placeholders); the compiled-graph echo regex matches the new format * fix(pii): 128-bit placeholder digests; redact tool-argument keys Review round 6 on #5577: - [P2] The 24-bit truncation collided distinct identities (reproduced: contact3513@ / contact3727@example.com both rendered [EMAIL_575dbb]). Retain 128 bits of the digest so separate people stay separate at any realistic volume; the compiled-graph echo regex and the AGENTS.md entry follow the new format. - [P2] _redact_strings redacted values but preserved mapping keys, so {"contacts": {"alice@example.com": "manager"}} kept the raw email as a key. Redact string keys along with values; deterministic 128-bit tokens make distinct-key collisions a hash-collision non-event, and a redacted key equal to another key merges the entries. Regression pins the email-keyed arguments shape. * docs(agents): trim PiiRedactionMiddleware entry; drop stale reserve wording The chain grew past the 98304-byte AG002 hard limit again (main growth plus the round-6 format example). Trim the entry to fit, and replace the stale reserve-indices wording with the value-derived semantics that replaced it. * docs(agents): shrink PiiRedactionMiddleware entry below the chain budget main moved (#5609) and the base comparison flipped the nearly-over chain back to an AG002 error (98441 > 98304). Trim the entry to the facts that are not documented elsewhere: seams, allowlist, order, enqueue boundary. * fix(pii): letters-only value-derived tokens; wire the factory SDK path Review round 7 on #5577: - [P2] Hex digests carry digit runs that later digit-anchored detectors re-scan, corrupting minted tokens into nested placeholders (reproduced: user280@example.com -> [EMAIL_…[PHONE_…]…], ~1/140 emails). The 128-bit digest is now encoded over a-z, so a minted token survives the full pinned detector order. - [P2] create_deerflow_agent (the SDK path) constructed MemoryMiddleware and DurableContextMiddleware without pii_redaction_config; the knob is now an explicit optional parameter threaded through _assemble_from_features (the factory reads no config files by design, so SDK callers pass it explicitly). - [Nit] hoist the Command rebuild comprehension out of the per-message loop in _redact_result. - [Nit] drop the dead _iter_tool_calls helper and Iterator import. * test(factory): pin pii_redaction_config wiring; document the parameter Review round 8 on #5577: - [P3] test_create_deerflow_agent.py had zero coverage of the pii_redaction_config parameter; two regressions now assert that create_deerflow_agent(pii_redaction_config=cfg) builds a MemoryMiddleware and a DurableContextMiddleware carrying cfg, and that the default is None (redaction off). - [Nit] document the parameter in the create_deerflow_agent docstring: None leaves redaction off; a middleware= takeover bypasses the assembly that would apply it. * fix(pii): HMAC-scoped placeholder tokens; install model-bound redaction in the SDK chain Review round 10 on #5577: - [P2] sha256(category||value) is an unkeyed, globally deterministic fingerprint: a provider or memory backend can confirm candidate values offline and link identical values across users and threads, contradicting the documented no-re-identification guarantee. Placeholders are now HMAC-derived over a deployment-scoped token_secret (new config field); the docstring documents the linkability scope for both modes. - [P2] _assemble_from_features() never appended PiiRedactionMiddleware, so the SDK knob enabled only the downstream seams while the primary model still received raw content. The middleware is now appended in the same relative position as the lead-agent base chain when the gate is on, with a model-call regression through the assembled SDK chain. * docs(pii): document token_secret; update the placeholder format examples Follow-up to the round-10 review on #5577: - config.example.yaml documents the new token_secret knob: a deployment-scoped secret makes placeholders HMAC-derived and only linkable within one deployment; leaving it unset keeps publicly computable fingerprints (documented trade). - the PiiRedactionConfig class docstring's placeholder example now matches the shipped format (128-bit HMAC digests over base-26 letters). * docs(pii): update the module docstring's placeholder format example The module docstring still showed the sequential [EMAIL_1] example; align it with the shipped value-derived format (128-bit HMAC digests over base-26 letters). * fix(pii): require a non-empty token_secret when redaction is enabled Review round 11 on #5577: with the secret optional, the default enabled configuration minted empty-key HMAC digests — publicly computable, globally linkable fingerprints of the raw values, confirmable by offline guessing for low-entropy identifiers such as phone numbers. - PiiRedactionConfig now rejects enabled=true without a non-empty token_secret (minimum 16 characters); the unkeyed mode is gone entirely. - The request-scoped redactor in _process_request now carries the configured key: it built its redactor without it, so user-message placeholders silently diverged from every other seam for the same value. The empty-key guard in _placeholder_token is the backstop that keeps every path keyed. - Module/config docstrings and config.example.yaml no longer describe the placeholders as irreversible or the secret as optional; the stale reserve-semantics wording and the "memory path not covered" note are corrected. - Regressions: validation errors for missing/blank/short secrets, and the user-message boundary minting the same keyed token as redact_text. * fix(memory): redact original_user_content on queued memory payloads Review round 12 on #5577: UploadsMiddleware preserves the raw user turn in additional_kwargs.original_user_content, but redact_queued_messages rewrote only content, tool_calls, and provider-format tool_calls — the queued copy still carried the raw value, so the memory queue and backend received raw PII with redaction enabled. The shared helper now redacts the provenance string field on the queued copy (same batch redactor, merged with the existing additional_kwargs rewrite, rebuilt via model_copy — thread-state messages stay untouched), covering both the enqueue boundary and the compaction flush hook. Regressions pin the redacted provenance field through both paths and the untouched original. * fix(memory): redact human_input_response on queued memory payloads Review round 13 on #5577: clarification replies keep the raw user answer in additional_kwargs.human_input_response.value (DeerMem reads that mapping via its read_human_input_response mirror), but redact_queued_messages rewrote only content, tool_calls, and original_user_content — the queued copy still carried the raw value into the memory queue and backend. The shared helper now rebuilds that mapping with value redacted (same batch redactor) while preserving the protocol fields (version/kind/source/ request_id/option_id/response_kind), merged with the existing additional_kwargs rewrite and rebuilt via model_copy so thread-state messages stay untouched. Regressions pin the redacted value, preserved protocol fields, the untouched original, and the flush-hook path. * chore(config): bump config_version to 47; generate token_secret on upgrade Review round 13 on #5577: existing deployments can legitimately sit at config_version 46 with pii_redaction.enabled: true and no token_secret; the mandatory-secret validator would fail their startup while config.example.yaml still said 46, so the upgrade checker considered them current and emitted no migration warning. - Bump config_version to 47 so the outdated-config warning fires for v46. - Add a v47 migration to scripts/config-upgrade.sh: when pii_redaction is enabled and token_secret is missing/blank, generate a random deployment- scoped secret (secrets.token_urlsafe) and persist it, so the upgraded deployment starts under the mandatory validation. - Regressions: a v46 enabled config is flagged outdated against the real example; the upgrade run generates a >= 16-char secret and the result passes AppConfig validation; a disabled config gets no secret. - Also switch the two v41 script tests from a hardcoded "bash" to SCRIPT_BASH/skipif, matching the documented Windows constraint and the neighboring tests (hardcoded bash resolves to the WSL stub there). * chore(helm): sync chart config_version to 47 Review round 14 on #5577: check_config_version.sh fails whenever the chart's embedded config_version lags config.example.yaml, and the chart workflow is triggered by config.example.yaml — so the v47 bump left this head failing that check. Update the values.yaml config block and the README example to 47; scripts/check_config_version.sh now passes. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Totoro
|
72933db4e9
|
fix(examples): harden the Jev fetched-content screening example (#5858)
Follow-up to #5833 after an independent review of the merged code. - A malformed answer (deeply nested JSON raising RecursionError, or a huge noul integer raising OverflowError outside the try) escaped the fail-open path, and in the per-message task group it cancelled the other messages' requests and dropped every flag of that tool call. Parsing now happens inside the try, which also catches those errors. - The 16 KiB response cap counted decoded bytes, so a compressed body could expand far past it first. Requests now ask for identity encoding and drop encoded responses. - A plain-HTTP loopback endpoint followed HTTP_PROXY/ALL_PROXY, sending the bearer key and excerpt to the proxy in cleartext. Such endpoints now ignore proxy environment variables; HTTPS endpoints keep them. Tests cover each case, including a real local server behind an unreachable proxy, and pin previously untested behavior: independent per-request deadlines, the sync hook without a task store, the >= threshold and all four named remote tools, warnings for messages that share a tool-call ID, consumed flags, and scanning past host messages. The root README no longer says the excerpt is always PII-redacted, and the example README documents the new limits and that the sync path's deadline does not cover a stalled DNS lookup. Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
Willem Jiang
|
c8d9ed1b08
|
chore(doc): updated the documents of the develope of Version 2.2 (#5862)
* chore(version):update the version to 2.2.0-dev * chore(doc):Updated the CHANGLOG with the 2.1.0 release version * chore(doc): Updated the CHANGLOG for the latest changes * chore(doc): Added the release SOP document |
||
|
Shxiao
|
39351315bf
|
fix(uploads): converge staged cleanup and docstrings with #5756 (#5763) | ||
|
yang rui
|
c814943215
|
feat(extensions): grant bounded host model invocation (#5796)
* feat(extensions): grant bounded host model invocation * fix(extensions): retain model budgets and isolate schema validation * fix(extensions): normalize provider-origin model cancellation --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
yetuge
|
b4a60fbfe2
|
fix(skills): strip section anchors from code-span resource refs (#5841)
* fix(skills): strip section anchors from code-span resource refs Markdown-link targets already drop their #fragment before the reference is resolved, but code-span refs did not: a prose reference like `references/faq.md#pricing` kept the anchor and was resolved as a literal path, producing a resource.missing finding for a file that exists and is referenced. The bare-path token scanner stops at '#', so only the code-span branch leaked fragments. Split the fragment off code-span tokens the same way the link branch does, then strip trailing sentence punctuation. Real dotted filenames keep their dots: the split happens at the first '#', and '.' is only stripped from the very end. Signed-off-by: yetuge <2219677952@qq.com> * fix(skills): prefer exact hash-bearing filenames over fragment stripping Package paths permit '#', so a code-span reference to a real file like references/C#.md was truncated to references/C by the fragment split, producing both a false resource.missing and an orphan report (the bare path token stopped at '#' the same way). Keep the whole reference in every extraction branch (link target, code span, bare path token now includes '#' in its character class) and resolve the fragment existence-aware: prefer the exact token when it names a real file and treat the suffix after the first '#' as a section fragment only when it does not. Red/green: a new test pins references/C#.md resolving to the real file while references/faq.md#pricing still strips to faq.md — it fails on the reviewed head and passes with the change; full test_skill_review_core.py is 36 passed. Signed-off-by: yetuge <2219677952@qq.com> * fix(skills): strip the fragment before normalizing the fallback path The exact-path preference checked the normalized reference, so a token like references/faq.md#/../other.md normalized to references/other.md; when that file existed the check succeeded and the fragment-removal branch was skipped, recording the wrong edge and orphaning faq.md. Check the literal token against the package files instead, and drop the fragment before the remaining path is normalized — normalizing a hash-bearing segment must never retarget an edge. Signed-off-by: yetuge <2219677952@qq.com> * fix(skills): canonicalize leading segments in the hash-filename lookup The literal-file check compared the verbatim joined path, whose leading relative segments are uncanonicalized: from references/sub/guide.md, a reference to ../C#.md joins to references/sub/../C#.md, matches no snapshot key, and falls through to fragment stripping — truncating the real references/C#.md into a false missing and an orphan. Compare against the normpath-canonicalized whole token instead, and treat the post-'#' text as a fragment whenever it is path-like (carries / or ..) so it can still never participate in normalization and retarget an edge. Signed-off-by: yetuge <2219677952@qq.com> * fix(skills): check the canonical whole path before stripping a fragment The path-like-fragment heuristic dropped any post-'#' text carrying '/': a real package file such as references/C#/readme.md (a '#' inside a directory name) was truncated to references/C, producing a false resource.missing and orphaning readme.md. Only a '..' segment in the post-'#' text forces the strip-first fallback — that is the one case where fragment text participates in normalization and can retarget an edge (faq.md#/../other.md). Every other fragment first gets the canonicalized whole-token lookup, which already canonicalizes leading relative segments for nested sources. Signed-off-by: yetuge <2219677952@qq.com> * fix(skills): always strip markdown link fragments and blind the bare-path pass to links Markdown link targets have fixed fragment semantics: the text after '#' is a URL fragment, never part of the filename, so a link to a file literally named references/faq.md#pricing would have to percent-encode it. The exact-file branch applied the hash-filename preference to link targets too, recording a valid edge and suppressing resource.missing for a broken link when a literal same-named file existed. Links strip the fragment again, and their full construct is blanked out of the residual text so the code-span and bare-path passes — literal-path contexts where '#' may be part of a real filename — never see link-internal text. Signed-off-by: yetuge <2219677952@qq.com> --------- Signed-off-by: yetuge <2219677952@qq.com> |
||
|
XuRuuuy
|
b2a2fb757a
|
fix(browserless): read reject_resource_types / reject_request_pattern from config (#5719)
* fix(browserless): read reject_resource_types / reject_request_pattern from config web_fetch_tool declared both parameters, initialised them to None, and passed them to the client -- but never read them from tool config, so they were always None on the wire. BrowserlessClient implements them (and is tested for it at `test_fetch_html_with_reject_params`), so the provider's resource blocking was unreachable without patching the source. Skipping images/media/fonts is the main lever for render latency and bandwidth against a self-hosted Browserless, and it could not be turned on from config.yaml. Read both keys through a new `_as_str_list` helper that accepts a YAML list or a comma-separated string and yields None for anything unusable, so an unparseable value omits the parameter instead of reaching the server. Both keys are now documented on the Browserless web_fetch entry in config.example.yaml. Default behaviour is unchanged: an unset key is still None, so the payload is identical to before for every existing config. Adds four regression tests covering the list form, the comma-separated form, the unset case, and unusable values. * fix(browserless): validate reject list entries --------- Co-authored-by: RXQ6 <RXQ6@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
38b82057e1
|
fix(config): sign the bytes the config cache parsed, not a second read (#5848)
* fix(config): sign the bytes the config cache parsed, not a second read _load_and_cache_app_config parsed config.yaml through AppConfig.from_file and then called get_config_signature, which opens and hashes the file again. A write landing between those two reads left the cache holding the older content under the newer content's signature. get_app_config() compares the on-disk signature against the recorded one, so from then on it saw "unchanged" and the edit only surfaced when yet another edit changed the signature — silently stale until then, with no self-healing. Read the file once through a new shared helper, file_signature.read_config_with_signature, which returns the bytes together with the (mtime, size, sha256) of exactly those bytes, and parse them through AppConfig._from_yaml_text (from_file's body, split out; from_file itself is unchanged). The recorded signature now always describes the parsed content: a write that races the load can only produce a signature mismatch on the next call, i.e. one extra reload. Signing before parsing but re-reading for the parse would not be enough — an edit that lands between the two reads and is then reverted would be served forever — so the tests pin that the parsed bytes are the signed bytes. The stale-metadata reload test now routes the loader's read through the same stale-stat simulation it already applies to the probe, since the loader no longer records the probe's result. * docs(changelog): reference #5848 in the config cache entries --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Zeren Wang
|
82671596f3
|
feat(authz): add plugin resource targets, provenance, action and management enforcement (#5842)
* feat(authz): add plugin resource targets, provenance, action and management enforcement
PR1 of the extension-tool-authorization series
(docs/plans/2026-09-24-extension-tool-authorization-spec.md): plugin
authorization plumbing, no new author-facing declaration.
- `authz/plugin_targets.py` is the single encoder/validator for composite
targets (`namespace/part`), with one validating composer behind three
kind-checked constructors; invalid input raises instead of degrading.
- `authz/plugin_authz.py` is the harness decision layer (`plugin_action`,
`plugin_management`, plus the batch filters the page slice will consume):
disabled is a no-op, an explicit deny raises, and provider errors follow
`fail_closed` like the sandbox gate.
- `POST /api/plugins/{ns}/actions/{name}` decides `plugin_action:invoke`
after the action resolves and before the body is streamed.
- `deerflow_extension_api.auth` publishes `require_plugin_management` /
`arequire_plugin_management` with two `app.state` resolver keys; the
Gateway installs both. Contract version 0.2.3 -> 0.2.4.
- `tools/tool_provenance.py` records host-written plugin provenance for
display (assembly descriptor, Layer-2 context later); no decision reads a
label.
- Plugin providers use a `(config signature, loop key)` cache so a
loop-affine provider is never handed to another loop or to a sync caller.
- RBAC gains `plugin_actions` / `plugin_management` aliases;
`config_version` 47 -> 48 across the three mirrors.
Tests: new plugin target/provenance/action/management suites plus a
blocking-IO anchor whose offloads were reverted to prove red->green.
* fix(authz): decide plugin resources from the request's config snapshot
Review round 1 on PR #5842 (willem-bd).
P1 — the decision layer no longer reloads the configuration. The six
`authz/plugin_authz.py` functions take a required `app_config` snapshot (the
same request-scoped config the Gateway used to resolve the provider), and
`resolve_/aresolve_plugin_authorization` return it as a third element, so
`enabled`/`fail_closed` can never come from a second read that drifted from
the one that produced the provider. A snapshot the caller could not read at
all is *unavailable*, not disabled: it fails closed as
`authz.config_unavailable`. The Gateway's `_plugin_app_config` now separates
an absent `config.yaml` (no gate, today's behavior) from a config that exists
but cannot be read or validated (deny) instead of swallowing both through
`safe_app_config`.
P2 — `_validated_decision` requires `type(decision.allow) is bool` on both
the sync `authorize` and async `aauthorize` paths, so a provider returning
`AuthzDecision(allow="false")` is a malformed verdict that follows
`fail_closed` instead of being read as an allow.
Both fixes are covered by regression tests that were verified red against the
pre-fix behavior (P1: 200 vs 403 and resolver True vs False; P2: 12 cases
across fail_closed/fail_open and sync/async), plus tests pinning that an
absent config and `authorization.enabled: false` keep today's behavior.
* fix(authz): treat a request-time config absence as unavailable
Review round 2 on PR #5842 (willem-bd).
P1 — `_plugin_app_config[_async]` no longer maps a missing `config.yaml` to
`None` ("no gate"). A Gateway cannot have started without a readable config
(`app/gateway/app.py` reads it through this same accessor in `lifespan`), so
absence *during a request* means the configuration this process was running on
became unavailable — a hot reload, an atomic replace, a ConfigMap remount. It is
unavailable, not disabled: the error propagates,
`resolve_/aresolve_plugin_authorization` raise
`_PluginAuthorizationUnavailable(fail_closed=True)`, the action dispatcher
returns 403 and the management resolvers answer `False`. The dead
`(None, None, None)` allow path is gone, the resolvers return a non-optional
`AppConfig` as their third element, and callers key "authorization disabled" on
`provider is None` alone. This matches the rule the decision layer already
documented (`authz.config_unavailable`) and the public docs.
P2 — `_deny_reason_code` is total: a `reasons` that is not iterable degrades the
code to `authz.denied` instead of raising `TypeError` outside the guarded
provider path, which turned a denial into an unhandled error. It is deliberately
not folded into `_validated_decision`: `allow` was a real `bool`, so the verdict
is a valid denial, and treating it as malformed would flip it to an allow under
`fail_closed: false`.
Both fixes are covered by regression tests verified red against the pre-fix
behavior (P1: resolver True vs False, async `PermissionError` not raised, route
200 vs 403; P2: `TypeError: 'NoneType' object is not iterable` at
`plugin_authz.py:113`). The action-route tests in
`tests/test_plugin_contributions.py` now install a readable
`authorization.enabled: false` config instead of relying on a config-less
process, and `docs/full-stack-plugins.md` plus the PR1 decision log record the
new rule.
* fix(authz): apply the running policy when the plugin config is unreadable
Follow-up to f03d2ea9 (review round 2 on PR #5842, willem-bd).
The first attempt read the P1 finding as "absence is never disabled" and failed
every config read closed unconditionally. That is too broad for this repo: a
config-less host is deliberately treated as having no policy to apply
(`_get_route_authorization_config` falls back to a disabled config,
`sandbox_authz.safe_app_config` documents the same no-op for "CI runners and
direct-call tests"), and the rule turned the config-less e2e plugin host
(`extension_test_fixtures.bookmark_plugin_gateway`) into a 403 — the three
`bookmark-plugin.spec.ts` cases lost the save action. Reproduced locally:
`POST /api/plugins/community.bookmarks/actions/save` against that fixture host
returned 403 before, 200 after this change.
The rule is now "has this process loaded a configuration?", not "was the last
read successful?":
- No loaded config (`peek_loaded_app_config()`, new read-only accessor on
`deerflow.config.app_config`) and no `config.yaml`: no policy to apply, so the
gate stays a no-op — unchanged for CI runners, direct-call tests and hosts
that mount only the plugins router.
- A loaded config whose file disappears or cannot be read: *unavailable*, never
*disabled*. The resolvers raise `_PluginAuthorizationUnavailable` carrying the
failure flag of the configuration this process runs on (authorization
disabled there -> no gate; enabled -> that policy's `fail_closed`, `True` by
default), so an enabled Gateway answers 403/False in the window instead of
letting protected plugin actions and management routes through.
- A config file that exists but cannot be read or validated right now: fail
closed when nothing is loaded (round 1's rule), otherwise the loaded policy
decides.
`peek_loaded_app_config()` is preferred over a Gateway-side startup snapshot: it
reflects the last successfully loaded config, so a hot-reloaded policy is what
applies during the window, and there is no lifecycle hook to keep in sync.
Evidence: the three no-config/lost-config branches in
`tests/test_plugin_management_guard.py` and
`tests/test_plugin_action_authorization.py`, round 1's unreadable-config tests
unchanged, `test_peek_loaded_app_config_survives_a_missing_file` in
`tests/test_app_config_reload.py`, and the fixture-host probe above. Docs
(`docs/full-stack-plugins.md`, `reference.mdx` en/zh) and the PR1 decision log
state the final rule; the log records the superseded variant.
* fix(authz): keep a denial whose reasons raise while being read
* fix(authz): intersect plugin batch filters with the candidates
The plugin_page/plugin_management batch helpers returned the provider's
`filter_resources` answer verbatim, so a custom provider could hand the
projections a target that was never asked about ("*", another namespace,
a phantom page). Every other filter call site re-projects from its own
candidate list; do the same here, state the intersection on the provider
contract, and pin it with a regression test that fails on the old
pass-through.
* docs(plugins): map a management denial to 403 in the route example
The contributed-route example called `arequire_plugin_management` and let
the documented `PermissionError` escape. The Gateway installs no
`PermissionError` handler, so a route copied from the example answered 500
on a normal policy denial instead of 403. Both example routes now translate
the denial, and the surrounding bullets say why the translation is the
route's job.
* docs(extensions): point the 0.2.4 contract row at the PR that ships it
Every other row in the version-history table links the pull request that
shipped the version; 0.2.4 linked the authorization tracking issue instead.
Both the English and Chinese tables now link this PR.
|
||
|
Totoro
|
f8ca969330
|
feat(examples): add opt-in advisory fetched-content screening (#5833)
* feat(examples): add opt-in fetched-content screening extension * fix(examples): screen sync and async results through supported middleware hooks * fix(examples): preserve empty message IDs when applying screening warnings * fix(examples): include screening policy in assembly identity * feat(examples): load fetched-content screening through the extension entry Package the screening example as a plugins: extension instead of an operator-configured middleware. install() validates the private config and, once enabled, contributes one TOOL_VISIBLE middleware for lead agents and subagents. The package imports only deerflow_extension_api. Extension tool wrappers are observational, so detection and the warning use two supported contracts: the tool wrapper classifies the visible result and records flagged tool-call IDs in the run's task store, and before_model replaces the matching messages of the latest tool step by ID. The excerpt sent to Jev is now the redacted, sanitized text the model sees. Runs without a task store send nothing. Unexpected local errors surface as isolation diagnostics; provider errors fail open. Tests load the package through load_extensions() and the host isolation wrapper and run real lead/subagent graphs with a task store bound as the Gateway worker binds it. The README records what the example exercised in the extension API and the gaps it found. * fix(examples): classify each tool message of a screened result on its own A Command result was classified as one concatenated excerpt and the flag landed on its first text message, so a benign first result could take the warning while an injected later result reached the model unmarked. Each text message now gets its own bounded excerpt and Jev request, for at most eight messages per tool call. The requests share one client and run concurrently in a task group, each under its own deadline, so one slow or failed request cannot hide another message's flag. Every flagged message's tool-call ID is recorded. The message cap joins the declared policy, and the README describes the per-message requests and data sharing. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> |
||
|
yetuge
|
4a92cb579a
|
fix(uploads): keep document-conversion stat and write-back off the event loop (#5830)
* fix(uploads): keep document-conversion stat and write-back off the event loop convert_file_to_markdown offloads the conversion itself for files above 1 MB (documented in its docstring), but the two bookend filesystem calls still ran on the event loop: the size stat before the threshold check and the write of the converted markdown afterwards — the latter a potentially multi-megabyte write in the per-upload ingestion path. Route both through run_file_io (the dedicated file IO pool that also keeps the caller's context), matching the off-the-loop fixes already applied to the sibling upload/document paths. Add a tests/blocking_io anchor: with the converter patched to a 2 MB payload it fails on the previous head (Blockbuster catches the blocking os.stat first; the broad except swallows it into a None return) and passes with this change. * fix(uploads): address review — drain conversion write-back, make run_file_io re-entrant Review follow-up on the off-loop conversion stat/write PR: 1. The write-back now goes through await_drained(): a plain run_file_io await cancels immediately while the worker keeps writing, so the ingestion scope could remove its private directory over an in-flight write and leave a large hidden partial file. The drain delivers the cancellation only after the worker settled, and the handler removes the partial output before re-raising. 2. run_file_io is now re-entrant: a nested offload issued from code already running inside a file-io worker (project-document conversion runs asyncio.run(convert_file_to_markdown(...)) inside a run_file_io worker) executes inline instead of submitting to the same bounded executor, which deadlocked with a single configured worker. Both fixes carry deterministic regressions: the cancellation test pins the drain ordering (cancel issued before the worker is released) and the partial-file cleanup; the re-entrancy test pins inline execution by thread identity. Both fail on the previous head. Full blocking_io + project-document + conversion suites show an identical failure set to the previous head (pre-existing Windows platform failures only). * fix(uploads): keep cancellation authoritative when conversion cleanup fails If the post-cancellation unlink of the staging output raised, the new exception escaped the CancelledError handler, was swallowed by the broad except as an ordinary conversion failure (None return), and left both the caller's cancellation and the partial file behind. Catch and log cleanup failures inside the handler and always re-raise the captured cancellation; a second cancellation still passes through untouched (CancelledError is not an Exception). Red/green: tests/blocking_io/test_file_conversion_offloop.py gains a regression where unlink fails while the write-back is in flight — on the reviewed head the task settled on None (DID NOT RAISE); with the change the CancelledError propagates and the failure is logged. Signed-off-by: yetuge <2219677952@qq.com> --------- Signed-off-by: yetuge <2219677952@qq.com> |
||
|
WuWenbo
|
cf3afe2bac
|
fix(middleware): preserve human message metadata (#5823)
* fix(middleware): preserve human message metadata * test(middleware): cover human message metadata injection |
||
|
lihongyuan99
|
33a423d375
|
docs(backend): drop references to APIs that do not exist (#5798)
* docs(backend): drop references to APIs that do not exist PATH_EXAMPLES.md's upload example imports THREAD_DATA_BASE_DIR from thread_data_middleware; the module exists but exports no such name, so the snippet fails with ImportError as written. Resolve the same directory through the accessor the middleware itself uses, get_paths().sandbox_uploads_dir(), which yields the layout the file's own path table documents. CONFIGURATION.md's Docker sandbox block lists `auto_start: true`. SandboxConfig declares use/port/container_prefix/mounts/network but not auto_start, and the provider is constructed as cls(**kwargs) from the caller of get_sandbox_provider() rather than from config extras, so the key is accepted by extra="allow" and then silently dropped. Remove the line from the example. * docs(backend): resolve the uploads example through the owner-scoped bucket get_paths().sandbox_uploads_dir(thread_id) without a user_id selects the legacy threads/<id>/ bucket, but Gateway uploads land in users/<user_id>/threads/<id>/user-data/uploads/, so the example imported cleanly yet still raised FileNotFoundError for a normally uploaded file. Use deerflow.uploads.manager.get_uploads_dir(), which resolves the effective user and validates the thread id. * docs(backend): show the real uploads path layout in every PATH_EXAMPLES demo The remaining sections still documented the pre-isolation threads/<thread_id>/ bucket and called `path` a backend-relative string, while the upload and list APIs return an absolute path under users/<user_id>/threads/<thread_id>/user-data/uploads/. Reproduced with deerflow.uploads.manager.get_uploads_dir() and uploads._list_uploaded_files_for_thread() against DEER_FLOW_HOME: the listed path is /<home>/users/default/threads/abc123/user-data/uploads/report.pdf. * docs(backend): clarify runtime home precedence --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
3448c3426e
|
fix(config): honor $VAR substitution for request_admission integer fields (#5838)
* fix(config): honor $VAR substitution for request_admission integer fields AppConfig.resolve_env_variables replaces every `$VAR` reference with the raw environment string. RequestAdmissionConfig.requests_per_minute and max_queue_size are declared `strict=True` so a bool or float cannot slip in, but strict mode also refuses the string "60", so `requests_per_minute: $RPM` failed the whole config load with "Input should be a valid integer" even when RPM=60 — contradicting config.example.yaml's promise that environment variables work for every field. They were the only strict fields in the config schema; every other integer coerces in lax mode. Wrap both fields in a BeforeValidator that converts a decimal integer literal arriving as a string and leaves everything else untouched, so the strict check behind it still rejects bools, floats, "60.0" and non-numeric strings. Tests pin the accepted literals, the rejections that must survive, and the end-to-end path from a config.yaml `$VAR` reference through AppConfig.from_file. * docs(changelog): reference #5838 in the request_admission entries |
||
|
lakers-abing
|
3a86278047
|
fix(mcp): preserve backslash paths during bare-filename rewriting (#5815)
* fix(mcp): preserve backslash paths during bare-filename rewriting * test(mcp): cover the trailing backslash boundary in bare-filename rewrite Review noted that treating `\` as a separator also suppresses a rewrite when a single trailing backslash precedes a newline, which is more often a Markdown hard break, and that neither the tests nor AGENTS.md covered that position. Lock the behavior in through the production result converter and document the tradeoff: a backslash is a legal filename character on POSIX, so forgoing that one rewrite is preferable to risking a corrupted path reference. Runtime behavior is unchanged. --------- Co-authored-by: lakers-abing <236384465+lakers-abing@users.noreply.github.com> |
||
|
RongJie G
|
842f561054
|
fix(runtime): drain agent stream close across cancellation (#5806)
* fix(runtime): drain agent stream close across cancellation * docs(runtime): condense stream teardown guidance * fix(runtime): surface stream close failures on cancellation --------- Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Hyeonsang Cho
|
174c6f7fd3
|
fix(mcp): run lazy MCP discovery once when initialization itself raises (#5836)
* fix(mcp): run lazy MCP discovery once when initialization itself raises get_cached_mcp_tools() wrapped the whole lazy-initialization call in the `except RuntimeError` that was only meant to catch asyncio.get_event_loop() raising when no loop is set. run_until_complete() and Future.result() re-raise the coroutine's own exception, so a RuntimeError from discovery — e.g. the deliberately re-raised McpTaskConfigurationError — took the same fallback and ran asyncio.run(initialize_mcp_tools()) a second time, respawning every stdio server and re-fetching OAuth tokens before giving up. Inside a running loop that fallback fails with "asyncio.run() cannot be called from a running event loop", so the logged traceback hid the real cause. Resolve the loop first, in its own try/except, and drive initialization exactly once: asyncio.run when no loop is set or the current loop is closed, a worker thread when the loop is running, run_until_complete otherwise. Any failure is logged once and returns [] as before. Regression tests pin one discovery call and the real logged cause with and without a running loop, and keep both fallbacks (no loop set, closed loop) covered. * test(mcp): surface worker-thread failures in the no-loop fallback test Thread.join() does not propagate exceptions, so a regression of the asyncio.run fallback failed this test with a bare KeyError: 'result' instead of the real traceback. Capture sys.exc_info() in the worker and re-raise it on the main thread before the assertions. |
||
|
Zeren Wang
|
c943d953cd
|
docs(checkpoints): add a ten-chapter checkpoint storage manual in both languages (#5845)
* docs(checkpoints): add a bilingual checkpoint storage manual Add the missing docs-site coverage for dual-mode checkpoint storage (`database.checkpoint_channel_mode: full | delta`) as a ten-chapter manual under `harness/checkpoints/` in both languages: concepts, quick start, channel modes, snapshot cadence, history cache, resume and rollback, operating checkpoints, observability, troubleshooting by symptom, and a reference of every configuration key, error message, HTTP status mapping, public symbol, load-bearing test, and pinned upstream defect. The index page is `asIndexPage`, so `/docs/harness/checkpoints` is the section itself and its `_meta.ts` lists only the nine child chapters. Discovery: `harness/_meta.ts` registers the section, the harness configuration module table gains a `database` row, and the thread persistence section of `agents-and-threads` links to the manual. Content is grounded in the code. Only the three measured values recorded in `backend/docs/runtime-guidance-details.md` are quoted; the upstream-derived 41x storage figure in the untracked delta-channel RFC is deliberately not cited. Pinned upstream defects are documented with their issue and PR numbers and the versions they are pinned at (`langgraph 1.2.9`, `langgraph-checkpoint 4.2.0`, `langgraph-checkpoint-postgres 3.1.2`). * docs(checkpoints): correct the history-cache restart requirement The history cache is built with the checkpointer at Gateway startup (`langgraph_runtime()` -> `make_checkpointer()` -> `make_checkpoint_cache()`), so editing `database.checkpoint_cache.*` does not change a running process. The manual previously marked the whole section "not restart-required", which conflated "not frozen for correctness" with "hot-reloadable" and could mislead an operator changing capacity, credentials, or retention. Distinguish the two properties everywhere the section is described: history-cache.mdx marks every field as needing a restart and adds a callout explaining the difference, reference.mdx redefines the Restart column and fixes the five rows and the effect list (adding the `asyncprov` evidence alias), troubleshooting.mdx replaces the "two keys need no restart" claim and gains a dedicated symptom entry, and observability.mdx notes that changed counters need a restart. The Chinese text is corrected in parallel. Verified: the docs link test and the docs unit suite pass, frontmatter parses for all 20 pages, and every in-page anchor resolves on the rendered English and Chinese history-cache, troubleshooting, reference, and observability pages. * docs(checkpoints): account for periodic snapshots in the storage-growth claim Every periodic snapshot stores the entire message list as of that step (`_DeltaSnapshot(ch.get())`, persisted as a checkpoint blob), so with a fixed snapshot frequency K the retained total after N turns is O(N²/K + N), not O(N). The manual claimed unconditional linear growth, which understated capacity planning — the more so because nothing in DeerFlow prunes those snapshots yet. Distinguish per-step delta writes from total retained storage in index.mdx (prose and table), modes.mdx (step-cost row), and operations.mdx; note in cadence.mdx that K is also the capacity knob that scales the snapshot bytes. The English and Chinese chapters are corrected in parallel. The reference chapter keeps the released #4292 wording as a verbatim quotation and now names the retained-storage bound beside it. Verified: `_DeltaSnapshot(ch.get())` and its blob serialization in the pinned langgraph (channels/delta.py:196-198, checkpoint/serde/types.py:19-25) confirm the premise; the docs link test and docs unit suite pass; all ten changed pages render with resolving anchors and no stale linear-growth wording. * docs(checkpoints): correct the run-error surfaces and repair two evidence pointers Review follow-ups on the checkpoint manual, both languages. Error-surface corrections: - Run creation is not always asynchronous: when a thread's run-event feed is empty but a checkpoint head exists, ensure_checkpoint_history_seeded reads that head inside start_run, where only ConflictError (409) and UnsupportedStrategyError (501) are mapped, so a mode mismatch returns a plain 500 before admission. Troubleshooting, observability and the reference route tables now scope the "does not pre-gate" claim. - The 503 symptom is retargeted. Only Gateway startup, make_lead_agent and DeerFlowClient.__init__ call the freeze helpers, so a config.yaml edit cannot make a state/history/thread route answer 503. A mode edit is injected over by the run worker and ignored until a restart; a cadence edit fails the next run with the restart-required message. The router's 503 mapping is marked defensive in modes and the reference. - The regenerate section title no longer groups the state endpoints, which map a mismatch to 409. - operations: the Textual TUI catches the mismatch through stream_actions(); only the headless --print/--json runners print a traceback. Evidence-pointer repairs: - reference: this branch's CHANGELOG insertion shifted the #5734 entry from 3032-3039 to 3040-3048; the citation is re-pointed. - reference: the #8548 row cited an untracked worktree plan file; replaced by the in-repo test anchor, and "mirror issue #8551" reworded to "related". - reference: new `agent` alias for the agent-factory freeze sites. - history-cache: the bare `deps:`/`asyncprov:` aliases are expanded to full paths; the alias legend lives only on the reference page. * docs(checkpoints): say a delta fork is accepted, not refused "Delta mode cannot fork" overstated the constraint. Nothing in the stack rejects the fork: LangGraph stores the second child, no saver checks for an existing sibling, and ensure_checkpoint_mode_compatible returns immediately when the process mode is delta. What breaks is the read — the walk replays the abandoned sibling's writes stored under the shared parent. Reproduced on the pinned stack (langgraph 1.2.9 / langgraph-checkpoint 4.2.0, AsyncSqliteSaver) by running this repo's own #4458 fixture without the linearization call: the forked resume returns ["h1", "a1", "h2", "a2", "a2-new"] instead of ["h1", "a1", "h2", "a2-new"], i.e. the superseded answer comes back. - resume-and-rollback: title/description/heading now say "cannot fork correctly"; new "Forks are accepted, not refused" subsection covering the unguarded write side. - troubleshooting: the "old answer reappears" cause states the same split. - reference: the POST /threads/{id}/state row notes that a client-supplied checkpoint_id older than the head is applied as a delta fork with no linearization — the write shape upstream tracks as #8551 (open), which the run-path linearization does not cover. * docs(checkpoints): scope the unguarded state-update fork to what the probe showed The previous commit said a mutation against an older checkpoint stores its writes under the shared parent and implied the sibling branch suffers for it. Probing the pinned stack (AsyncSqliteSaver, delta messages channel) shows: - the write does land under the shared parent and joins the write set the parent's other children walk (raw walk for a sibling includes the abandoned head's own write); - but reducer channels are written with Overwrite, which replaces the accumulated value, so the forked branch reads clean and the abandoned branch is unchanged in that shape — #8551 did not reproduce; state the masking mechanism and the upstream tracking issue, not a corruption claim. * docs(checkpoints): fix a frontmatter description that broke the docs build `resume-and-rollback.mdx` shipped `description: Delta mode cannot fork a thread correctly: a forked delta read ...` in the previous commit. The unquoted ": " turns the value into a nested mapping, so Nextra's YAML frontmatter parse failed and the e2e lane could not start its web server (YAMLParseError at line 2, column 14) — the exact failure class this manual warns docs authors about, introduced by me two commits ago. - Replace the colon with an em dash in the English description (the Chinese one uses a full-width colon and was unaffected). - Add tests/unit/content/docs-frontmatter.test.ts: every `src/content/**/*.mdx` frontmatter block must be YAML-parseable, implemented as the one rule that actually breaks it (an unquoted scalar containing ": "), because the unit suite never compiles MDX and only a build or a dev render catches it. Verified the guard fails on the broken line and passes on the fix. Verified with a clean `BETTER_AUTH_SECRET=local-dev-secret pnpm build` on this tree: all pages prerender, resume-and-rollback EN/ZH included. |
||
|
FanouZeng-TT
|
55f5d6b629
|
fix(sandbox): clamp negative read_file range bounds (#5655)
* fix(sandbox): clamp a negative start_line in provider read_file LocalSandbox.read_file clamps the start line with max(start_line or 1, 1), but e2b, OpenSandbox and BoxLite used start_line or 1, so a negative value reached Python's negative-index slicing and read from the end of the file instead of the first line. Clamp all three the same way, and pin the boundary cases in their existing range tests: a past-EOF start still returns an empty string, a negative start reads from line 1. Surfaced by the review of #5616 (e2b has the same gap, consistency nit). * fix(sandbox): clamp negative end_line across providers Complete the range-bound consistency fix for e2b, OpenSandbox and BoxLite: clamp a negative end_line to zero like LocalSandbox instead of letting Python's negative-index slicing silently drop trailing lines. Add the symmetric boundary assertion to each provider's existing range test. |
||
|
dong
|
3d5cc1efae
|
fix(runtime): persist consumed middleware tool results (#5671)
* fix(runtime): persist consumed middleware tool results * docs(agents): shorten journal reconciliation guidance --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Jin
|
e68b4e14e4
|
fix(gateway): treat a blank GATEWAY_WORKERS and Lark broker port/timeout as unset (#5840)
Follows #5801 for the remaining numeric env reads that break on an empty value (`-e VAR=`, or `${VAR}` with VAR unset): - The WeChat QR worker guard read a blank GATEWAY_WORKERS as an invalid count and refused QR login, although the compose command starts one worker for it (`--workers ${GATEWAY_WORKERS:-1}`) and the other worker checks treat it as 1. A blank value now falls through to WEB_CONCURRENCY and then 1, so the guard still fails closed for real multi-worker setups. - The Lark broker aborted startup on int("") for a blank DEERFLOW_LARK_BROKER_PORT or DEERFLOW_LARK_BROKER_TIMEOUT; it now uses the defaults. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
yang rui
|
1da284bb3a
|
feat(prompts): add operator overlays for core system prompts (#5794)
* feat(prompts): add operator overlays for core system prompts * test(prompts): isolate SOUL forwarding test from operator config * fix(prompts): wrap complete subagent system message with overlay * fix(subagents): serialize prompt overlays in durable batches --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Jin
|
71a115d926
|
fix(exa): coerce configured result limits before calling the SDK (#5839)
* fix(exa): coerce configured result limits before calling the SDK
`$VAR` references in config.yaml resolve to strings, and exa-py rejects a
string `num_results` ("Invalid value for option 'num_results': 10. Expected
one of [<class 'int'>]"), so `max_results: $EXA_MAX_RESULTS` made every Exa
web search return an error. `contents_max_characters` reached the API as a
string the same way.
Coerce both like the DDG and SearXNG providers do: parse an integer, and warn
and keep the default for a blank, non-numeric, zero or negative value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(exa): reject fractional and boolean limits instead of truncating them
Review follow-up: int() turned an unquoted YAML `3.5` into 3 and `true`
into 1, so a misconfigured limit was used with a different value. Accept
only non-boolean integers and integer-form strings; everything else warns
and keeps the default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(feishu): stabilize file batch timing
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
lihongyuan99
|
2a76c9db71
|
docs(harness): make the custom memory storage guide followable (#5844)
`deerflow.agents.memory.storage` does not exist (memory/__init__.py says DeerMem-private symbols are imported from deerflow.agents.memory.backends.deermem.deermem.core.*), the documented class shape cannot be built by create_storage() (it instantiates cls(config)) or called by the memory updater (load/reload/save all pass user_id=, save also expected_revision=), and storage_class is split on the last dot, so the `module:Class` value the guide shows never resolves. memory.mdx also promised a fallback that the code explicitly refuses. |
||
|
林SO
|
bc262743d7
|
fix(slack): log Socket Mode connect failures instead of dropping them (#5640)
* fix(slack): log Socket Mode connect failures instead of dropping them SlackChannel.start() ran SocketModeClient.connect() through loop.run_in_executor() and discarded the returned future. When the connect raised (rejected app token, network error, TLS failure) the exception was stored on a future nothing awaited, so the only trace was asyncio's "Future exception was never retrieved" at garbage collection. Meanwhile the channel had already logged "Slack channel started" and reported itself running. Route the executor call through _connect_socket_mode(), which logs the failure with logger.exception() the way the Telegram and Discord channel threads already do. The regression test injects a failing connect and asserts the error reaches the channel logger. * fix(slack): bind queued connect to its client --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
xbzz1018
|
29dbce45a1
|
fix(client): exclude prior turns from resumed stream events (#5811)
* fix(client): exclude prior turns from embedded stream events * docs(client): tighten streaming guidance --------- Co-authored-by: xbzz1018 <186267668+xbzz1018@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
illusion
|
83c42a61e7
|
fix(skills): strip trailing sentence punctuation from extracted resource refs (#5739)
* fix(skills): strip trailing sentence punctuation from extracted resource refs
The resource-graph extractor's _PATH_TOKEN_RE includes '.' in its path
character class, so a bare package-relative path at the end of a sentence
(e.g. "Read references/setup.md.") is extracted with the trailing period
still attached. build_resource_graph then emits a false-positive
resource.missing finding for "references/setup.md." and, because the
real file is never matched, reports the actual references/setup.md as an
unreferenced orphan.
Other sentence punctuation (",;:!?()") already falls outside the regex
character class and is correctly dropped; only '.' is ambiguous because it
is both a valid path character and prose punctuation.
Strip trailing '.', '?' and '!' from every extracted reference in
_extract_references. Real dotted filenames are unaffected: rstrip only
removes punctuation at the very end, so references/config.yaml and a link
target such as references/v1.0.md keep their dots.
Adds two regression tests covering prose-terminated bare paths and real
dotted filenames.
* style(tests): format skill review tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Nan Gao
|
bcaf6db36a
|
feat(models): add declarative reasoning capability contract (#5780)
* feat(models): add declarative reasoning capability contract (#5073) Introduce an optional per-model `reasoning:` block beside the legacy `supports_thinking` / `supports_reasoning_effort` booleans. It declares thinking availability (unsupported | optional | required), the accepted effort values with aliases, default, and serialization path, the payload dialect, and the reasoning-history requirement. When present the booleans are derived from it and contradictory profiles fail at config load. `deerflow.models.reasoning` normalizes legacy and declared profiles into one immutable contract and resolves every caller's generic request against it. `create_chat_model` is the single enforcement point: required-thinking models never receive a synthesized disable payload, unsupported effort values never reach the provider, and the dialect synthesizes the on/off payload when no template exists. Legacy profiles keep the historical path byte-for-byte. The lead agent and subagent descriptor resolve the same policy first so run metadata reports effective values. `/api/models` and `DeerFlowClient` project the contract as `reasoning`; the frontend derives mode clamping and effort choices from it through `core/models/reasoning.ts`, and user preferences accept provider-specific effort tokens. The setup wizard's Z.AI GLM-5.3-Flash profile migrates to the contract, restoring its low/high/max effort control. * docs(models): move the reasoning contract guidance into models/AGENTS.md backend/AGENTS.md sits within 46 bytes of the 96 KiB effective-chain hard limit for the middlewares, runtime, sandbox, and subagents guides, so the new section pushed scripts/check_agent_guidance.py over it. The models guide is the nearest file for the factory and the contract module, and its chain has ample room. * fix(models): address review on the reasoning contract - vLLM dialect: the enable path now mirrors whichever chat_template_kwargs switch the template declares, like the disable path already did, so a template using the older `thinking` key no longer also grows `enable_thinking`. - Validate operator-supplied effort values at `effort.path` inside `when_thinking_enabled`, `when_thinking_disabled`, and the `thinking` shortcut against the contract, not only the profile-level value; they are forwarded when the caller chooses nothing, so the "unknown values never reach the provider" guarantee now holds for them too. - Constrain `effort.path` to dotted identifiers and reject single-segment paths that would replace a whole mapping (`extra_body`, `thinking`, `model_kwargs`, `default_headers`, `default_query`). - Z.AI GLM-5.3-Flash profile: `default: high` instead of `max`, because the default also governs summarization, title generation, and subagent calls that never choose an effort; `max` stays selectable. Documented in the config example, the configuration manual, and the design note. * fix(models): close reasoning contract review gaps * fix(models): preserve native Ollama reasoning config * fix(models): keep native reasoning strings and project reasoning into the descriptor - `ModelConfig.reasoning` also accepts the level string ChatOllama takes for gpt-oss style models (`low` / `medium` / `high`); the factory forwards both the boolean and the string on the legacy path, so profiles that passed `reasoning: high` through before the contract field existed keep loading and keep sending it. - The assembly descriptor no longer excludes `reasoning` from `model_parameters`: the native ChatOllama value and a declared contract's dialect/history/effort change the request payload, so they now move the assembly fingerprint. - e2e: the sidecar spec pinned the pre-contract body where a model without effort control still received `reasoning_effort: minimal`; the preset is now dropped for such models (the Gateway discarded it anyway), so the assertion expects the key to be absent. |
||
|
YxinMiracle
|
7d9025c052
|
fix(sandbox): scope viewed image host reads to the current thread (#5824)
* fix(sandbox): scope viewed image host reads to the current thread * docs(gateway): trim agent guidance to fit CI limit * fix(images): honor thread data paths for saved host copies * test(images): align authorization fixture with thread paths --------- Co-authored-by: YxinMiracle <“939157765@qq.com”> |
||
|
ZJPex
|
c1149ebd0d
|
feat(tools): add stable pagination to historical upload discovery (#5570)
* feat(tools): add bounded continuation for uploaded file discovery * docs(tools): use English for upload pagination schema * refactor(tools): remove redundant upload symlink check --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Grapette.L
|
f250d11e73
|
fix(memory): reject a Honcho base_url that can never resolve (#5607)
* fix(memory): reject a Honcho base_url that can never resolve
HonchoConfig only did str(...).rstrip('/') on backend_config.base_url, so a
scheme-less or non-http value was accepted at Gateway startup and then made
every memory read return empty and every write drop silently, with only a
per-operation WARNING. httpx turns 'localhost:8000' into the bogus scheme
'localhost:'. The same check already exists in the mem0 and OpenViking
backends, and honcho/README.md plus HonchoMemoryManager.from_config both
promise a bad URL fails fast at startup.
* fix(memory): cover the scheme-case and host-less variants of the guard
- drive the api_key-over-plain-HTTP check off urlsplit(...).scheme: urlsplit
lowercases the scheme while startswith("http://") does not, so
"HTTP://internal:8000" reached httpx over plain HTTP with the key set and
allow_insecure_http still False
- read hostname instead of netloc in the base_url guard, so an authority
without a host ("http://", "http://:8000") is rejected at config time like
the scheme-less shape this PR already removes
- test direct HonchoConfig(base_url=...) construction, the path the guard's
placement in __post_init__ exists to protect
|
||
|
Zhengxu (Joshua) Jin
|
39726573aa
|
docs: fix the documented skill and MCP enable/disable endpoints (#5778)
The docs site pointed at POST /api/extensions/{skills,mcp}/{name}/enable
and /disable, and backend/docs/API.md at POST /api/skills/{name}/enable
and /disable. None of these routes exist. The gateway toggles a skill
with PUT /api/skills/{skill_name} and an MCP server with
PATCH /api/mcp/config, both admin-only, which is also what the
frontend calls.
|
||
|
xbzz1018
|
f83bc05a5f
|
fix(mcp): resolve generated file paths containing spaces (#5819)
* fix(mcp): resolve generated file paths containing spaces * fix(mcp): avoid partial rewrites at whitespace suffixes --------- Co-authored-by: xbzz1018 <186267668+xbzz1018@users.noreply.github.com> |
||
|
Wenchao An
|
df63139321
|
feat(plugins): add standalone Jev context pruning example (#5731)
* feat(plugins): add standalone Jev context pruning example * fix(plugins): address Jev pruning review findings * fix(examples): package Jev plugin with DeerFlow license * fix(examples): address Jev plugin review feedback * docs(examples): remove unrelated Jev project reference * test(examples): isolate Jev live script import path * Update branch pattern for push event * Fix YAML syntax for branch pattern --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Daoyuan Li
|
f242164604
|
fix(uploads): preserve link errors during staged cleanup (#5756)
* fix(uploads): preserve link errors during staged cleanup * Keep published uploads successful when staging cleanup fails * docs: keep upload guidance within chain budget --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Zeren Wang
|
577c39643b
|
feat(guardrails): add TypeSafe (Jev) pre-execution tool risk gate (#5712)
* feat(guardrails): add TypeSafe (Jev) pre-execution tool risk gate Adds TypeSafeGuardrailProvider, a network guardrail provider that asks one `noul` question per probed tool call and denies calls whose Jev risk probability reaches `threshold`. It reuses the existing GuardrailProvider slot: no middleware, config-schema or dependency changes. Semantics that are load-bearing: - state is strict JSON (no `default=`, `allow_nan=False`, string keys only); unserializable arguments, lone surrogates among them, and argument text above `max_state_chars` are denied locally with zero network requests, never truncated and sent. - the async path cancels in-flight requests through `asyncio.timeout`; the sync path checks the deadline after the response headers arrive, around the body read and before accepting the result, and drops late results. No total return-time bound is promised on the sync path. - clients are created per evaluation inside a context manager, and the injected `transport_factory` is called once per network-needing evaluation: httpx closes an injected transport with its client, so a shared instance would be dead on the second call. - denial reason messages carry probability, threshold, served model version and state digest, so the threshold comparison can be replayed from the run journal; local denials record no probability or model version, and allow decisions stay unpersisted (the audit path is unchanged). - provider errors carry no response body or arguments: malformed responses report the field and the offending type only, so a response echoing tool arguments cannot reach middleware logs or the evaluation report. The online evaluation harness (`backend/scripts/eval_typesafe_risk_gate.py` plus its labeled case set) is the threshold/tools evidence that must precede enabling interception. It reports four mutually exclusive populations (uncached network, cache hits, unprobed tools, local refusals), separates model classification from local refusals and fail-closed errors, measures latency from the caller entering `aevaluate` through cleanup, discloses censored samples, probes DNS/TCP/TLS separately, and flags unprobed tools as a coverage gap rather than a pass. Real responses are never written to disk. Cache-pass samples are classified from `decision.metadata['cached']` and counted once, warming calls are retained as an auxiliary group with their failures named in the report and JSON, and an optional connection-probe diagnostic can never discard completed evaluation results. Tests: the provider suite over MockTransport plus the middleware integration, a blocking-IO anchor that drives `aevaluate` against a real loopback HTTP server with a meta-check proving the sync path on the loop trips the gate, and harness cases covering the reporting defects. Every fix above was verified to fail its new tests when reverted. Docs: GUARDRAILS.md gains Option 4 (privacy, local-refusal trade-off, deadline semantics, audit boundary, coverage/bypass risk, Allowlist->RBAC migration, lone-surrogate local refusals and the no-response-body property of error messages) and config.example.yaml gains a smoke-scope example. Config: the new example block bumps `config_version` 46 -> 47, with the Helm chart's embedded version and README example kept in step as `scripts/check_config_version.sh` (chart.yaml CI) requires. * feat(guardrails): whitelist tool calls inside the TypeSafe risk gate Static allow rules now live in the provider instead of waiting on a composite provider. `whitelist` mirrors `AllowlistProvider.allowed_tools`: omitted means "not enforced", `[]` refuses every tool, and a list refuses every tool outside it locally (`typesafe.tool_not_whitelisted`) before state construction, caching and networking, independently of `fail_closed`. Whitelisted tools are still probed and still denied when their risk probability reaches `threshold`, and the whitelist is checked before the probe scope so an unlisted tool cannot inherit an allow from being unprobed. Deny rules stay with `authorization.*`. The policy version moves to 1.1.0, the evaluation script reports whitelist refusals as their own population, and `GUARDRAILS.md` / `config.example.yaml` document the allow-half migration. * fix(guardrails): address TypeSafe risk gate review findings - Status errors report only the numeric status. The HTTP status line is server-controlled like the body, and this message reaches middleware logs and the evaluation report, so a hostile endpoint could echo the state (raw tool arguments) through it. - Expired cache entries are removed with `pop(key, None)`. Sync evaluation runs on executor threads when a turn produces parallel tool calls, so a second caller can remove the same expired entry; a bare `del` raised KeyError, which the middleware reports as a provider error and, under fail_closed, turns into a spurious denial. - The evaluation harness states its acceptance gates inline (no missed dangerous call, system false-block <= 5%, uncached network p95 <= 1s) instead of citing a design spec that is not in the repository, and reports the whitelist-refusal population alongside the other four. - `--tools` / `--whitelist` strip whitespace and drop empty entries, so `--tools "bash, write_file"` no longer probes only bash while treating `" write_file"` as an unknown tool name. - The case set's `padding` note says top-level argument fields only, matching what the runner implements. Each fix was verified by running its new test against the pre-fix code. * refactor(guardrails): name the TypeSafe permission list allowed_tools Align the key with `AllowlistProvider.allowed_tools` so an allowlist migration copies it verbatim instead of translating `allowed_tools` into `whitelist`. The companion names follow: reason code `typesafe.tool_not_allowed`, metadata `tool_not_allowed`, harness population `not_allowed`, and the `--allowed-tools` flag. Semantics are unchanged: omitted = not enforced, `[]` = refuse every tool, tools outside the list are refused locally before any state, request or cache entry, and listed tools still face the risk gate. * fix(guardrails): record only a bounded served model token; fail the eval exit code Two findings from the re-review at d3f0b072. `model` is response content, and `_model_decision` interpolated it verbatim into the reason message -- which reaches the denied ToolMessage, the run journal, middleware logs and the evaluation report on disk, and which the cache retains for replay. A malformed or hostile endpoint can put an echoed tool argument, injected text or a multi-megabyte string there. The token is now recorded as itself only when it matches a conservative shape; anything else is recorded as `unrecorded:sha256:<digest>`, which keeps the verdict and a stable provenance record without echoing the value. Rejecting the response instead would turn an odd version string into a fail-closed denial of every call the endpoint answers. The digest path encodes with `surrogatepass` so an escaped lone surrogate -- a value `json.loads` happily produces -- cannot raise where this must not fail. `scripts/eval_typesafe_risk_gate.py` returned 0 even when a gate failed, so an operator scripting `... && enable` got a green light on a failing evaluation. `main()` now returns 1 when any gate in `report["score"]["gates"]` fails, after the JSON report is written, and prints the failed gate names. Tests: `test_an_unrecordable_served_model_is_digested_instead_of_echoed` (an echoed argument, a space, a lone surrogate and an oversized value, plus the cached replay) and `test_main_exit_code_carries_the_gate_verdict` (both directions). Both fail against the pre-fix source. docs/GUARDRAILS.md audit boundary updated to match. 113 passed: tests/test_typesafe_guardrail_provider.py, tests/test_typesafe_risk_eval_script.py, tests/blocking_io/test_typesafe_guardrail_provider.py. * fix(evals): fail the risk-gate run when a labeled population is uncovered Review finding on dcd38cc3. The new exit code made `_score`'s vacuous passes load-bearing: unprobed calls are excluded from the score, so `risky_misses_zero` is true over an empty risky population. `--tools read_file` excludes all 18 risky cases in the bundled set, so the run printed every gate as PASS and exited 0 after evaluating no risky operation at all. `_score` now gates coverage itself: `risky_coverage_present` and `safe_coverage_present` fail when the probe scope leaves that labeled population with nothing evaluated, so a scope that cannot substantiate the verdict no longer reports one. The safe side had the same vacuity -- `system_false_block_within_target` passes trivially with zero safe cases. Verified end to end against a loopback mock endpoint with the bundled case set: `--tools read_file` exits 1 with `GATE risky_coverage_present: FAIL` while the other four gates still PASS, where dcd38cc3 exits 0 on the same command. Tests: `test_main_fails_when_the_tool_scope_leaves_no_risky_case` and `test_main_fails_when_no_safe_case_is_in_scope` (both fail against dcd38cc3), `test_bundled_case_set_keeps_both_labels_the_coverage_gates_require`, and the exit-code test now runs with both populations present. 116 passed: tests/test_typesafe_guardrail_provider.py, tests/test_typesafe_risk_eval_script.py, tests/blocking_io/test_typesafe_guardrail_provider.py. |
||
|
Stellar鱼
|
a89cd2682c
|
fix(sandbox): reject overflowing ownership lease TTL (#5104)
* fix(sandbox): reject overflowing ownership lease TTL * fix(sandbox): bound Redis ownership lease TTL * fix(sandbox): reserve Redis lease expiry headroom * fix(sandbox): reject sub-millisecond Redis lease TTL * fix(sandbox): preserve fractional Redis lease TTLs * docs(sandbox): align ownership TTL constraints * docs(tests): align ownership TTL comment --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |