mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-09 21:49:37 +00:00
3165 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Stellar鱼
|
468eab4b5d
|
fix(frontend): stabilize model load error feedback (#5021) | ||
|
Wuong
|
e12925458a
|
feat(streaming): make heartbeat interval configurable (#5017)
Co-authored-by: Wuong <26929475+Wuong@users.noreply.github.com> |
||
|
hataa
|
2f8d1cfc21
|
fix(subagents): harden background-task registry and capacity snapshot edge cases (#5086)
* fix(subagents): harden background-task registry and capacity snapshot edge cases - execute_async drops the just-registered background entry when submitting to the isolated loop fails. The caller sees the exception and never polls, and cleanup_background_task refuses non-terminal entries, so the entry would otherwise stay as a PENDING zombie forever. - SubagentExecutionCapacity.snapshot derives queued from len(_waiters) instead of iterating it. snapshot is read from non-loop threads (e.g. configure_subagent_execution_capacity) while the loop thread mutates the deque, so iteration can raise 'deque mutated during iteration'. The raw length may count a waiter that just timed out but has not removed itself yet, which only makes the busy-check more conservative. * fix(subagents): close failure-path gaps around background submit Address both review findings on the background submission lifecycle: - execute_async() copies the isolated-loop context before registering the _background_tasks entry, so a context-copy failure (callback- manager copy or loop-bound handler filtering) can no longer strand a permanent PENDING entry the caller will never poll. - _submit_to_isolated_loop_in_context() resolves the loop before calling the coroutine factory. As direct run_coroutine_threadsafe arguments the coroutine was created first, so a loop-startup failure stranded a never-awaited coroutine (RuntimeWarning + retained captures until collection). Both call sites share the fix. New tests verified red on the previous implementation, green after: - context-copy failure leaves no registry residue - the real submit helper (only the loop getter patched) never invokes the coroutine factory when loop startup fails * fix(subagents): close the coroutine when scheduling rejects it run_coroutine_threadsafe can itself raise once the coroutine exists (e.g. the loop closes between the lookup and the internal call_soon_threadsafe). Wrap the call, close the rejected coroutine, and re-raise; a focused test patches only run_coroutine_threadsafe and asserts the created coroutine reaches CORO_CLOSED. |
||
|
Aniket Wagh
|
8a830f6354
|
fix(deps): depend on renamed tenki package instead of tenki-sandbox (#5087)
* fix(deps): depend on renamed tenki package instead of tenki-sandbox tenki-sandbox has been removed from PyPI and republished as tenki. Its old wheel URL still resolves, so existing lockfiles keep installing and the breakage is invisible to anyone with a warm lock; any fresh resolution fails with 'tenki-sandbox was not found in the package registry'. tenki 1.0.2 still ships the tenki_sandbox module, so the imports in community/tenki/provider.py and sandbox.py are unchanged. Fixes #5081 * fix(tenki): point install guidance at the renamed distribution The rename to `tenki` left the user-facing remediation still naming the removed package. `_import_client` raised "pip install tenki-sandbox" on the missing-extra path — the exact instruction this change proves now 404s on PyPI, handed to the user at the exact moment they need it to work. Update that message and the remaining `tenki-sandbox` references in the provider, sandbox adapter, README, sandbox AGENTS.md and the test docstring. The imported module stays `tenki_sandbox`, so the distribution and module names now differ; each mention says so rather than just swapping the string. No behavior change beyond the error text. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(tenki): migrate the provider to the 1.x workspace-only API Renaming the dependency was not enough. tenki 1.0.2 keeps the tenki_sandbox module name but not its contract: Client.create dropped project_id and has no **kwargs to absorb it, and IdentityWorkspace no longer carries `projects` (the attribute is gone from the package entirely). Both configuration paths therefore failed before a sandbox could be created — explicit project scope raised TypeError, and automatic scope raised AttributeError walking workspace.projects. Scope is now the workspace alone. _resolve_scope returns a single workspace id, auto-selecting when the account has exactly one, and project_id is gone from create_kwargs and from the documented config surface. A stale project_id in config.yaml warns rather than fails. SandboxConfig is extra="allow", so simply not reading the key would leave it scoping nothing with no signal; it also used to short-circuit the identity lookup, so operators with more than one workspace need to know they must now set workspace_id. The suite passed against the broken provider because the fake client took **kwargs and swallowed the project_id the real SDK rejects. The double now mirrors 1.0.2 — keyword-only, no **kwargs — so an unexpected argument is a TypeError in tests exactly as it is against the SDK. Reintroducing the old create call fails 20 tests; before this change it failed none. Verified against the exact locked wheels: every other kwarg the provider passes (name, workspace_id, sticky, wait, max_duration, image, cpu_cores, memory_mb, env) and every SDK surface it touches (who_am_i, Identity.workspaces, wait_ready, exec, close, the fs API, the four terminal exception classes) is unchanged in 1.0.2. Reported by willem-bd in review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(config): drop sandbox.project_id from the Tenki example The canonical example still documented project_id as a supported optional key after the provider stopped honouring it, so an operator following it could set the key, get no scope from it, and hit a workspace-resolution failure with nothing in the example to explain why. Replaced with a migration note rather than a silent deletion: someone upgrading already has the key in their config.yaml and needs to know it is inert now and that workspace_id is what scopes a sandbox on Tenki 1.x. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Aniket Wagh <aniketwaghh@users.noreply.github.com> |
||
|
Zeren Wang
|
bb75f8d736
|
feat(sandbox): share sandbox identity derivation and acquire serialization (#4741) (#5089)
* feat(sandbox): share sandbox identity derivation and acquire serialization (#4741) Remote providers (AIO, E2B, BoxLite, Tenki, OpenSandbox) each inlined the same sha256(user:thread)[:16] sandbox-id expression and kept per-scope lock dicts that grew unboundedly until shutdown. This extracts both mechanisms into shared components without changing provider lifecycle, ids, capacity semantics, or public tool behavior: - sandbox/identity.py: keyword-only derive_sandbox_scope_token (byte-pinned compatibility contract) + is_sandbox_scope_token; per-provider golden vectors pin current behavior including BoxLite's raw-None quirk and each provider's private user_id resolution. - sandbox/acquire_serialization.py: AcquireSerializer — per-key lock table with holder/waiter refcount reclamation, bounded dedicated executor (async waits off both the event loop and the default executor), worker-owned cancellation cleanup (no event-loop callback dependency), idempotent close(). - Each provider adopts both components; AIO/E2B key by (user_id, thread_id) with acquire and (E2B) release serialized; BoxLite/Tenki/OpenSandbox key by derived sandbox id and offload the whole sync acquire to the serializer's executor so a cancelled awaiter cannot overlap a retried same-scope body (leaked-remote-VM regression caught in review). - thread_id=None acquires stay unserialized; provider shutdown()/reset() close the serializer; E2B capacity/ledger/reconciliation and AIO ownership/flock machinery untouched. - blocking-IO anchor proves contended OpenSandbox acquire_async stays off the event loop (teeth verified red/green); AGENTS.md documents the shared components. * refactor(sandbox): address review on acquire serialization (#5089) - Replace unreachable checkin branch with an assertion: run() returns False only after abandon(), which the except handler always re-raises; the old _checkin would have double-decremented the refcount. - Document the task.cancelling() == 0 assumption in hold_async. - Drop unused thread_id/user_id kwargs from BoxLite and Tenki _acquire_scope_locked (OpenSandbox still forwards them). * fix(sandbox): preserve request ContextVars in acquire executor bridge (#5089) loop.run_in_executor() does not copy contextvars, unlike the inherited SandboxProvider.acquire_async() which used asyncio.to_thread(). The BoxLite/OpenSandbox/Tenki acquire_async bridges introduced in this PR therefore dropped the request trace id (logged as trace_id=-). Add AcquireSerializer.run_on_executor(), which copies the calling context and runs the callable through ctx.run, and route all three providers through it. Add regression tests binding request_trace_context and verifying the worker thread observes it. |
||
|
Sunshine
|
bf740ffa90
|
feat(auth): add personal access tokens for programmatic API access (#5041)
* feat(auth): add personal access tokens for programmatic API access (#4849) Backend-first implementation of the PAT contract from #4849: show-once dfp_ tokens bound to their owning user (AUTH_SOURCE_PAT, is_internal=false), digest-only storage (migration 0017), strict credential precedence (invalid Bearer is a 401, never cookie fallback), CSRF double-submit skipped only for Bearer requests while auth-endpoint origin checks still run, scopes intersecting the authz route permissions, session-auth-only PAT management and password changes, and throttled best-effort last_used_at stamps. * fix(auth): harden PAT scope boundary and schema parity from adversarial review Independent review of the initial draft found: (1) scopes only constrained the threads/runs permission axis while admin routes treated a PAT as its (possibly admin) owner — is_admin_user now rejects PAT callers outright since no scope grants admin capability; (2) the model declared a column UNIQUE constraint while migration 0017 created a named unique index, so downgrade failed on create_all-bootstrapped DBs — both now use the named unique index; (3) auth-disabled mode is an operator override and now stays ahead of the Bearer check so a stray Authorization header cannot 401 an E2E sandbox; plus wiring the previously-unused constants, bounding the last_used_at stamp cache, and four new tests (middleware-level expiry, expires_in_days, admin-capability rejection with session control, and the auth-disabled precedence). * docs(api): document personal access tokens for programmatic API access * fix(auth): close PAT security boundaries from review (default-deny routes, extension admin suppression) P1-1: scope intersection only constrains @require_permission routes, so undecorated mutation routes (DELETE /api/memory, POST /api/agents, Lark credential switching, channel config) accepted a PAT holding a single read scope. AuthMiddleware now enforces a default-deny route policy in auth/pat.py: PAT requests are admitted only to the thread/run lifecycle routes the v1 scopes govern; everything else answers 403 regardless of scopes. Session-cookie callers are unaffected. P1-2: the extension principal resolver projected is_admin/roles from the raw system_role, so an admin-owned PAT passed deerflow_extension_api.require_admin on contributed routes despite the documented no-admin guarantee. The projection is now PAT-aware and suppresses every admin signal for PAT callers, mirroring deps.is_admin_user. Both fixes carry regression tests (route outside policy 403 + session control; production resolver admin suppression), and API.md documents the default-deny boundary. * fix(auth): enforce PAT scopes on stateless run entry and harden decorator Follow-up hardening from an independent audit of the P1 fixes: - POST /api/runs/stream and /api/runs/wait were the only allowlisted run entrypoints without @require_permission, so a threads:read-only PAT could still start runs (same bug class as P1-1, now closed): both now carry @require_permission("runs", "create"). POST /api/threads and POST /api/threads/search gain threads:write / threads:read for the same reason. Authorization-disabled deployments see no change (the permission set resolves to all permissions). - require_permission now binds the wrapped signature to locate a positionally-passed request before injecting the test stub, fixing 'got multiple values for argument' on direct positional unit-test calls. - API.md: the intro PAT example used GET /api/models, which the new default-deny policy 403s — replaced with GET /api/threads; the default-deny route list now spells out method sets. Regression test: threads:read-only PAT is 403 on the decorated stateless entry while a runs:create PAT passes. * fix(auth): address review P2s (empty Authorization header, PAT name trimming, API example) - CSRFMiddleware treats an explicitly empty Authorization header as present (is None), so an invalid credential always reaches AuthMiddleware's uniform 401 instead of a CSRF 403 that varies by method/CSRF state. Regression: empty-header request dies at auth. - PATCreateRequest strips the name and rejects whitespace-only values before token generation; created names are stored trimmed. - API.md intro PAT example now uses the implemented POST /api/threads/search endpoint (GET /api/threads does not exist). - AGENTS.md trimmed back under the guidance soft budget after the upstream merge. * fix(auth): tighten PAT route policy to implemented methods only The allowlist admitted GET /api/threads, a method no router implements. Pre-authorizing a dead method weakens the default-deny boundary: a future GET collection route added without a permission decorator would become PAT-reachable without an explicit policy change. Restrict the rule to POST, fix the stale GET description in API.md's PAT constraints, and document the default-deny boundary accurately in the gateway AGENTS.md guidance (only the threads/runs allowlist is PAT-reachable; every other authenticated route 403s PAT callers). Audited every remaining rule against the mounted routers: all other method+path entries map to real routes. Regression: test_pat_policy_does_not_pre_authorize_unimplemented_methods. * test(auth): guarantee the negative digest test mutates the token token[:-1] + "X" is identical to the original whenever the generated token already ends in X (1/62), making the negative digest assertion fail intermittently. Choose the replacement character based on the existing tail so the mutated token always differs. * fix(auth): require runs:cancel for cancel-then-stream requests stream_existing_run is gated at runs:read so action-less stream joins work with read-only credentials, but its ?action=interrupt|rollback branch cancels the run — a separate permission. A runs:read-only PAT passed both the PAT route policy and the route decorator and could interrupt or roll back an active run, bypassing the runs:cancel scope. Decorators cannot express query-parameter-conditional permissions, so the check lives in require_cancel_permission_when_action(), applied at the top of the handler. Regression drives the real helper through the production middleware: runs:read-only PAT + action is 403, the same token joins action-less, runs:read+cancel passes, session control unaffected. * docs(changelog): add the PAT feature entry * docs(readme): add personal access tokens section Repo documentation-update policy requires user-facing features to update README.md in the same changeset; the PAT feature previously touched only backend/docs/API.md and the gateway AGENTS.md. * fix(auth): require runs:cancel for mutating multitask strategies All five run-creation entrypoints were gated only by runs:create, but RunCreateRequest.multitask_strategy accepts interrupt/rollback and start_run forwards it to create_or_reject, which terminates an already-active run. A runs:create-only PAT could therefore kill an existing run through a create request, bypassing runs:cancel. Decorators cannot express body-parameter-conditional permissions, and per-route checks leave the same hole for the next entrypoint, so the gate lives in start_run itself — the single choke point every run-creation path (HTTP routes and internal launchers) flows through. Regenerate launches pass multitask_strategy="reject" and are unaffected; requests without a stamped auth context (internal/test compositions) skip the gate. The check is the shared authz.require_cancel_permission_if primitive; require_cancel_permission_when_action now delegates to it, so every request dimension that carries cancel capability (query action, body strategy) flows through one gate. Regression drives the real middleware stack: runs:create-only PAT + interrupt/rollback is 403 with the exact detail, reject (explicit and default) stays available, runs:create+cancel passes, session control unaffected; a source anchor pins the gate inside start_run. * fix(runs): keep observer joins from applying creator cancel-on-disconnect sse_consumer's finally block applied the record's on_disconnect=cancel policy on ANY consumer's disconnect. The join surfaces (GET /join and the action-less GET/POST stream join) feed it the existing RunRecord, so anyone with thread read access — including a runs:read-only PAT — could cancel a locally-owned running run simply by closing the SSE connection, without runs:cancel. The policy expresses the creator's intent for their own connection; an observer's disconnect must never be read as that intent. sse_consumer gains apply_on_disconnect (default True). The two join surfaces pass False; the creating endpoints (thread-scoped and stateless create-and-stream) keep the creator semantics unchanged. wait_for_run_completion needs no change: its callers are creator-side or post-explicit-cancel paths only. Regression exercises a real generator close — the same machinery Starlette drives on client disconnect — against the production sse_consumer: creator stream disconnect cancels, observer join disconnect does not; a wiring anchor pins both join call sites and the creator defaults. API.md documents the cancel-capability constraint (this fix plus the action/strategy gates) in PAT Constraints. * test(auth): pin the multitask gate behaviorally; state wait invariant Independent adversarial review of the round-5 fixes found the P1-a regression only mirror-pinned: the source anchor could be satisfied by a comment, and deleting the gate from start_run would not fail the suite. This drives the production start_run directly — a create-only auth context gets 403 with the exact detail for interrupt, and a reject request with no cancel permission at all proceeds past the gate (never a permission 403). Also documents wait_for_run_completion's creator-side invariant (every caller is the creating endpoint or post-explicit-cancel) so a future observer wiring thinks twice before reusing it — the one-caller- away variant of the observer-disconnect P1. * docs(changelog): correct the PAT entry's digest and route-policy description The entry said HMAC digests (the implementation stores SHA-256 digests, as documented in API.md and pinned by the repository tests) and claimed the route policy admits 'implemented stateless endpoints' (it admits the thread/run lifecycle routes, narrowing further by scopes). Also notes the cancel-capability gate now covering action and multitask strategies. * fix(auth): enumerate the PAT runs route policy per implemented subroute The runs subtree rule was a GET|POST /runs(/.*)? wildcard — it pre-authorized every current and future subroute under /runs, including methods the router never implemented (e.g. GET /runs/stream), which is the same latent default-deny weakening the threads collection rule was tightened for: a future route added under /runs would become PAT-reachable without an explicit policy change. The wildcard is replaced with six segment-precise rules covering exactly the 14 implemented method+path combinations; the {run_id} slot necessarily matches any single segment, so the POST-only collection names (stream, wait, regenerate, edit-regenerate) are excluded from the GET run-id rule via negative lookahead — no dead method stays pre-authorized. Behavior for implemented routes is unchanged. test_pat_runs_policy_admits_exactly_the_mounted_routes derives the expected set from the mounted thread_runs router instead of a hand-maintained list: every implemented GET/POST route under /runs must be admitted, routes in this router outside the subtree stay denied, and representative unimplemented neighbors are denied — so adding a route under /runs now fails CI until it is explicitly allowlisted, and a removed route leaves a dead rule visible. API.md's PAT constraints list the enumerated routes and drops a feedback mention that belonged to the stateless /api/runs axis. * docs(migration): add the 0017 renumbering coordination note to 0017 The PR's migration-coordination comment states each migration file carries the note; the file did not. Adds it: numbering was generated against main head 0016 alongside #5078 and #4843; whoever merges first keeps the slot, the others renumber on rebase (revision/down_revision plus the bootstrap head assertions). * fix(auth): pad base62 tokens to a fixed 43-char width int.from_bytes discards leading zero bytes, so the unpadded encoder returned a variable-length body — empty for all-zero input, and shorter than 40 characters for any draw below 62**39 (~1 in 14.5M), leaving test_generate_pat_token_format probabilistically flaky and the token body without stable width (review round 6, P3). _base62 now left-pads with "0" to _base62_width(len(data)) — the exact integer digit count (62^43 > 2^256 > 62^42, so 43 for 32 bytes). The format test asserts the exact fixed width instead of a probabilistic floor, and a new unit test pins the all-zero, leading-zero-byte, and max-value edges deterministically. |
||
|
Nefelibata
|
c6f6a01f56
|
fix(runtime): prevent IndexError in MemoryStreamBridge._make_gap on empty events buffer (#5047)
* fix(runtime): prevent IndexError in MemoryStreamBridge._make_gap on empty events buffer * fix(runtime): handle empty stream replay gap bounds across backend and frontend - Clamp MemoryStreamBridge queue_maxsize at 1 and validate StreamBridgeConfig.queue_maxsize >= 1 - Update StreamGap docstring to clarify None retained bounds - Allow StreamReplayGapData and parseStreamReplayGap in frontend to accept string | null bounds, safely resuming when bounds are null - Add backend and frontend regression unit tests for queue clamping and null bounds replay gap * docs(stream-bridge): bump config_version and document empty buffer replay gap behavior * docs: document nullable gap bounds and sync helm config_version to 37 |
||
|
Zeren Wang
|
3b592c2053
|
feat(harness): subagent receipt citation verification (#5076)
* feat(harness): subagent receipt citation verification - add receipt citation verification core - harvest subagent tool receipts at terminal status - transport subagent receipts and citation verdict via status contract - verify subagent report citations at task write-back - render citation verdicts in the delegation ledger * fix(gateway): strip forged receipt verdicts from the delegations channel normalize_input() and the checkpoint-state mutation sanitizer only stripped server-owned metadata from message-shaped values, so an external caller could submit a delegation entry carrying a forged receipt_verdict that render_delegation_ledger would present as runtime-owned execution evidence. Strip receipt_verdict from caller-supplied delegation entries on both the run-creation and thread-state mutation paths, with regression coverage for each. * fix(harness): close silent-pass gaps in the zero-citation heuristic The action-claim detector missed the most common completion verbs (fixed/added/tested/changed/...) and had no CJK coverage at all, so reports like 'I fixed the bug and added tests.' or '我已经创建了文件并运行了测试。' were treated as claim-free: citation_resolved=True with no ledger warning. Broaden the verb lists and add a language-independent safety net: when the run harvested receipts and a nontrivial (>=240 char) report cites none of them, flag it UNVERIFIED. Short claim-free confirmations remain a vacuous pass. |
||
|
早上肚子疼
|
b41354d75f
|
fix(scripts): invoke repo shell scripts through an explicit interpreter (#5031)
Recipes and scripts ran sibling shell scripts bare (./scripts/x.sh), so
any checkout that lost the executable bit -- zip/tarball download,
core.fileMode=false, non-POSIX filesystem -- failed with:
make: ./scripts/docker.sh: Permission denied
make: *** [Makefile:181: docker-start] Error 127
The tracked modes are already 100755, so chmod cannot fix it. Name the
interpreter instead: the POSIX branch of RUN_SHELL_SCRIPT (renamed from
RUN_WITH_GIT_BASH) now expands to $(BASH) rather than nothing, and the
five script-to-script call sites are prefixed with bash.
Fixes #2903
Co-authored-by: zaoshangduziteng <309590849+zaoshangduziteng@users.noreply.github.com>
|
||
|
qingbo1011
|
2eba65449f
|
eval(memory): add a reproducible hybrid eviction evaluation (#4810)
* eval(memory): scaffold reproducible eviction evaluation * refactor(eval): align with benchmark layout * eval(memory): add deterministic QA grading Implement the disclosed deterministic-overlap-v1 grader as a pure offline module. Grading is blind by construction: grade_answer() accepts only the prediction and reference strings, never a policy identity. The undisclosed stopword list is committed as a fixed part of this grader version; yes/no/not are deliberately excluded because negation can be the entire answer. Before freezing, the grader locally reproduced all 90 historical (prediction, grade) pairs disclosed in #4789 with zero mismatches and no post-hoc tuning. validate-contracts now rejects a config whose qa.grader_version does not match the committed grader. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): add environment-configured QA runner Add the exact answer-prompt renderer (retained facts sorted by ID, CURRENT DATE line omitted when absent), an OpenAI-compatible provider adapter configured only through the environment variable names pinned in the config, and a resumable run-qa command that calls both policies with identical versioned settings. Each row persists as its own file on success, so a partial paid run resumes without repeating completed calls; qa_run.json binds an output directory to one config identity. Row files and errors carry predictions and non-secret metadata only -- never questions, references, memory text, credentials, or response headers. All tests are offline via mocked transports; run-qa fails fast before touching the dataset when the provider environment is missing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): add blind QA grading report and paired statistics Add grade-qa: it recomputes the deterministic selector output, rejects any answer row whose kept facts, capacity, or policy disagree with it, grades every prediction through the policy-blind grade_answer(prediction, reference) call, and only then joins grades back through stable row IDs. Published artifacts are qa.rows.jsonl (graded rows with non-secret metadata), qa.summary.json (accuracy by source/scenario/policy; official and synthetic suites never folded together), and qa.stats.json (exact paired McNemar and seeded paired bootstrap difference for the official, synthetic, and overall suites using the pinned statistics parameters). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): pin the official DeepSeek model ID The historical protocol recorded the answer model with an aggregator-style namespace (deepseek/deepseek-v4-flash). The live run calls the same underlying model (DeepSeek-V4-Flash-0731, released before the historical run) directly through DeepSeek's official OpenAI-compatible API, whose canonical ID is deepseek-v4-flash. The served model is recorded from the provider response in every answer row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): publish paired eviction QA results Publish the equal-budget live QA artifacts for pr4789-reproduction-v1: provenance, 90 graded rows, per-scenario summary, and paired statistics. At capacity 7 with identical settings, confidence answers 24/45 and hybrid-v1 40/45 (official 23/40 vs 35/40, exact McNemar p=0.0042; overall p=0.0004). The noisy-signal control is the one scenario where hybrid-v1 scored below the baseline (10/10 vs 8/10) and is reported separately. The offline suite now verifies the published statistics are recomputable from the published rows and that the artifacts carry no dataset text or credentials. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(eval): decompose the noisy-signal QA cell Both policies retained the support fact in all ten noisy-signal cases, so the two rows hybrid-v1 lost are grader phrasing boundaries (verbose numeric answers rejected by the numeric-conflict rule), not eviction failures. Documented from the published rows; the grader stays frozen. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): address review hardening findings - ignore the responses/ directory the runner actually writes instead of the stale provider-responses/ entry - cover the official selection-rule recomputation with direct synthetic tests: matching manifests pass, rule-breaking IDs and missing eligible rows fail, and every published exclusion is load-bearing - align the report docstring and README with the statistics contract: the summary never folds sources; the explicitly labeled overall suite is reported alongside the separate official and synthetic suites - recompute the published bootstrap intervals (not only McNemar) in the published-results test - wire required_policy_version to the production EVICTION_POLICY_HYBRID_V1 constant so validate-contracts rejects policy drift Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): restore historical evidence rendering and harden resume identity Address both blocking findings from the #4789 artifact cross-check. The evidence renderer now emits the historical SESSION {id} AT {date} line instead of the divergent bracket format. The byte representation is protocol-critical: the witness record 35a27287 renders at 697 characters again, stays inside the 700-character distractor-bank bound, and 60d45044 leaves the bank, restoring row-level pool reproduction. Deterministic capacity-7 retention is unchanged at 27/45 vs 45/45. qa_run.json now binds a run directory to the SHA-256 of all five protocol inputs (config, both manifests, answer prompt, dataset) and names the changed artifact when it refuses to resume. Stored rows are reused only when row identity, policy, capacity, kept facts, and the request fingerprint recomputed from the current task all match; the disclosed probe (changed message under the same config) is now a regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): republish QA results under the historical protocol Replace the published artifacts with the fresh equal-budget run executed at 497ff3d0 under the restored historical evidence rendering; the earlier run under the divergent rendering is discarded entirely. At capacity 7 with identical settings, confidence answers 24/45 and hybrid-v1 38/45 (official 23/40 vs 33/40, exact McNemar p=0.0129; overall p=0.0013). The confidence control is the one scenario below baseline for hybrid-v1 (8/10 vs 6/10); both lost rows retained the support fact and are grader phrasing/abstention boundaries, documented from the published rows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): adopt the historical fact IDs and prompt serialization Pool facts now carry the historical protocol IDs (gold_{case} for the support fact, d_{case}_{index}_{source} for distractors in bank-draw order), and the rendered STORED MEMORY joins fact blocks with a blank line. Sorting by these IDs reproduces the historical selection tie-break: witness case 41698283 at capacity 7 again keeps the 58bf7951 distractor and evicts 001be529 under both policies. Deterministic capacity-7 retention is unchanged at 27/45 vs 45/45. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): republish QA results under the historical serialization Replace the published artifacts with the fresh equal-budget run executed at 01f99d61 under the historical fact IDs and prompt serialization; earlier runs under divergent serializations are discarded entirely. At capacity 7 with identical settings, confidence answers 24/45 and hybrid-v1 40/45 (official 23/40 vs 35/40, exact McNemar p=0.0018; overall p=0.0001). The single row below baseline (1cea1afa, confidence-control) retained its support fact; the model abstained. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * eval(memory): self-certify the publish path and pin offset coverage grade-qa now verifies (read-only) that the run marker's five protocol artifact hashes match the current inputs, rebuilds every answer task, and rejects any stored row whose request fingerprint does not match the task recomputed from the current protocol — the staleness class that previously required an out-of-band cross-check to detect. Verified end-to-end against the published run: all 90 rows pass and regrade to byte-identical artifacts, while a tampered fingerprint is refused by row ID. The distractor offset derivation and wraparound selection are now pinned by unit tests with hardcoded indices, including a wrapping offset, so a digest-slice or modulus regression can no longer stay green offline. Closes both non-blocking suggestions from the re-review. * fix(bench): bind persisted answer rows to their expected case identity Grading derived the reference case from the stored row's embedded case_id, so reassigning a valid row to another valid case passed every integrity check while silently changing the published grade. The resume path had the same gap: _row_matches_task() never compared case_id, source, or scenario. The recomputed task is now authoritative in both paths: grade_answer_rows() resolves the reference case from the expected PolicyResult and rejects any mismatch in the persisted row_id/case_id/source/scenario, and _row_matches_task() checks the same identity fields so a reassigned row is re-run instead of reused. Regressions tamper each field individually and exercise both paths. * docs(bench): document where to download the pinned LongMemEval file The README named the dataset but never said it lives on Hugging Face or how to fetch the pinned revision, so a reviewer could not run the offline commands. Add the direct download URL, the expected SHA-256, and the mirror and huggingface-cli alternatives; the CLI still never downloads anything. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Nan Gao
|
73e3699347
|
feat(frontend): render markdown artifacts in the "open in new window" view (#5056)
* feat(frontend): render markdown artifacts in the new window The artifacts panel's "open in new window" action handed the browser the raw Gateway response. For markdown that is a `text/markdown` body the browser can only show as source, so the new window was a text dump rather than a reader. Route markdown artifacts to a new `/artifacts/view` page that renders them with the same components the panel uses (SafeStreamdown + the artifact rehype chain + citation links/panel), including the truncated-preview banner and its "load full file" action. Everything else keeps the raw Gateway URL — notably HTML/SVG, which the Gateway deliberately serves as a download so active content never executes in the application origin. - `core/artifacts/viewer.ts` centralizes which stored artifacts are markdown (`.skill` archives included, since they hold a SKILL.md), so the panel and the viewer route cannot drift. - `ArtifactFilePreview` and its siblings move out of `artifact-file-detail.tsx` into `artifact-file-preview.tsx`; otherwise the standalone route would pull the CodeMirror editor into its bundle. - The window title comes from the route's `generateMetadata`, not `document.title`, which the App Router overwrites after hydration. - The viewer reads content through `useStandaloneArtifactContent`, which shares `useArtifactContent`'s query key but not its `useThread` dependency, since a detached window has no thread context. Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC * fix(frontend): keep the artifact target across re-authentication Review found the standalone viewer unrecoverable from an expired session. The window's target lives entirely in `?path=...&thread_id=...`, and both auth paths dropped it: - The layout guard redirected to `/login` with no `next` at all. A layout cannot read `searchParams`, so the guard moves into the page, which can — and rebuilds the full viewer address for `next`. The layout loses its AuthProvider along the way: nothing under this route reads `useAuth`, and the guard now makes a single `getServerSideUser` call per request. - The shared fetch wrapper built `next` from `window.location.pathname`, which silently truncated the query string. It now carries `search` too, so any route holding state in the query survives a 401, not just this one. `validateAuthNextPath` already accepts a query string. `buildArtifactViewerURL` is split out of `resolveArtifactOpenURL`: the guard needs the route itself, never the Gateway fallback that the latter takes for non-markdown targets. Tests: the login round trip (unit — the rebuilt URL survives `validateAuthNextPath` and parses back to the same target), the fetch wrapper preserving the query on 401 (unit), and the expired-session window reaching `/login` with the artifact intact (E2E). The E2E asserts on the popup's navigation *requests*, since `(auth)/layout` answers `/login` with a server redirect under DEER_FLOW_AUTH_DISABLED and no navigation commits. `tests/unit/core/models/api.test.ts` stubbed `window.location` without `search`; a real Location always has it. Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC * fix(frontend): keep public showcase artifacts out of the auth gate Review found that the viewer's access check regressed `/showcase`. Those pages render with `isMock`, their artifacts are served by the unauthenticated demo route, and the raw artifact URL this window replaced stayed public — so gating the window unconditionally bounced every logged-out showcase visitor to /login for a document that is already public. `requiresAuthenticatedViewer` exempts a mock target only when `resolveStaticDemoArtifact` would actually serve it. The allowlist is the authority rather than the flag: `mock=true` is caller-supplied, so a target the demo route answers with 404 — a non-allowlisted path, or a thread that is not a demo thread — still needs a session. Covered in `tests/e2e-auth/`, since the default E2E config disables auth and cannot see this: a public showcase artifact renders without a session, while a non-allowlisted path and a missing mock flag both land on /login. Verified the positive case goes red without the exemption. Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC |
||
|
nonoge
|
0cb356858b
|
fix(frontend): truncate selected model names (#5050)
* fix(frontend): truncate selected model names * test(frontend): scope model selector overflow guard |
||
|
cui fliter
|
adfc307677
|
fix(channels): synchronize ChannelStore reads (#5083)
Signed-off-by: cuishuang <imcusg@gmail.com> |
||
|
Syt3s
|
6e5a41fd9a
|
feat(frontend): add conversation outline navigation for long chats (#5025)
* feat(frontend): add conversation outline navigation for long chats * fix(frontend): escape bottom lock before outline navigation |
||
|
Nan Gao
|
bf3e792a6a
|
feat(models): add GLM-5.3-Flash thinking workaround (#5074) | ||
|
zhang
|
23d8e4b3a3
|
feat(scripts): support skipping frontend build on make start (#5053)
* feat(scripts): support skipping frontend build on make start * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(scripts): validate skip-frontend-build before stop_all and format test --------- Co-authored-by: PoetryLin <PoetryLin@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Jholly
|
2d0568a14f
|
fix(frontend): format structured upload error details (#5071)
* fix(frontend): format structured upload error details * fix(frontend): preserve generic message error details |
||
|
ChaseMoon
|
9c1dd11160
|
fix(mcp): preserve pooled stdio sessions after task timeouts (#5027) | ||
|
Serply
|
4dbfe37ff3
|
feat(community): add Serply web search tool (#5023)
Add deerflow.community.serply.tools:web_search_tool, a Google SERP provider for the web_search slot that also covers Google News and Google Scholar through an optional `vertical` config option. Reads the key from api_key in config.yaml or SERPLY_API_KEY, clamps max_results to Serply's 1-100 range, and returns the same structured JSON errors as the Serper and Brave tools. Register the provider in config.example.yaml, scripts/doctor.py, scripts/wizard/providers.py, .env.example, backend/docs/CONFIGURATION.md, the en/zh tools.mdx provider tabs, and tools/AGENTS.md. Tests mock httpx. |
||
|
Terminator666666
|
e09b2d48df
|
fix(uploads): keep deduplicated filenames within the 255-byte limit (#5059)
normalize_filename accepts names up to 255 UTF-8 bytes, but claim_unique_filename appended _N to the stem without re-checking the budget. A duplicate at maximum length therefore produced a 257-byte name, and the write path (open_upload_file_no_symlink -> normalize_filename) rejected it with ValueError. In the Gateway upload route that error falls into the generic handler: the whole request fails with a 500 and files already written in the same batch are rolled back — including unrelated ones. The same helper backs the Feishu and DingTalk channel downloads and client-side attachment staging. Truncate the stem on a UTF-8 code-point boundary when appending the dedupe tag would exceed 255 bytes, so the result always round-trips through normalize_filename. Names short enough to fit keep the exact dedupe shape they had before. Tests: red on main, green here — - unit: max-length dedupe stays within the limit and round-trips; repeated collisions stay unique; multibyte stems truncate on a code-point boundary; short names keep the historical _N shape - router: a batch with a max-length duplicate now succeeds and keeps every file instead of failing with a 500 Co-authored-by: Terminator666666 <Terminator666666@users.noreply.github.com> |
||
|
georgelichen
|
24001e80b7
|
fix(skills): safely tokenize portable allowed-tools patterns (#4984)
* fix(skills): accept portable frontmatter forms * fix(skills): normalize portable tool names * Safely preserve parenthesized portable skill tool patterns Portable Agent Skills declarations such as Bash(tvly *) contain spaces inside a command pattern. Keep those patterns as single literal entries while preserving exact names from the existing YAML-list form, so skill loading no longer fragments valid metadata or rewrites mixed-case MCP tools. Constraint: DeerFlow's current skill policy matches exact tool names and does not inspect Bash arguments Constraint: Agent Skills scalar syntax uses whitespace-separated entries with parenthesized command patterns Rejected: raw.split() | fragments Bash(tvly *) into unrelated tool names Rejected: normalize YAML-list entries | breaks case-sensitive MCP/runtime tool names Rejected: map Bash(...) to bash | broadens command-scoped declarations into unrestricted shell access Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep Bash(...) entries literal and inactive until DeerFlow has an explicit command-pattern authorization model Tested: 175 focused parser, validation, installer, review, loader, and tool-policy tests; Ruff check and format; compileall; git diff --check Not-tested: Full backend suite stopped at pre-existing Windows mode assertion test_runtime_config_store_file_is_owner_only Related: #4912 * Preserve exact custom tool names in portable skill parsing Portable scalar frontmatter needs alias normalization for known DeerFlow-compatible names, but generic case conversion corrupts MCP and custom tool identifiers. The tokenizer also treated quoted or escaped parentheses as structural delimiters, rejecting valid command patterns. Preserve unknown names and parse quoted or escaped patterns without broadening Bash(...) into bash. Constraint: Runtime skill policy uses exact tool-name matching Constraint: Parenthesized patterns remain literal because argument-level authorization is not implemented Rejected: Generic CamelCase-to-snake_case for every scalar | rewrites custom/MCP names Rejected: Map Bash(...) to bash | broadens command-scoped declarations into unrestricted shell access Confidence: high Scope-risk: narrow Reversibility: clean Directive: Add an explicit alias before supporting another portable tool name; keep command-pattern authorization separate Tested: 225 skills tests passed, 1 skipped; Ruff check; Ruff format --check; compileall; git diff --check Not-tested: Full backend suite remains affected by unrelated Windows permissions/path and missing Lark CLI tests Related: #4984; #4912 * Preserve case-sensitive exact tool authorities Case-folding a scalar declaration before alias lookup can turn literal write into write_file, substituting a different runtime authority. Keep exact portable spellings as aliases and preserve lowercase, custom, and MCP names; strengthen activation coverage for spaced Bash patterns and command fragments. Constraint: Runtime skill policy uses exact tool-name matching Constraint: Bash(...) remains literal and inactive because command-pattern authorization is not implemented Rejected: Case-insensitive alias lookup | maps lowercase runtime tools onto built-in authorities Rejected: Broaden the parser into command-pattern authorization | outside this PR's scope Confidence: high Scope-risk: narrow Reversibility: clean Directive: Add aliases only for documented portable spellings; preserve all other scalar names verbatim Tested: 226 skills tests passed, 1 skipped; Ruff check; Ruff format --check; compileall; git diff --check Not-tested: Full backend suite remains affected by unrelated Windows permissions/path and missing Lark CLI tests; GitNexus index refresh remains stale Related: #4984; #5016297602 * Support portable Glob and Grep skill aliases Portable Agent Skills commonly declare Glob and Grep, but DeerFlow exposes the runtime tools as glob and grep. Add explicit exact-spelling aliases and activation coverage so imported skills retain search-tool access without broad normalization. Constraint: Runtime skill policy uses exact tool-name matching Constraint: Alias conversion is limited to documented portable spellings Rejected: Case-fold all scalar names | can substitute custom or MCP authorities Rejected: Map arbitrary names by convention | breaks exact runtime compatibility Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep the alias table explicit and preserve unknown scalar names verbatim Tested: 228 skills tests passed, 1 skipped; Ruff check; Ruff format --check; compileall; git diff --check Not-tested: Full backend suite has unrelated environment failures on Windows; GitNexus index reports stale line mappings Related: #4984; #5026257899 --------- Co-authored-by: kriptoburak <kriptoburak@users.noreply.github.com> |
||
|
早上肚子疼
|
0d97fdc770
|
docs(zh): sync missing security subsections into README_zh (#5029)
The English Security Notice has four subsections; the Chinese one had two.
Translate the two that were missing:
- Gateway Admin Is Equivalent to Code Execution
- Deployment Defaults
Both describe deployment-time security behavior, so a stale translation
leaves Chinese-speaking operators without the loopback-default and
first-run-setup guidance that English readers get.
Verified against the code rather than translated blind:
- stdio MCP allowlist defaults to {npx, uvx} and is extended via
DEER_FLOW_MCP_STDIO_COMMAND_ALLOWLIST
(backend/app/gateway/routers/mcp.py)
- entry port publishes as ${BIND_HOST:-127.0.0.1}:${PORT:-2026}
(docker/docker-compose.yaml)
- /setup exists as the first-run admin creation route
(frontend/src/app/(auth)/setup/page.tsx)
Co-authored-by: zaoshangduziteng <309590849+zaoshangduziteng@users.noreply.github.com>
|
||
|
陈志谦
|
9e2c1be697
|
fix(sandbox): harden local Docker sandbox containers and port binding (#4986)
* fix(sandbox): harden local Docker sandbox containers and port binding Root causes (security audit SBX-1/SBX-2) in the local container backend: - _resolve_docker_bind_host published sandbox ports on 0.0.0.0 whenever DEER_FLOW_SANDBOX_HOST was non-loopback (docker-compose defaults to host.docker.internal), exposing the unauthenticated /v1/shell/* exec API on every host interface. - _start_container ran every sandbox with seccomp=unconfined and no capability, privilege-escalation, or resource limits, so untrusted model-authored code could exhaust the host, escalate privileges, and reach internal networks / cloud metadata endpoints directly. Hardening changes and defaults: - Port binding: non-loopback sandbox hosts now bind the Docker default bridge gateway instead of 0.0.0.0, discovered dynamically via `docker network inspect bridge` with a static 172.17.0.1 fallback. host.docker.internal resolves to that gateway through host-gateway, so DooD gateways and the Docker host still reach the sandbox while external interfaces no longer see the port. DEER_FLOW_SANDBOX_BIND_HOST=0.0.0.0 restores the legacy broad bind. - seccomp=unconfined is no longer unconditional: sandboxes run with Docker's default seccomp profile; opt back in with DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED=1, only when the sandbox image is verified to require syscalls the default profile blocks. - Add --cap-drop=ALL and --security-opt no-new-privileges (Docker only; the Apple Container CLI does not support these flags). - Bounded resources with env overrides: --memory 2g (DEER_FLOW_SANDBOX_MEMORY), --cpus 2 (DEER_FLOW_SANDBOX_CPUS), --pids-limit 512 (DEER_FLOW_SANDBOX_PIDS_LIMIT); each also accepts "0"/"none" to disable the limit. - No --user is forced by default (the default AIO sandbox image's user is upstream-controlled and unverified), but DEER_FLOW_SANDBOX_CONTAINER_USER passes one through for deployments that know their image. - DEER_FLOW_SANDBOX_NETWORK passes --network so sandboxes can be attached to a dedicated egress-controlled network; default networking is unchanged. backend/docs/CONFIGURATION.md documents the new bind behavior and every override; tests cover each default and escape hatch. * fix(sandbox): follow host-gateway mapping for binds; keep image-required seccomp default Review follow-ups on the hardening change: - Bind: resolve the sandbox host itself and bind that address, instead of assuming the default bridge IPv4. host.docker.internal follows the daemon host-gateway-ip mapping (customizable, possibly IPv6), so the resolved address is exactly where the gateway connects — the published port and advertised URL always match. IPv6 is bracketed for docker -p, zone ids stripped, wildcard resolutions ignored; unresolved hosts fall back to the bridge gateway with a warning pointing at DEER_FLOW_SANDBOX_BIND_HOST. - seccomp: the shipped AIO image needs seccomp=unconfined for its Chromium browser (upstream quick-start always passes it; the upstream FAQ documents the browser failing under Docker default profile), so that option returns as the default. Tightening stays possible via DEER_FLOW_SANDBOX_SECCOMP_PROFILE=<path to a restricted, Chromium-compatible profile> or DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED=0 for images verified to work with Docker's default profile. - cap-drop/no-new-privileges and the resource limits are unchanged. - Tests updated for both behaviors; 37 pass. * fix(sandbox): bracket bare IPv6 bind overrides; state seccomp default accurately DEER_FLOW_SANDBOX_BIND_HOST was returned verbatim, so a bare IPv6 literal like fd00::1 produced an invalid publish spec (fd00::1:port:8080); Docker requires the bracketed form. Normalize raw and already-bracketed IPv6 literals (IPv4/hostnames untouched), with resolver-level and argv-level tests covering the explicit IPv6 override. The CONFIGURATION.md overview claimed Docker's default seccomp profile stays active, contradicting the seccomp=unconfined default the table (and the code) actually ship for the Chromium-based image; spell out the relaxed default and where to change it. * style(sandbox): apply ruff format to local_backend * fix(sandbox): reject host networking, force builtin seccomp opt-out, resolve hostname binds Review follow-up on #4986 (willem-bd): - P1: DEER_FLOW_SANDBOX_NETWORK=host (and container:<name>) now raise a RuntimeError at start instead of silently voiding the hardened port bind — Docker discards -p/--publish in host mode and shares the network namespace for container:<name>, which would re-expose the unauthenticated exec API on the host's interfaces. Two regression tests cover both rejections. - P2: the seccomp opt-out now passes seccomp=builtin explicitly instead of omitting the option, so a daemon configured with an unconfined or custom default cannot weaken the documented opt-out; the test asserts the flag. - P2: hostname values in DEER_FLOW_SANDBOX_BIND_HOST resolve to an address before use (Docker publish specs require an IP literal as the host part, so host.docker.internal previously produced an invalid spec that prevented every sandbox from starting); unresolvable names raise a clear configuration error. Tests cover resolution and rejection; CONFIGURATION.md updated for all three behaviors. 43/43 pass in tests/test_aio_sandbox_local_backend.py; ruff check + format clean. * fix(sandbox): reject DEER_FLOW_SANDBOX_NETWORK=none (loopback-only, breaks published API port) * fix(sandbox): validate the effective Docker network target; normalize IPv6 sandbox hosts once name=host / name=none dodge raw-string checks but attach like the bare words; strip name= prefixes and validate the effective target (network IDs keep passing). Bracketed IPv6 sandbox hosts now resolve for the bind and bare IPv6 hosts produce bracketed URL authorities — both input forms give identical bind and URL addresses. * fix(sandbox): parse the full Docker network long syntax before validating Docker accepts comma-separated key=value fields in any order (name=, gw-priority=, alias=, ...); a name=host field hides the host network behind surrounding fields. Parse the CSV and validate the parsed name= target (last occurrence wins, fields lowercased, mirroring opts/network.go); no-name values fall through like Docker's own rejection. * fix(sandbox): keep CHOWN/SETUID/SETGID through cap-drop=ALL for the default image The shipped image's entrypoint starts as root, creates the gem user, chowns /opt/jupyter and drops to that user via su; without those three capabilities the set -e script dies before the readiness endpoint exists. no-new-privileges stays (it blocks gaining privileges via exec, not using the added caps). Adds a docker-gated real-image startup smoke test. * fix(sandbox): let pre-initialized non-root images drop the startup capabilities The CHOWN/SETUID/SETGID re-add only exists for the shipped image's root entrypoint handoff. A custom image that never runs as root gets an explicit opt-out (DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS=0) so those capabilities are not left available to sandboxed code (chown on bind mounts, UID/GID impersonation). * test(sandbox): gate the real-image smoke test behind the live marker The default offline suite (make test = -m 'not live') must not depend on a third-party registry: mark the smoke test live, probe the daemon inside the test body (never at collection time), and allow pinning the image reference via DEER_FLOW_SANDBOX_SMOKE_IMAGE for a dedicated integration job. * test/docs: isolate DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS in tests; add table row; split custom-image guidance _clear_hardening_env now clears the new knob so a developer shell or .env preset cannot flip the default-path tests. CONFIGURATION.md gains the table row, and the custom-image guidance becomes its own paragraph with the no-new-privileges scope stated correctly (it does not mitigate the retained CAP_SETUID/SETGID risk). * test(sandbox): make the live smoke test diagnosable 300s readiness budget (cold pull + cold start must not be conflated with broken capabilities) and dump the container's last 40 log lines on failure so the next live run tells us whether the capability set is incomplete (chown/useradd/su errors) or the services are merely slow. * test(ci): align the smoke test with the 60s provider deadline; add a dedicated live smoke workflow Single-source the readiness deadline as SANDBOX_LOCAL_PROVIDER_READY_TIMEOUT (used by both provider paths and the smoke test) so the validation cannot drift from the production contract again. New sandbox-image-smoke.yml runs the live test on a dedicated job, with the image reference pinnable via the SANDBOX_SMOKE_IMAGE repository variable (digest resolved and recorded in the job summary when falling back to :latest). * test(sandbox): pull the failing program's own logs on smoke failure supervisord only surfaces exit codes in docker logs; nginx's stderr lands in files inside the container. Dump supervisor program logs, nginx -t, and the nginx error log on failure so the next run names the exact broken line. * ci(sandbox): export an immutable repo@digest reference for the smoke run docker pull once on the runner platform, resolve RepoDigests[0], and pass that immutable reference to the test via GITHUB_ENV — the recorded and executed images can no longer diverge when the tag moves, and platform selection is left to the daemon instead of jq over the manifest index. * fix(sandbox): add DAC_OVERRIDE — the root nginx master writes gem-owned logs The image's root nginx master opens /var/log/nginx/{access,error}.log, which belong to the gem user, for the container's lifetime; without CAP_DAC_OVERRIDE it dies with 'open() failed (13: Permission denied)' on every start (FATAL under supervisord) and readiness never arrives. Four capabilities now: CHOWN/SETUID/SETGID for the entrypoint handoff plus this runtime log-write need. |
||
|
Ryker_Feng
|
848eea289c
|
feat(frontend): duplicate scheduled tasks (#5064)
* docs: design scheduled task duplication * docs: plan scheduled task duplication * feat(frontend): build scheduled task duplicate drafts * feat(frontend): duplicate scheduled tasks into create form * docs: document scheduled task duplication * test(frontend): verify duplicate schedule preview * refactor(frontend): simplify scheduled task duplication |
||
|
yong
|
846c716523
|
feat(search): add Tencent Cloud WSA provider (#5057)
* feat(search): add Tencent Cloud WSA provider * docs: restore README to upstream * docs: remove README changes from WSA provider PR * fix(doctor): validate Tencent WSA API key |
||
|
Sunshine
|
9600414002
|
fix(channels): retain strong reference to Discord ack-reaction tasks (#5049)
* fix(channels): retain strong reference to Discord ack-reaction tasks The event loop keeps only weak references to scheduled tasks, so the bare asyncio.create_task(self._add_reaction(message)) at both message-acceptance sites could be garbage-collected mid-flight and silently drop the ✅ acknowledgment. The module-level retention set holds each task until completion and logs failures, mirroring the deferred subagent cleanup fix in #4928 and the stream bridge cleanup fix in #4931. * fix(channels): drain in-flight ack reactions on Discord channel shutdown Adversarial follow-up to the retention fix: stop() cancelled typing tasks but left in-flight ack-reaction tasks in the module retention set when the channel stopped mid-HTTP-call, pinning the channel instance and the discord Message object graph across restart cycles. _cancel_ephemeral_tasks / _discard_ephemeral_tasks now cover both families at every stop() cleanup point and the _run_client finally. Also tests the done-callback error-log branch and documents the GC test as a contract pin. * test(channels): pin Discord ack-reaction drain wiring and sharpen wording Second adversarial pass found the drain test cancelled a never-started task (no yield between schedule and cancel), nothing exercised the real stop() cross-loop wiring, and the cleanup log messages still said "typing-task" while now covering both families. The drain test now yields so the task genuinely suspends first, a new test drives stop() through a real background loop and fails if the wiring reverts to typing-only cleanup, wording covers ephemeral tasks, and the discard path snapshots the set before iterating. * refactor(channels): scope Discord ack-reaction retention to the channel instance The retention set was module-level, so one DiscordChannel's stop() would cancel every other instance's in-flight ack reactions. Instance-level matches the typing-task bookkeeping and keeps shutdown scoped; pinned by a two-channel isolation test. * test(channels): drive two-channel ack isolation through full stop() Independent review of the instance-scoping refactor confirmed the P2 fix but suggested matching the review's letter exactly: the isolation test now shuts the first channel down via stop() instead of the drain helper, and one stale docstring reference to the former module-level set is corrected. |
||
|
Willem Jiang
|
b1b3a31ebe
|
chore(ci):upgrade the node version from 22 to 24 (#5063)
* chore(ci):upgrade the node version from 22 to 24 |
||
|
georgelichen
|
bdd68469c1
|
fix(frontend): stabilize Windows development startup (#5036)
* Stabilize Windows frontend development startup Next.js Turbopack can infer a user-level workspace root and hit an internal Rust task panic on Windows. Keep the frontend root explicit and select Webpack on Windows while preserving Turbopack elsewhere. Constraint: The existing non-Windows development workflow should keep Turbopack. Rejected: Disable Turbopack on every platform | unnecessary performance regression. Confidence: high Scope-risk: narrow Reversibility: clean Directive: Revisit the Windows Webpack fallback after the upstream Turbopack panic is resolved. Tested: Prettier, ESLint, TypeScript, 139 frontend test files with 1060 tests, and Windows pnpm dev smoke test on port 3010. Not-tested: Production build and browser E2E suite. Related: #4957 * Make the frontend bundler fallback diagnosable Keep the Windows Webpack fallback while allowing local diagnosis to select either supported Next.js bundler. Resolve the Turbopack root through the Node 20-compatible URL API so the explicit workspace boundary remains effective across the declared Next.js engine range. Constraint: Windows defaults to Webpack because of the observed Turbopack panic, while non-Windows defaults to Turbopack. Rejected: Accept arbitrary bundler values | Next.js only supports the two explicit flags and invalid values should fail early. Confidence: high Scope-risk: narrow Reversibility: clean Directive: Revisit the default Windows fallback when the upstream TaskGuard issue is resolved; the override is intentionally retained for diagnosis. Tested: Frontend format, ESLint, TypeScript, 139-file/1062-test Rstest suite, target launcher tests, and next.config.js absolute-root load check. Not-tested: Production build and browser E2E suite. |
||
|
rayhpeng
|
ed336ec3dd
|
fix(gateway): enforce run-create authorization on stateless endpoints (#5030)
* fix(gateway): enforce authz on stateless runs * fix(gateway): guard scheduled run creation |
||
|
青榆牧
|
a94b2d8897
|
feat(mcp): map request-scoped secrets to MCP HTTP/SSE headers (#5010)
* feat(mcp): map request-scoped secrets to HTTP/SSE headers `user_auth` binds a credential to a configured DeerFlow user, so a caller that picks the credential per request — a multi-tenant gateway, a per-run API key, one shared MCP server fronting several environments — had to register one MCP server entry per credential. Add a declarative `mcpServers.<server>.headers_from_context` block mapping HTTP header names to keys of the run request's `config.context.secrets` carrier. A new built-in interceptor resolves the mapping on every tool call and rewrites those headers, mirroring `user_scoped_auth`. The config file stores names only, never a credential, so the Gateway returns the block unmasked. Registered after OAuth and `user_auth` in the interceptor chain: the later interceptor runs closer to the transport, and the value chosen for this one request is the most specific, so it wins. Fail-closed by default — a mapped key missing from the request raises a `ToolException` naming only that key, because falling back to the server's discovery credential would send one tenant's call under another tenant's authority. `on_missing: "passthrough"` opts out. Durable background tasks are excluded: `McpTaskToolCaller` drives status and cancel polls after the Agent run ends, where no run context exists, so the fail-closed interceptor would deny every poll. Those calls keep using server-level credentials, and a server declaring both `headers_from_context` and `task_toolsets` now logs a warning. Also corrects the custom-interceptor example in docs/MCP_SERVER.md (and the matching claim in skills/AGENTS.md), which read request secrets from `langgraph.config.get_config()["context"]`. That key is `None` inside a tool call — the run context rides the LangGraph runtime, not the RunnableConfig propagated to child runnables — so interceptors written from that example never saw a value. The example now reads `request.runtime`, and tests/test_mcp_context_headers.py pins LangGraph's runtime-injection rule by driving a real langchain-mcp-adapters tool through a real graph with the ambient-runtime fallback disabled. Closes #5005 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): resolve credential headers case-insensitively, carry them on durable submit Review follow-ups on `headers_from_context`. HTTP field names are case-insensitive, but every dict on the path to the wire is not: `build_server_params` copies the operator's static `headers` spelling verbatim, and langchain-mcp-adapters merges interceptor overrides into the connection with a plain `{**connection_headers, **override_headers}` splat. A static `authorization` and an injected `Authorization` therefore both reached httpx as separate field lines, and a server reading the field with a single-value accessor got the static discovery credential — inverting the documented `headers` < `oauth` < `user_auth` < `headers_from_context` precedence and running a per-request call under the shared credential. Normalizing inside the interceptor cannot fix that on its own: the adapter builds the request with `headers=None`, so an interceptor never sees the connection's static headers and cannot displace them however it spells its own key. A new `mcp/headers.py::apply_header_overrides` therefore drops any key differing only in case and emits the spelling the connection already uses. Applied to `headers_from_context`, `user_auth`, the OAuth interceptor, the OAuth discovery-header write, and the durable-task connection merge, which all carried the same collision. `headers_from_context.headers` now also rejects one header mapped under two spellings at config load, in both the harness model and the Gateway mirror. Durable submit now carries the mapped headers, as docs/MCP_SERVER.md already promised. `McpTaskToolCaller` disabled the interceptor for the whole caller, but that caller serves submit as well as the polls, and submit is awaited inline inside the Agent's tool call — where the run's LangGraph runtime is still the ambient contextvar, so no secret has to be threaded through `TaskSubmitRequest` or reach durable storage. The caller builds one chain and keeps a second view of it without the context-headers interceptor; `call_tool` takes `request_scoped_headers`, set only by `OrdinaryMcpTaskDriver.submit`. Status and cancel keep server-level credentials, so background polls still cannot fail closed, and the startup warning now describes the half it actually covers. `_merge_preserving_secrets` restores masked extras inside `headers_from_context` instead of writing the `***` sentinel back over the stored value, matching the treatment `user_auth` extras and server-level extras already get; extras a PUT omits carry over as well, while the declared mapping still replaces verbatim so a round trip can remove an entry. `extra="allow"` plus name-based sensitivity detection means the usual casualty is a name-valued key such as `tokenHeader`, not only a credential. The existing override test seeded the static header onto `request.headers`, which production never does, so it modelled a merge that really happens one layer down; the new tests drive a real adapter tool through a real connection and assert on the headers the session is opened with, and the durable-submit test runs through a real tool node with no runtime patching. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): reject case-insensitive duplicate static header names * fix(mcp): preserve omitted headers_from_context fields on partial updates --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Hyeonsang Cho
|
cb24bc2699
|
perf(middleware): stop checkpointing view_image base64 payloads (#5014)
ViewImageMiddleware injected the viewed-image message from before_model and removed it again from after_model. before_model, model, and after_model are separate graph nodes, so every view_image turn cost two extra nodes and two state writes, and up to 20MB of base64 sat in two checkpoints for the duration of the model call. A run interrupted in that window (user cancel, restart) stranded the payload in history for good. Inject from wrap_model_call instead, so the message lives only in ModelRequest.messages and is never returned as a state update: - before_model/after_model (and the async pair) are replaced by wrap_model_call/awrap_model_call; _remove_image_context_messages and its RemoveMessage bookkeeping go with them. The async hook keeps the existing asyncio.to_thread offload for the file read and base64 encode. - _should_inject_image_message gates on request.messages rather than state, so the decision is made against what the model will actually see. - _inject sweeps this middleware's own message out of the request before rebuilding it. Dropping after_model also drops the cleanup it did on every call, so without the sweep a payload stranded by an older interrupted run would ride along in every later request for the life of the thread. Matching requires both the reserved id prefix and the server-owned marker, and Gateway strips that marker from client input, so a user message is never dropped. Chain position is unchanged, and wrap_model_call nests first-registered outermost, so TokenBudgetMiddleware still sees the image message and enforces the input budget against it. Checkpoint rows that already hold a stranded payload keep it on disk. It is inert -- never sent to a provider, and strip_data_url_image_blocks keeps it off the wire -- and reclaiming it would mean keeping the node this change removes. tests/test_view_image_middleware.py is rewritten around the new hook (43 tests): sync/async at unit and graph level, the stranded sweep, and the client-message protection. Docs: middleware chain entry 23, Vision Support, the middleware-execution-flow hook matrix and diagrams, and the strip_data_url_image_blocks docstring. |
||
|
green3sf
|
788a890bd0
|
fix(tui): preserve transcript scroll position (#4975)
* fix(tui): preserve transcript scroll position * test(tui): cover output during queued paging |
||
|
ChaseMoon
|
287f890c66
|
fix(mcp): reconnect ordinary stdio tools after disconnect (#5018) | ||
|
wutongyuonce
|
013dca6352
|
fix(skills): refresh user storage after config reload (#4972)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
zhangwei139623
|
431892e16a
|
feat(knowledge): add read-only RAGFlow retrieval (#4955)
* feat(knowledge): add read-only RAGFlow retrieval * test(knowledge): cover RAGFlow retrieval contracts * docs(knowledge): document retrieval-only RAGFlow setup * refactor(knowledge): move RAGFlow settings to tool config * fix(ragflow): bind retrieval to configured datasets * docs(ragflow): record validated response versions * fix(ragflow): bind retrieval by dataset id * fix(ragflow): search all datasets by default * fix(ragflow): retrieve mixed embeddings by group * docs(ragflow): keep feature details out of agent guides * docs(ragflow): remove agent guide changes * docs(ragflow): remove root readme changes * fix(ragflow): handle unresolved and empty datasets * fix(ragflow): harden dataset scope and errors |
||
|
georgelichen
|
1c219b6864
|
fix(sandbox): scope MSYS path conversion exclusions (#5003)
* Preserve Windows CLI compatibility for local sandbox commands MSYS path conversion must remain disabled for DeerFlow virtual paths, but applying a blanket environment override to every POSIX command breaks host-native CLI shims on Windows. Limit MSYS argument-conversion exclusions to safe non-root virtual path prefixes, omit values that would broaden the exclusion pattern, and document the contract. Constraint: Preserve the virtual-path protection introduced by #2765/#2766 Rejected: Disable MSYS conversion for every command | breaks Windows CLI shims Rejected: Toggle blanket conversion only for commands containing virtual paths | host CLIs can receive virtual-path arguments and still need normal conversion for their own paths Confidence: high Scope-risk: moderate Reversibility: clean Directive: Keep regression coverage for virtual-path arguments, root mounts, and host-native CLI launchers Tested: test_local_sandbox_encoding.py (12 passed); related sandbox suite (197 passed, 8 skipped, 7 failures matching origin/main); ruff check; ruff format --check; git diff --check; direct LocalSandbox CLI and virtual-path smoke tests Not-tested: Full offline suite completion; stopped at 6% after unrelated Windows and optional-runtime failures Related: #2765 Related: #2766 * Keep MSYS regression tests portable across CI operating systems The Windows-shell environment tests patched os.name to nt while mounting Windows-specific paths. On Linux and macOS, pathlib then attempted to construct WindowsPath during command resolution or output masking, so the backend merge gate failed before exercising the environment contract. Stub the exclusion boundary in execute-command tests and retain mapping-specific filtering coverage in the helper test. Constraint: Backend unit tests run on Linux, while the behavior under test is Windows-only Rejected: Skip the tests outside Windows | would remove CI coverage of the environment contract Rejected: Patch pathlib internals | couples tests to implementation details and hides the platform boundary Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep OS-specific subprocess assertions independent from host-path resolution Tested: test_local_sandbox_encoding.py (12 passed); ruff check; ruff format --check; git diff --check Not-tested: Linux runner execution locally because Docker Desktop is unavailable and WSL cannot access this linked worktree Related: #5003 Related: https://github.com/bytedance/deer-flow/pullrequestreview-5013380238 |
||
|
Willem Jiang
|
2a261d2276
|
chore(doc): update the CHANGLOG with the latest change in main branch (#5004)
* Update the CHANGELOG with latest changes * update the Chinese version of CHANGELOG |
||
|
Xinmin Zeng
|
943d148e5e
|
feat(threads): distinguish branched conversations (#4983)
* feat(threads): number branched conversation titles * feat(frontend): show branch lineage in recent chats * fix(threads): allocate unique branch suffixes * fix(threads): preserve suffix and filter semantics |
||
|
Beautyl0ve
|
e8410cebfc
|
fix(gateway): preserve exact history attribution beyond event page limits (#4953)
* fix(gateway): preserve exact history run attribution * fix(gateway): make history migration authoritative * docs(runtime): keep history contract within guidance budget * fix(runtime): fence final run duration write |
||
|
Aari
|
ff0a6768c2
|
feat(subagents): add unified capacity and durable batch execution (#4998)
* feat(subagents): add capacity controls and durable batches * fix(helm): sync subagent config schema version * fix(subagents): preserve batch history without worker * fix(subagents): support explicit factory runtimes * fix: address durable batch review findings |
||
|
陈志谦
|
8989173c8d
|
fix(frontend): restore sanitization in custom streamdown rehype chains (#4987)
* fix(frontend): restore sanitization in custom streamdown rehype chains Streamdown 2.5 replaces its entire default rehype chain [rehype-raw, rehype-sanitize, rehype-harden] with whatever array the caller passes via the rehypePlugins prop. Every custom chain in this repo therefore rendered LLM/stored markdown without any sanitization: - Artifact markdown previews (markdown-preview-plugins.ts + artifact-file-detail.tsx) parse raw HTML via rehypeRaw, so a generated .md artifact could inject <style>/<iframe>/on* handlers into the workspace DOM (stored XSS; only javascript: anchors were blocked by the ArtifactLink component). - The memory settings summary (memory-settings-page.tsx) spread the shared preset without component overrides, so a hostile <a href="javascript:..."> in stored memory content rendered as a clickable anchor. Fix strategy: - Add rehype-sanitize (already resolved in the lockfile via streamdown) as a direct dependency and re-insert a [rehypeSanitize, schema] step in the shared preset (core/streamdown/plugins.ts). It runs after rehypeRaw (raw HTML must be parsed into hast before it can be cleaned) and before rehypeKatex/rehypeSlug (their output is trusted and would otherwise be filtered or clobbered) - the same raw -> sanitize -> math ordering streamdown itself uses. - The schema extends rehype-sanitize's GitHub-style defaultSchema (the base of streamdown's own sanitize schema) so legitimate authored artifact HTML (tables, details, images, alignment/size attributes) keeps working while script/iframe/style, on* handlers and non-allow-listed URL schemes (javascript:, data:, ...) are dropped. The only extensions are tel: hrefs and the math-inline/math-display class markers remark-math emits and rehype-katex detects. - Position rehypeSlug after the sanitize step in the artifact chain so sanitize's id clobbering (id="x" -> id="user-content-x") cannot break the heading anchors it creates. - Pass a: createMarkdownLinkComponent() on the memory settings page as defense in depth, matching the chat rendering path. Unit tests feed a hostile payload (<a href="javascript:...">, <img onerror>, <script>, <iframe>, <style>, ontoggle) through both render paths and assert no executable/clickable equivalent survives, plus regression guards for heading anchors, legitimate HTML and KaTeX math rendering. * fix(frontend): keep the sanitize clobber prefix on heading anchors; minimal lockfile Review follow-ups on the sanitization change: - Heading anchors: rehypeScopedSlug replaces rehype-slug in the artifact chain. It runs after the sanitize step (so raw-HTML headings are also anchored) but keeps rehype-sanitize's user-content- id prefix — an untrusted heading like "## current" cannot mint an unprefixed id="current" (the DOM-clobbering shape the sanitizer guards against). In-page fragment links are translated to the prefixed anchors so they still resolve; external URLs, bare "#", already-prefixed fragments and sanitize-prefixed raw-HTML ids are left untouched. - Lockfile: regenerated as a minimal diff — only the two direct-dependency importer entries (rehype-sanitize, github-slugger for the scoped slug) are added; the libc platform selectors on the 64 native package records are preserved byte-for-byte instead of being dropped by lockfile normalization. Full frontend suite: 1034 tests passing; tsc and prettier clean. * style: reorder github-slugger import ahead of the hast type import * test(e2e): expect the clobber-prefixed heading anchor in artifact preview The scoped slug plugin gives generated heading ids rehype-sanitize's user-content- prefix and translates fragment links to match, so the anchor-scroll test must locate the prefixed id. * fix(frontend): reset the scoped slugger per tree; keep footnote anchors single-prefixed Review follow-ups: - The scoped slug attacher holds one GithubSlugger, but streamdown caches the unified processor by plugin name, so the instance survived across parses and repeated renders of the same heading grew -1/-2 suffixes (the artifact-anchor e2e could not find the id on re-render). The transformer now resets the slugger per tree, as rehype-slug does; a regression test renders identical artifact markdown twice. - remark-rehype emits GFM footnote anchors already clobber-prefixed (user-content-fn-1); the sanitize step prefixed those ids again while their hrefs stayed single-prefixed, breaking footnote navigation in every chain built on the shared preset. A new rehypeClobberFragments step runs right after sanitize: double-prefixed ids are normalized back to one prefix, and unprefixed fragment hrefs are translated to the prefixed form (already-prefixed and external links untouched). The artifact slug now inserts after this step; covered by a footnote regression test on the shared render path. Unit suite 1036 passing; artifact-preview e2e verified locally (9/9, including the heading-anchor scroll test). |
||
|
YZJF,YCDG,DJLY,ZZZB
|
851e76661b
|
fix(docker): don't abort Docker startup when .env is missing (#4956)
* fix(docker): create compose env files and keep Windows compose paths relative Windows Docker reports a generic file-not-found when env_file targets are missing, or when compose paths are doubled. Make docker-start copy .env examples and invoke compose with filenames relative to docker/. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(docker): make dev compose env files optional and repair test fixture Address review feedback on #4956. [P1] prepare_compose_env aborted before the mocked COMPOSE_CMD in test_compose_commands_set_deer_flow_root_before_compose, because the temp root had no compose file or .env examples. Seed them in the fixture so the preflight reaches the mock. [P2] .env is gitignored, so a fresh clone has none and a direct `docker compose -f docker/docker-compose-dev.yaml up --build` aborts on Windows before scripts/docker.sh can help. Mark the dev env_file entries `required: false` so a missing .env is not fatal, and document that direct Compose must be run from the repository root. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(docker): declare Compose 2.24 floor and keep non-start commands read-only Address the second review round on #4956. - Document Compose >= 2.24 (CONTRIBUTING, README, compose header) and fail early from make docker-start with an actionable message; probe both `docker compose` and the hyphenated `docker-compose` binary. - Document DEER_FLOW_ROOT for direct Compose callers (bash + PowerShell); leave the variable without a $PWD fallback because PowerShell/cmd do not export it. - Split prepare_compose_env: compose_preflight is shared and read-only; ensure_env_files runs only from start. - Expand tests for version boundaries, hyphenated fallback, env-file creation, and read-only stop/logs/restart behavior. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(docker): reuse the probed Compose binary for wrapper operations The version probe could accept a standalone docker-compose install while COMPOSE_CMD stayed hardcoded to `docker compose`, so preflight passed and start/logs/stop/restart then failed. Keep the selected executable in COMPOSE_BIN (array), refresh COMPOSE_CMD from it in the current shell, and extend the fallback test through an actual stop invocation. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
georgelichen
|
641a4147e7
|
fix(skills): parse Responses API content blocks in moderation scanner (#4936)
* Fix skill moderation parsing for Responses API content blocks Normalize LangChain Responses API text blocks before parsing the security moderation decision, while preserving the existing fail-closed behavior for unavailable or invalid moderation results. Add regression coverage for mixed content blocks and document the compatibility boundary. Constraint: Responses API AIMessage content is list-shaped while Chat Completions content is string-shaped Rejected: Disable security scanning | would weaken the skill write safety boundary Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep moderation parsing provider-format tolerant without including reasoning or tool blocks in the decision payload Tested: 27 security scanner tests; ruff check; ruff format check Not-tested: Live moderation request against the configured external endpoint * Reuse shared LLM response text normalization Route skill moderation responses through the existing provider-format normalizer so only text and output_text blocks participate in JSON parsing. Strengthen regression coverage with reasoning and tool blocks that contain misleading text fields.\n\nConstraint: Responses API content is shared across multiple harness consumers\nRejected: Keep a private normalizer | duplicated provider-shape policy diverges and can reintroduce reasoning-block contamination\nConfidence: high\nScope-risk: narrow\nReversibility: clean\nDirective: Extend the shared normalizer when a new provider content shape is verified; do not add divergent local parsers\nTested: 118 related backend tests; regression test red against the previous parser; Ruff check and format check\nNot-tested: Live GitHub CLA status refresh * Restore trusted external skill package loading Skill discovery follows one-level package-directory symlinks, but activation path validation rejected the resolved external path. Restore that compatibility for configured custom-skill category roots while keeping file-level symlinks and deeper escapes blocked. Add regression coverage for local and user-scoped storage plus slash activation, and document the boundary. Constraint: Existing skill discovery follows directory symlinks and operator-managed external packages must remain loadable Rejected: Allow arbitrary resolved paths | would weaken the skill path trust boundary Confidence: high Scope-risk: moderate Directive: Keep the final SKILL.md file symlink-free and preserve one-level category-root validation Tested: 79 targeted skill storage, loader, slash activation, and user-scoped tests passed; Ruff check and format check passed; GitNexus staged change detection reported low risk Not-tested: Real symlink activation on this Windows host lacks SeCreateSymbolicLinkPrivilege and is skipped Related: Skill projection copies sources into sandbox-visible views * Exercise real filesystem symlink boundaries in skill storage tests Replace global Path.resolve/is_symlink mocks with real directory and file symlinks, preserving the Windows privilege skip. Add regression coverage for deeper custom-root escapes and symlinks under non-custom categories so the one-level allowance remains explicit. Constraint: Symlink creation requires SeCreateSymbolicLinkPrivilege on some Windows runners Rejected: Keep global path-method mocks | they validate the mock behavior rather than filesystem semantics Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep security-boundary tests on real filesystem primitives; skip only when the runner lacks symlink privilege Tested: 76 targeted loader/storage/slash tests; Ruff check; Ruff format check Not-tested: Windows symlink-enabled execution on this host Related: #4936 * Pin the actual nested symlink escape boundary Place the second symlink below a real custom package directory so the test reaches the one-level relative-parent guard instead of returning early on a non-symlink parent. Keep the public-category rejection coverage unchanged. Constraint: The security boundary depends on both symlink depth and category root Rejected: Link the outer package directory directly | the parent is not a symlink at validation time, so the depth guard is never evaluated Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep this regression tied to the exact relative_parent.parts depth check Tested: Targeted storage, loader, and slash suites; GitNexus staged detection Not-tested: Symlink-enabled execution on this Windows host Related: #4936 * Make the nested symlink regression reach the depth guard The test now validates the SKILL.md directly through the nested symlink, so the symlink is the immediate parent and the relative-parent depth check is executed. Constraint: Windows test execution may skip when symlink privilege is unavailable Rejected: Keep the extra nested path segment | it bypasses the symlink-depth guard through an early return Confidence: high Scope-risk: narrow Reversibility: clean Directive: Mutation tests must fail when the depth restriction is removed Tested: Targeted test (skipped on this Windows host without symlink privilege); Ruff check and format check Not-tested: Real symlink execution on Windows; Linux CI will exercise the case Related: #4936 * Keep sandbox projections fresh for linked external skill packages The storage layer intentionally accepts one-level custom package-directory symlinks, but projection freshness previously hashed only the link inode. Follow the permitted target tree during custom and legacy source-signature scans so edits to SKILL.md, scripts, references, or assets trigger a rebuild before sandbox use. Constraint: Preserve the existing one-level custom/legacy symlink boundary and do not follow public, integration, nested, or file symlinks Rejected: Invalidate projections only from /api/skills/reload | sandbox acquisition must also detect edits made directly in external targets Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep target-tree traversal limited to the storage paths that explicitly permit external package-directory links Tested: 77 projection, user-scoped storage, and lifecycle tests passed; Ruff check and format check passed; git diff --check passed; GitNexus staged detection reported low risk Not-tested: Real external symlink execution on this Windows host without SeCreateSymbolicLinkPrivilege; existing tests skip that platform limitation |
||
|
hataa
|
cc6a2657e7
|
feat(authz): enforce sandbox:execute authorization at sandbox acquisition (#4063 Phase 3) (#4911)
Sandbox is an execution environment, not a named resource: multiple tools
(bash, read_file, write_file, glob, grep, ...) depend on it, all funneled
through ensure_sandbox_initialized / ensure_sandbox_initialized_async. Gate
the single acquisition entry point (single source of truth) instead of
maintaining a sandbox-tool-name set in middleware:
- authorize_sandbox_execution helper (authz/sandbox_authz.py) checks
authorize("sandbox", "execute", target="*") — a binary judgment
(can this role use the sandbox at all); RBAC allow:"*"/true permits,
allow:[]/false denies.
- lazy path: ensure_sandbox_initialized (+ async) calls the gate before
provider.acquire.
- eager path: SandboxMiddleware.before_agent / abefore_agent call the gate
before _acquire_sandbox.
- deny raises SandboxAuthorizationError (SandboxError subclass) which
propagates through tool execution as a friendly ToolMessage (RFC §9:
'not a crash').
- authorization.enabled: false is a no-op everywhere; provider errors
follow fail_closed (deny) / fail_open (allow).
12 tests in tests/test_sandbox_authorization.py cover disabled/allow/deny/
deny-via-bool/no-policy-unrestricted/provider-error-fail-closed/open/
internal-caller + ensure_sandbox_initialized deny (never acquires) and
allow (acquires) integration paths.
|
||
|
ChaseMoon
|
cff8b74ec3
|
fix(harness): offload ACP workspace creation from event loop (#4965)
* fix(harness): offload ACP workspace creation from event loop * fix(harness): complete ACP event-loop offload |
||
|
Aari
|
645ca08f16
|
fix(scheduler): enqueue busy scheduled task runs (#4918)
* fix(frontend): clarify reuse-thread scheduling behavior * fix(scheduler): enqueue overlapping scheduled runs * fix(scheduler): preserve queue lease fencing * fix(scheduler): close queue concurrency races * fix(scheduler): harden queue timeout bookkeeping * fix(scheduler): preserve manual failure schedule --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Tsai Yuan
|
336cd3acc4
|
fix(sandbox): reject non-finite ownership timings (#4960) | ||
|
Aari
|
1aa813ddb3
|
feat: add managed subagents and delegation scopes (#4887)
* feat: manage and scope subagents * fix: address subagent review feedback * fix: address managed subagent review feedback * fix: harden subagent settings semantics * fix: harden managed subagent cache invalidation * fix: reuse assembled lead agent inputs * fix: migrate managed subagent definitions --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
陈志谦
|
9232e1e6a9
|
fix: DeerFlow brand casing in disclaimer, What's New title, setup-sandbox .PHONY (#4970)
* fix: DeerFlow brand casing in disclaimer, 'What's New' title, setup-sandbox .PHONY - en-US disclaimer said 'Deerflow'; every other user-facing string brands it 'DeerFlow' (rendered under the chat composer) - landing section title 'Whats New in DeerFlow 2.0' -> 'What's New' - setup-sandbox was the only Makefile target missing from .PHONY * test: align disclaimer assertions with DeerFlow brand casing The unit and e2e suites still expected the old "Deerflow" spelling in inputBox.disclaimer; update both to the corrected locale string so the i18n load test and the chat footer assertion pass again. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
wutongyuonce
|
6688d01c8f
|
fix(frontend): avoid recreating browser stream after reconnect (#4951)
* fix(frontend): avoid recreating browser stream after reconnect * test(frontend): cover reconnect delay reset |