* feat(scheduler): add interval schedule type
Allow scheduled tasks to fire every N seconds from last dispatch, not
only wall-clock cron or a single run_at. Cadence is UTC now+N with no
missed-beat catch-up, bounded by min_once_delay_seconds and 30 days.
* fix(scheduler): let interval tasks create, edit, and keep next run
Create/edit now keep every_seconds. Unchanged interval spec no longer
resets next_run_at, including timezone-only PATCH.
* fix(scheduler): keep non-minute intervals on edit
Stop rounding every_seconds to whole minutes in the form. Values that
are not whole minutes or hours now use a seconds unit so edit/duplicate
round-trips the stored cadence instead of rewriting it and resetting
next_run_at. Document that min_once_delay_seconds is also the interval
floor.
* fix(scheduler): clamp interval seconds to the default 60s floor
The new seconds unit allowed 1–59, which the API rejects under the
default min_once_delay_seconds. Clamp the form to >= 60 and show the
floor next to the preview. Also mention interval in the scheduler
field_doc, matching config.example.yaml.
* fix(scheduler): do not clamp interval amount while typing
Keystroke clamp made 90 become 9 -> 60, then 600, and backspace could
not leave 60. Keep the raw field text and apply the 60s floor on blur
and emit only.
* test(scheduler): cover interval input editing
* fix(frontend): preserve saved interval cadence until edited
* style(tests): format scheduled task router tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): let scheduled tasks pin a custom agent
Create and update accept optional assistant_id, defaulting to lead_agent.
Custom names are normalized and must already exist for the task owner.
The workspace form exposes the same choice, and duplicate copies it.
Fixes#5286
* fix(scheduler): keep assistant-id PR free of interval tests
Drop the six interval tests that belonged to the interval schedule PR
and fail here because this tree still only accepts once/cron.
Treat lead_agent case-insensitively so LEAD_AGENT / lead-agent store
as the default. Omit unchanged assistant_id on edit so a deleted custom
agent does not 422 unrelated PATCH (rename, reschedule).
* fix(scheduler): format task page and browser tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* test(checkpoint): retention deletion contract + growth baseline
Six contract scenarios x memory/sqlite/postgres pin what retention deletions
must never break (branch ancestors, explicit resume targets, pending writes,
duration-only chain links), prove the two safe shapes (leaf sibling branches,
trailing duration leaves), record the full-vs-delta growth baseline in the
normalized bench shape, and add an item 4 probe showing the default
ToolOutputBudgetMiddleware already externalizes oversized tool results.
Refs #4189
* test(checkpoint): make the retention contract load-bearing per review
Review findings from willem-bd and Ricky-7-Yan:
- scenario D pins its own row: before/after stats delta plus a serde
round-trip of the stored write, instead of an always-true > 0 check
- _delete_checkpoint now performs the joint delete the doc mandates
(checkpoint row + writes rows + blobs unreachable from surviving
checkpoints), so E1/E2 exercise the shape they prescribe
- E1 builds the real runtime duration shape via persist_run_durations
(parent dict clone, fresh id/ts, real metadata), which surfaces the
shared-version case: the leaf's blobs are the surviving parent's rows
- contract doc: blob reachability must be computed from surviving
checkpoints in a whole-thread pass; shared-version/duration-only
hazard called out explicitly; memory data model includes saver.blobs
- _stats counts memory blob rows and returns the full normalized shape
(logical byte totals included)
- probe: drops the unused middleware/outputs_dir graph parameters and
discloses the manual-harness scope limit in the module docstring
- E1/E2 assert default head resolution (protected set item 5); unused
graph_for helper and DURATION_ONLY_METADATA stand-in removed
Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
* fix(checkpoint): scope probe cleanup to owned dirs, key report by backend
Second-round review findings on #5255:
- [P1] bench_tool_result_probe.py removed the whole user-supplied
--outputs-dir (and the shared .probe-tmp) in its finally block, so
pre-existing files were deleted on success and failure alike. The run
now writes into (and removes) a fresh owned probe-run-* child beneath
the requested directory, and SQLite databases live in a unique
mkdtemp'd temp directory that is removed with the run. Regression
tests pin that unrelated pre-existing files survive both a successful
and a simulated failing run.
- [P2] the optional retention report keyed every backend's measurements
under one shared name, so a multi-backend invocation kept only the
last backend's numbers. _report() now takes the parameterized backend
explicitly (saver_env.kind); regression pins that memory and sqlite
entries coexist in one report file.
Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
---------
Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
Co-authored-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
* feat(gateway): support idempotent thread runs
Accept Idempotency-Key on thread-scoped create, stream, and wait endpoints, scoped by owner and thread before durable admission.\n\nRefs #5257.
* fix(gateway): handle idempotent run reuse on wait and stream
Reused store-only records have no local task. /wait now waits on the
bridge when it can observe the stream, and otherwise returns durable
status instead of a stale checkpoint. A reused terminal stream that has
been evicted emits gap/reload_durable_state. Replay is bound to the
original input and assistant_id.
* fix(gateway): 409 reused in-flight streams on this worker
A store-only running record on a process-local bridge has no owner
stream. POST /runs/stream used to subscribe anyway, which created an
empty log and waited forever. Match join: 409 unless the run is already
terminal, so missing-stream retries can still emit gap.
* fix(gateway): keep observer joins off the idempotent stream-gap path
sse_consumer keyed missing-stream gap on the sticky
idempotency_reused flag, so a later join of a terminal run inherited
it. Gate that branch on apply_on_disconnect, which already separates
creating streams from joins. Document the retry outcomes clients have
to handle.
* fix(gateway): gate missing-stream gap on creating retry
Reuse apply_on_disconnect to pick gap vs end changed sse_consumer default path, so a missing stream started returning gap for default callers and for out-of-scope POST /api/runs/stream. Keep that branch behind emit_gap_on_missing_stream and pass it only from thread-scoped /runs/stream on this request reuse.
* fix(gateway): keep wait reuse off later checkpoints
Direct handler calls were crashing because FastAPI Header() leaked in as the Python default. Bind Idempotency-Key with Annotated so the default is None, and ignore non-str keys.
A reused completed /wait was still reading the latest thread checkpoint. After a later run on the same thread that is the later run's result. Return durable status instead of claiming the head as this run's output.
* fix(gateway): snapshot wait reuse and refresh store status
idempotency_reused lives on the shared cached record. Capture it before awaiting completion so an overlapping retry cannot suppress the original creating /wait checkpoint.
A store-only peer record still holds admission-time status after the owner publishes END. Refresh durable status/error before returning them.
* feat(community): add Sofya web search provider
Add a community provider backed by Sofya (https://sofya.co). Its search
endpoint returns the content of the result pages, not only their snippets,
and its fetch endpoint returns a page as markdown. Both are plain JSON over
HTTP, so this needs no extra Python package (uses httpx, already a
dependency).
Changes:
- backend/packages/harness/deerflow/community/sofya/__init__.py
- backend/packages/harness/deerflow/community/sofya/tools.py
Implements web_search_tool and web_fetch_tool using httpx.
API key is read from the config.yaml `api_key` field or the SOFYA_API_KEY
env var. Follows the same interface and output shape as the existing
ddg_search and serper providers, including the max_results parameter with
config override and the structured "No results found" error.
- backend/tests/test_sofya_tools.py
Unit tests covering API key resolution, config overrides, result mapping,
time range, HTTP errors, empty results, and fetch failures.
- config.example.yaml: add commented-out Sofya web_search and web_fetch
examples alongside the other providers
- .env.example: add SOFYA_API_KEY placeholder
- backend/docs/CONFIGURATION.md: list Sofya under web_search, web_fetch and
the environment variables
* fix(sofya): honor caller max_results, validate search_depth, join time_range contract test
- Caller-supplied max_results now wins; config is used only when the
argument is omitted, matching GroundRoute.
- search_depth is clamped to basic/snippets; an unsupported value logs a
warning and falls back to basic.
- Sofya added to the shared time_range schema contract test.
* fix(sofya): cap per-result content so a search stays inline
An unbounded search payload (up to 20 read pages) crossed the tool output
budget middleware's externalize_min_chars threshold, which replaces the
result list with a file reference. Cap each result's content at
contents_max_characters (default 2000, 0 disables), matching Exa's config
key. Five capped results stay under the 12000 char threshold.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* fix(sofya): list Sofya in the recency contract, coerce non-string content
_clip subscripted its input, so a non-string content or description from
the API raised TypeError instead of degrading. Coerce to text first, the
way _sofya_post and _response_results guard the shapes around it. Also add
Sofya to the Web Search Recency section in backend/AGENTS.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* fix(sofya): coerce web_fetch content, list sofya in the tools guide, add changelog
web_fetch sliced its content the same way web_search did before the last
push: a truthy non-string from the API passed the falsiness guard and then
raised TypeError. Reuse _clip, keeping the `or ""` so empty content still
reports "No content found".
Also add sofya to the community provider inventory in
packages/harness/deerflow/tools/AGENTS.md and an [Unreleased] changelog entry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* docs(zh): add the missing InfoQuest and Firecrawl web_fetch tabs
The ZH web_fetch tab list named five providers where EN names seven. Both
tabs mirror their EN counterparts, so the two locales list the same
web_fetch providers again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(summarization): resolve fraction triggers from declared context_window, degrade instead of crashing the agent build
A fraction trigger/keep clause requires profile["max_input_tokens"], which any
third-party OpenAI-compatible model lacks, so SummarizationMiddleware
construction raised ValueError out of create_summarization_middleware and failed
the whole agent build (#3103).
- factory: translate a declared model context_window into the langchain
profile (metadata-only, never reaches the provider payload); explicit
caller/override profiles win
- summarization factory: drop unusable fraction trigger clauses (absolute
clauses survive), fall a fraction keep back to the messages default, and
disable compaction with an actionable warning only when no usable trigger
clause remains — the agent build never dies from summarization config
- docs: config.example.yaml, ModelConfig.context_window, summarization.md
* refactor(summarization): share the default keep constant with the fraction fallback
The fraction-keep degradation fallback hardcoded ("messages", 20),
duplicating SummarizationConfig.keep's default_factory literal. Move the
value to a shared DEFAULT_KEEP constant so the two cannot drift apart.
* fix(summarization): keep trigger-null + fraction-keep constructing after degradation
A trigger of None with a fraction keep hit the all-clauses-dropped branch
(has_usable_trigger=False) and disabled compaction, and the accompanying
warning claimed configured triggers were all fraction-based when none were
configured. Only report nothing-usable when trigger clauses actually
existed; trigger:null keeps constructing the never-firing middleware with
the degraded keep, matching its behavior outside the degradation path.
* fix(summarization): address review — keep manual compaction, validate ContextSize, pin wiring
Review follow-ups on #4901:
- When every configured trigger is a dropped fraction clause, keep
constructing the never-firing middleware (trigger=None) instead of
returning None: manual /compact runs with force=True and never consults
trigger clauses, so it must keep working for a profile-less model
rather than reporting 'compaction is disabled'. The warning now says
auto-compaction will not fire while manual compaction remains.
- ContextSize gains a config-load validator: fraction values must be in
(0,1] (a percent-style 80 instead of 0.8 previously produced a threshold
the context could never reach — a silently inert trigger), absolute
values must be positive.
- New un-monkeypatched integration test pins the shipped wiring
(context_window declared -> real factory attaches profile -> fraction
clause survives -> middleware constructs), which the stubbed
middleware-side tests and kwarg-capturing factory-side tests each
stopped short of.
- Docs (summarization.md + config.example.yaml) clarify that the fraction
resolves against the summary/anchor model's context_window
(summarization.model_name when set, else the run model), including the
mismatch caveat for a larger-window summary model.
* fix(summarization): reject non-finite ContextSize values at config load
YAML .nan / .inf pass pydantic's float parsing, and nan <= 0 is False,
so the positivity check alone let them through as dead thresholds
(count >= nan is always False) — the same silent-inert-trigger class the
range validator was added to close. Guard with math.isfinite first,
consistent with the existing non-finite guards on mem0 timeout_seconds
and poll_after_seconds.
* fix(summarization): merge context_window into inferred profile, require whole message counts
- construct the model first, then merge max_input_tokens into the
provider-inferred langchain profile: passing profile= to the
constructor replaced the whole inferred metadata (tool_calling,
structured_output, io capabilities, output limits) with the single
key. An explicitly configured profile is still never clobbered.
- reject non-integral ContextSize values for type=messages at config
load: langchain slices the message list with them, so a float index
raised TypeError mid-compaction.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(auth): make login rate-limit parameters configurable, fixes#5108
Add auth.local.max_login_attempts (default 5) and auth.local.lockout_seconds
(default 300) so operators can tune the per-IP login throttle: raise the
ceiling for shared-egress-IP offices behind proxies/NAT, or tighten it for
stricter posture. Policy is live-read per call (matching the
_local_registration_enabled precedent), so a config reload applies without a
Gateway restart; raising the threshold mid-lockout immediately unblocks
affected IPs.
Review feedback addressed (willem-bd):
- Only FileNotFoundError falls back to the hardcoded defaults; a malformed
config propagates, mirroring _local_registration_enabled, so an operator
who tightened the policy never silently gets the more permissive defaults.
- _check_rate_limit looks up the record before resolving the policy, so a
clean IP pays zero config reads (get_app_config re-hashes config.yaml per
call and login_local is an unauthenticated async endpoint).
Bumps config_version to 39 in config.example.yaml and the Helm chart
(values.yaml + README example) so the chart drift check stays green.
* fix(auth): reject max_login_attempts=1 and honor live lockout_seconds for active lockouts
* fix(auth): close live-policy state gaps in login throttle (resurrection, count reset, broken-config verification)
* fix(auth): commit evaluated lockout duration on decreases too, preventing raise-resurrection
* test(auth): pin broken-config fail-closed sequence through the login route
* fix(auth): sweep expired locks by stored sentence and keep policy reads off the event loop
* fix(auth): re-read throttle record after the policy-resolution yield point
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(mcp): reject credentials that cannot travel as HTTP header values
A request-scoped secret or user_auth credential with a trailing newline
(the usual result of reading a token from a file, or a CRLF env-file),
CR/LF, surrounding whitespace, or characters outside Latin-1 sailed
through the credential interceptors into the HTTP client, where httpx/h11
reject it with an exception that echoes the full value:
LocalProtocolError: Illegal header value b'Bearer sk-...\n'
ToolErrorHandlingMiddleware copies that message into a model-visible
ToolMessage, so the secret landed in the prompt, the checkpoint, and
traces - everywhere headers_from_context promises it never goes.
Add illegal_header_value_reason to mcp/headers.py, mirroring the
transport's own rules (Latin-1 encodable; h11's field_vchar is [^\x00\s]
with SP/HTAB legal only between visible characters), and fail closed in
both interceptors before the value can reach the client. The denial names
only the secret key (plus the reason) and never repeats the value.
Illegal values are denied regardless of on_missing: the key is present,
so a passthrough fallback would silently run the call under the shared
discovery credential - the exact authority confusion the deny default
exists to prevent.
Values the transport accepts are not rejected: embedded SP/HTAB
('Bearer <token>'), Latin-1 high bytes, and DEL all still pass, pinned
by tests against h11's observed behaviour.
* fix(mcp): tighten header value validation to httpx's ASCII boundary
The validator mirrored h11's Latin-1 boundary, but the transport rejects
more than h11 does: build_server_params hands dict[str, str] headers
through the MCP SDK's create_mcp_http_client into httpx.AsyncClient, and
httpx (pinned 0.28.1) encodes str header values as ASCII - so a Latin-1
high byte like 'Bearer caf\xe9' passed validation here only to raise
UnicodeEncodeError inside httpx before h11 ever ran, with the exception
message repeating the offending value.
Validate str values against ASCII instead, flip the tests that pinned
Latin-1 high bytes as transportable, and pin the boundary against the
real client: create_mcp_http_client must reject what the validator
flags and construct cleanly for what it accepts (embedded SP/HTAB and
DEL still pass).
Addresses review feedback on the ASCII vs Latin-1 boundary.
* fix(mcp): validate OAuth and static header values at the same boundary
The validator added for headers_from_context and user_auth left two paths
uncovered. A token endpoint returning an access_token or token_type with a
newline reached httpx/h11, which raise with the full token in the message, and
ToolErrorHandlingMiddleware copies that message into a model-visible
ToolMessage -- the leak this PR set out to close. The operator's static headers
had the same hole.
OAuthTokenManager.get_authorization_header now renders the Authorization value
through one checked helper, so the tool interceptor, the initial discovery
headers and the durable task path are all covered by a single guard. The
rendered value is what gets checked rather than the two fields separately,
because that is what the transport sees: an access_token with leading
whitespace is legal once it follows "Bearer ".
build_server_params applies the same check to statically configured headers.
build_servers_config already isolates a per-server failure, so a bad value
drops that one server and logs the reason instead of the value.
* docs(mcp): correct which transport echoes the full header value
The rationale claimed httpx and h11 both render the full value into their
exception message. Only h11 does, on the line break and surrounding whitespace
cases. httpx's ASCII failure is a UnicodeEncodeError naming the offending
character and its position, not the credential, so at most one character
escapes there; refusing the value up front buys an actionable error rather than
an encode failure raised from inside the client.
Corrected in headers.py and in every copy of the claim: context_headers.py,
user_scoped_auth.py, oauth.py, client.py, mcp/AGENTS.md, docs/MCP_SERVER.md,
the frontend mcp.mdx, and the test comments carrying the same wording. No
behavior change.
---------
Co-authored-by: Terminator666666 <Terminator666666@users.noreply.github.com>
* feat(e2b-sandbox): make mount upload deadline configurable
Replace the hardcoded 120-second mount upload deadline with a
configurable `mount_upload_deadline_seconds` key read from
SandboxConfig (extra=allow). The value is validated: zero and
negative inputs are clamped to 1 second. Omitting the key
preserves the existing 120-second default.
This addresses the follow-up from PR #4842 review: operators
with large mounts or slow networks can now size the deadline to
their deployment without changing code.
* fix(e2b-sandbox): address review feedback on configurable deadline
- Remove import-time default capture from _mount_deadline_reason()
and _MountUploadBudget.deadline_seconds to prevent silent drift.
- Add warning log when mount_upload_deadline_seconds is clamped to 1
(was silent before).
- Update AGENTS.md E2B Mount Uploads section: deadline is now
configurable, not fixed 120.
- Add mount_upload_deadline_seconds to YAML examples in provider
docstring and __init__.py.
- Add config-path test that exercises SandboxConfig -> _load_config ->
_apply_mounts end-to-end.
* fix(e2b-sandbox-provider): handle non-numeric mount_upload_deadline_seconds
Guard _resolve_mount_upload_deadline against None, non-numeric strings,
and other invalid values. None returns the default; non-numeric strings
like '120s' or 'abc' log a warning and fall back to the 120-second
default instead of crashing provider init with TypeError/ValueError.
Extend the parametrized clamp test with None, suffix, and alpha cases,
and add a warning assertion. Update CONFIGURATION.md with the new
mount_upload_deadline_seconds key and its behavior.
* fix(sandbox): handle infinite mount deadline
* feat(auth): add personal access tokens for programmatic API access (#4849)
Backend-first implementation of the PAT contract from #4849: show-once
dfp_ tokens bound to their owning user (AUTH_SOURCE_PAT,
is_internal=false), digest-only storage (migration 0017), strict
credential precedence (invalid Bearer is a 401, never cookie fallback),
CSRF double-submit skipped only for Bearer requests while
auth-endpoint origin checks still run, scopes intersecting the authz
route permissions, session-auth-only PAT management and password
changes, and throttled best-effort last_used_at stamps.
* fix(auth): harden PAT scope boundary and schema parity from adversarial review
Independent review of the initial draft found: (1) scopes only constrained
the threads/runs permission axis while admin routes treated a PAT as its
(possibly admin) owner — is_admin_user now rejects PAT callers outright
since no scope grants admin capability; (2) the model declared a column
UNIQUE constraint while migration 0017 created a named unique index, so
downgrade failed on create_all-bootstrapped DBs — both now use the named
unique index; (3) auth-disabled mode is an operator override and now stays
ahead of the Bearer check so a stray Authorization header cannot 401 an
E2E sandbox; plus wiring the previously-unused constants, bounding the
last_used_at stamp cache, and four new tests (middleware-level expiry,
expires_in_days, admin-capability rejection with session control, and the
auth-disabled precedence).
* docs(api): document personal access tokens for programmatic API access
* fix(auth): close PAT security boundaries from review (default-deny routes, extension admin suppression)
P1-1: scope intersection only constrains @require_permission routes, so
undecorated mutation routes (DELETE /api/memory, POST /api/agents, Lark
credential switching, channel config) accepted a PAT holding a single read
scope. AuthMiddleware now enforces a default-deny route policy in
auth/pat.py: PAT requests are admitted only to the thread/run lifecycle
routes the v1 scopes govern; everything else answers 403 regardless of
scopes. Session-cookie callers are unaffected.
P1-2: the extension principal resolver projected is_admin/roles from the
raw system_role, so an admin-owned PAT passed
deerflow_extension_api.require_admin on contributed routes despite the
documented no-admin guarantee. The projection is now PAT-aware and
suppresses every admin signal for PAT callers, mirroring
deps.is_admin_user.
Both fixes carry regression tests (route outside policy 403 + session
control; production resolver admin suppression), and API.md documents the
default-deny boundary.
* fix(auth): enforce PAT scopes on stateless run entry and harden decorator
Follow-up hardening from an independent audit of the P1 fixes:
- POST /api/runs/stream and /api/runs/wait were the only allowlisted run
entrypoints without @require_permission, so a threads:read-only PAT
could still start runs (same bug class as P1-1, now closed): both now
carry @require_permission("runs", "create"). POST /api/threads and
POST /api/threads/search gain threads:write / threads:read for the
same reason. Authorization-disabled deployments see no change (the
permission set resolves to all permissions).
- require_permission now binds the wrapped signature to locate a
positionally-passed request before injecting the test stub, fixing
'got multiple values for argument' on direct positional unit-test
calls.
- API.md: the intro PAT example used GET /api/models, which the new
default-deny policy 403s — replaced with GET /api/threads; the
default-deny route list now spells out method sets.
Regression test: threads:read-only PAT is 403 on the decorated stateless
entry while a runs:create PAT passes.
* fix(auth): address review P2s (empty Authorization header, PAT name trimming, API example)
- CSRFMiddleware treats an explicitly empty Authorization header as
present (is None), so an invalid credential always reaches
AuthMiddleware's uniform 401 instead of a CSRF 403 that varies by
method/CSRF state. Regression: empty-header request dies at auth.
- PATCreateRequest strips the name and rejects whitespace-only values
before token generation; created names are stored trimmed.
- API.md intro PAT example now uses the implemented
POST /api/threads/search endpoint (GET /api/threads does not exist).
- AGENTS.md trimmed back under the guidance soft budget after the
upstream merge.
* fix(auth): tighten PAT route policy to implemented methods only
The allowlist admitted GET /api/threads, a method no router implements.
Pre-authorizing a dead method weakens the default-deny boundary: a
future GET collection route added without a permission decorator would
become PAT-reachable without an explicit policy change. Restrict the
rule to POST, fix the stale GET description in API.md's PAT
constraints, and document the default-deny boundary accurately in the
gateway AGENTS.md guidance (only the threads/runs allowlist is
PAT-reachable; every other authenticated route 403s PAT callers).
Audited every remaining rule against the mounted routers: all other
method+path entries map to real routes. Regression:
test_pat_policy_does_not_pre_authorize_unimplemented_methods.
* test(auth): guarantee the negative digest test mutates the token
token[:-1] + "X" is identical to the original whenever the generated
token already ends in X (1/62), making the negative digest assertion
fail intermittently. Choose the replacement character based on the
existing tail so the mutated token always differs.
* fix(auth): require runs:cancel for cancel-then-stream requests
stream_existing_run is gated at runs:read so action-less stream joins
work with read-only credentials, but its ?action=interrupt|rollback
branch cancels the run — a separate permission. A runs:read-only PAT
passed both the PAT route policy and the route decorator and could
interrupt or roll back an active run, bypassing the runs:cancel scope.
Decorators cannot express query-parameter-conditional permissions, so
the check lives in require_cancel_permission_when_action(), applied at
the top of the handler. Regression drives the real helper through the
production middleware: runs:read-only PAT + action is 403, the same
token joins action-less, runs:read+cancel passes, session control
unaffected.
* docs(changelog): add the PAT feature entry
* docs(readme): add personal access tokens section
Repo documentation-update policy requires user-facing features to
update README.md in the same changeset; the PAT feature previously
touched only backend/docs/API.md and the gateway AGENTS.md.
* fix(auth): require runs:cancel for mutating multitask strategies
All five run-creation entrypoints were gated only by runs:create, but
RunCreateRequest.multitask_strategy accepts interrupt/rollback and
start_run forwards it to create_or_reject, which terminates an
already-active run. A runs:create-only PAT could therefore kill an
existing run through a create request, bypassing runs:cancel.
Decorators cannot express body-parameter-conditional permissions, and
per-route checks leave the same hole for the next entrypoint, so the
gate lives in start_run itself — the single choke point every
run-creation path (HTTP routes and internal launchers) flows through.
Regenerate launches pass multitask_strategy="reject" and are
unaffected; requests without a stamped auth context (internal/test
compositions) skip the gate.
The check is the shared authz.require_cancel_permission_if primitive;
require_cancel_permission_when_action now delegates to it, so every
request dimension that carries cancel capability (query action, body
strategy) flows through one gate.
Regression drives the real middleware stack: runs:create-only PAT +
interrupt/rollback is 403 with the exact detail, reject (explicit and
default) stays available, runs:create+cancel passes, session control
unaffected; a source anchor pins the gate inside start_run.
* fix(runs): keep observer joins from applying creator cancel-on-disconnect
sse_consumer's finally block applied the record's on_disconnect=cancel
policy on ANY consumer's disconnect. The join surfaces (GET /join and
the action-less GET/POST stream join) feed it the existing RunRecord,
so anyone with thread read access — including a runs:read-only PAT —
could cancel a locally-owned running run simply by closing the SSE
connection, without runs:cancel. The policy expresses the creator's
intent for their own connection; an observer's disconnect must never
be read as that intent.
sse_consumer gains apply_on_disconnect (default True). The two join
surfaces pass False; the creating endpoints (thread-scoped and
stateless create-and-stream) keep the creator semantics unchanged.
wait_for_run_completion needs no change: its callers are creator-side
or post-explicit-cancel paths only.
Regression exercises a real generator close — the same machinery
Starlette drives on client disconnect — against the production
sse_consumer: creator stream disconnect cancels, observer join
disconnect does not; a wiring anchor pins both join call sites and the
creator defaults. API.md documents the cancel-capability constraint
(this fix plus the action/strategy gates) in PAT Constraints.
* test(auth): pin the multitask gate behaviorally; state wait invariant
Independent adversarial review of the round-5 fixes found the P1-a
regression only mirror-pinned: the source anchor could be satisfied by
a comment, and deleting the gate from start_run would not fail the
suite. This drives the production start_run directly — a create-only
auth context gets 403 with the exact detail for interrupt, and a
reject request with no cancel permission at all proceeds past the gate
(never a permission 403).
Also documents wait_for_run_completion's creator-side invariant
(every caller is the creating endpoint or post-explicit-cancel) so a
future observer wiring thinks twice before reusing it — the one-caller-
away variant of the observer-disconnect P1.
* docs(changelog): correct the PAT entry's digest and route-policy description
The entry said HMAC digests (the implementation stores SHA-256 digests,
as documented in API.md and pinned by the repository tests) and claimed
the route policy admits 'implemented stateless endpoints' (it admits
the thread/run lifecycle routes, narrowing further by scopes). Also
notes the cancel-capability gate now covering action and multitask
strategies.
* fix(auth): enumerate the PAT runs route policy per implemented subroute
The runs subtree rule was a GET|POST /runs(/.*)? wildcard — it
pre-authorized every current and future subroute under /runs, including
methods the router never implemented (e.g. GET /runs/stream), which is
the same latent default-deny weakening the threads collection rule was
tightened for: a future route added under /runs would become
PAT-reachable without an explicit policy change.
The wildcard is replaced with six segment-precise rules covering exactly
the 14 implemented method+path combinations; the {run_id} slot
necessarily matches any single segment, so the POST-only collection
names (stream, wait, regenerate, edit-regenerate) are excluded from the
GET run-id rule via negative lookahead — no dead method stays
pre-authorized. Behavior for implemented routes is unchanged.
test_pat_runs_policy_admits_exactly_the_mounted_routes derives the
expected set from the mounted thread_runs router instead of a
hand-maintained list: every implemented GET/POST route under /runs must
be admitted, routes in this router outside the subtree stay denied, and
representative unimplemented neighbors are denied — so adding a route
under /runs now fails CI until it is explicitly allowlisted, and a
removed route leaves a dead rule visible. API.md's PAT constraints list
the enumerated routes and drops a feedback mention that belonged to the
stateless /api/runs axis.
* docs(migration): add the 0017 renumbering coordination note to 0017
The PR's migration-coordination comment states each migration file
carries the note; the file did not. Adds it: numbering was generated
against main head 0016 alongside #5078 and #4843; whoever merges first
keeps the slot, the others renumber on rebase (revision/down_revision
plus the bootstrap head assertions).
* fix(auth): pad base62 tokens to a fixed 43-char width
int.from_bytes discards leading zero bytes, so the unpadded encoder
returned a variable-length body — empty for all-zero input, and shorter
than 40 characters for any draw below 62**39 (~1 in 14.5M), leaving
test_generate_pat_token_format probabilistically flaky and the token
body without stable width (review round 6, P3).
_base62 now left-pads with "0" to _base62_width(len(data)) — the exact
integer digit count (62^43 > 2^256 > 62^42, so 43 for 32 bytes). The
format test asserts the exact fixed width instead of a probabilistic
floor, and a new unit test pins the all-zero, leading-zero-byte, and
max-value edges deterministically.
* fix(runtime): prevent IndexError in MemoryStreamBridge._make_gap on empty events buffer
* fix(runtime): handle empty stream replay gap bounds across backend and frontend
- Clamp MemoryStreamBridge queue_maxsize at 1 and validate StreamBridgeConfig.queue_maxsize >= 1
- Update StreamGap docstring to clarify None retained bounds
- Allow StreamReplayGapData and parseStreamReplayGap in frontend to accept string | null bounds, safely resuming when bounds are null
- Add backend and frontend regression unit tests for queue clamping and null bounds replay gap
* docs(stream-bridge): bump config_version and document empty buffer replay gap behavior
* docs: document nullable gap bounds and sync helm config_version to 37
Add deerflow.community.serply.tools:web_search_tool, a Google SERP
provider for the web_search slot that also covers Google News and Google
Scholar through an optional `vertical` config option. Reads the key from
api_key in config.yaml or SERPLY_API_KEY, clamps max_results to Serply's
1-100 range, and returns the same structured JSON errors as the Serper
and Brave tools.
Register the provider in config.example.yaml, scripts/doctor.py,
scripts/wizard/providers.py, .env.example, backend/docs/CONFIGURATION.md,
the en/zh tools.mdx provider tabs, and tools/AGENTS.md. Tests mock httpx.
* fix(sandbox): harden local Docker sandbox containers and port binding
Root causes (security audit SBX-1/SBX-2) in the local container backend:
- _resolve_docker_bind_host published sandbox ports on 0.0.0.0 whenever
DEER_FLOW_SANDBOX_HOST was non-loopback (docker-compose defaults to
host.docker.internal), exposing the unauthenticated /v1/shell/* exec
API on every host interface.
- _start_container ran every sandbox with seccomp=unconfined and no
capability, privilege-escalation, or resource limits, so untrusted
model-authored code could exhaust the host, escalate privileges, and
reach internal networks / cloud metadata endpoints directly.
Hardening changes and defaults:
- Port binding: non-loopback sandbox hosts now bind the Docker default
bridge gateway instead of 0.0.0.0, discovered dynamically via
`docker network inspect bridge` with a static 172.17.0.1 fallback.
host.docker.internal resolves to that gateway through host-gateway,
so DooD gateways and the Docker host still reach the sandbox while
external interfaces no longer see the port.
DEER_FLOW_SANDBOX_BIND_HOST=0.0.0.0 restores the legacy broad bind.
- seccomp=unconfined is no longer unconditional: sandboxes run with
Docker's default seccomp profile; opt back in with
DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED=1, only when the sandbox image
is verified to require syscalls the default profile blocks.
- Add --cap-drop=ALL and --security-opt no-new-privileges (Docker only;
the Apple Container CLI does not support these flags).
- Bounded resources with env overrides: --memory 2g
(DEER_FLOW_SANDBOX_MEMORY), --cpus 2 (DEER_FLOW_SANDBOX_CPUS),
--pids-limit 512 (DEER_FLOW_SANDBOX_PIDS_LIMIT); each also accepts
"0"/"none" to disable the limit.
- No --user is forced by default (the default AIO sandbox image's user
is upstream-controlled and unverified), but
DEER_FLOW_SANDBOX_CONTAINER_USER passes one through for deployments
that know their image.
- DEER_FLOW_SANDBOX_NETWORK passes --network so sandboxes can be
attached to a dedicated egress-controlled network; default networking
is unchanged.
backend/docs/CONFIGURATION.md documents the new bind behavior and every
override; tests cover each default and escape hatch.
* fix(sandbox): follow host-gateway mapping for binds; keep image-required seccomp default
Review follow-ups on the hardening change:
- Bind: resolve the sandbox host itself and bind that address, instead of
assuming the default bridge IPv4. host.docker.internal follows the
daemon host-gateway-ip mapping (customizable, possibly IPv6), so the
resolved address is exactly where the gateway connects — the published
port and advertised URL always match. IPv6 is bracketed for docker -p,
zone ids stripped, wildcard resolutions ignored; unresolved hosts fall
back to the bridge gateway with a warning pointing at
DEER_FLOW_SANDBOX_BIND_HOST.
- seccomp: the shipped AIO image needs seccomp=unconfined for its
Chromium browser (upstream quick-start always passes it; the upstream
FAQ documents the browser failing under Docker default profile), so
that option returns as the default. Tightening stays possible via
DEER_FLOW_SANDBOX_SECCOMP_PROFILE=<path to a restricted,
Chromium-compatible profile> or DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED=0
for images verified to work with Docker's default profile.
- cap-drop/no-new-privileges and the resource limits are unchanged.
- Tests updated for both behaviors; 37 pass.
* fix(sandbox): bracket bare IPv6 bind overrides; state seccomp default accurately
DEER_FLOW_SANDBOX_BIND_HOST was returned verbatim, so a bare IPv6 literal
like fd00::1 produced an invalid publish spec (fd00::1:port:8080); Docker
requires the bracketed form. Normalize raw and already-bracketed IPv6
literals (IPv4/hostnames untouched), with resolver-level and argv-level
tests covering the explicit IPv6 override.
The CONFIGURATION.md overview claimed Docker's default seccomp profile
stays active, contradicting the seccomp=unconfined default the table (and
the code) actually ship for the Chromium-based image; spell out the relaxed
default and where to change it.
* style(sandbox): apply ruff format to local_backend
* fix(sandbox): reject host networking, force builtin seccomp opt-out, resolve hostname binds
Review follow-up on #4986 (willem-bd):
- P1: DEER_FLOW_SANDBOX_NETWORK=host (and container:<name>) now raise a
RuntimeError at start instead of silently voiding the hardened port
bind — Docker discards -p/--publish in host mode and shares the
network namespace for container:<name>, which would re-expose the
unauthenticated exec API on the host's interfaces. Two regression
tests cover both rejections.
- P2: the seccomp opt-out now passes seccomp=builtin explicitly instead
of omitting the option, so a daemon configured with an unconfined or
custom default cannot weaken the documented opt-out; the test asserts
the flag.
- P2: hostname values in DEER_FLOW_SANDBOX_BIND_HOST resolve to an
address before use (Docker publish specs require an IP literal as the
host part, so host.docker.internal previously produced an invalid
spec that prevented every sandbox from starting); unresolvable names
raise a clear configuration error. Tests cover resolution and
rejection; CONFIGURATION.md updated for all three behaviors.
43/43 pass in tests/test_aio_sandbox_local_backend.py; ruff check +
format clean.
* fix(sandbox): reject DEER_FLOW_SANDBOX_NETWORK=none (loopback-only, breaks published API port)
* fix(sandbox): validate the effective Docker network target; normalize IPv6 sandbox hosts once
name=host / name=none dodge raw-string checks but attach like the bare
words; strip name= prefixes and validate the effective target (network IDs
keep passing). Bracketed IPv6 sandbox hosts now resolve for the bind and
bare IPv6 hosts produce bracketed URL authorities — both input forms give
identical bind and URL addresses.
* fix(sandbox): parse the full Docker network long syntax before validating
Docker accepts comma-separated key=value fields in any order (name=, gw-priority=,
alias=, ...); a name=host field hides the host network behind surrounding fields.
Parse the CSV and validate the parsed name= target (last occurrence wins, fields
lowercased, mirroring opts/network.go); no-name values fall through like Docker's
own rejection.
* fix(sandbox): keep CHOWN/SETUID/SETGID through cap-drop=ALL for the default image
The shipped image's entrypoint starts as root, creates the gem user,
chowns /opt/jupyter and drops to that user via su; without those three
capabilities the set -e script dies before the readiness endpoint exists.
no-new-privileges stays (it blocks gaining privileges via exec, not using
the added caps). Adds a docker-gated real-image startup smoke test.
* fix(sandbox): let pre-initialized non-root images drop the startup capabilities
The CHOWN/SETUID/SETGID re-add only exists for the shipped image's root
entrypoint handoff. A custom image that never runs as root gets an explicit
opt-out (DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS=0) so those capabilities are
not left available to sandboxed code (chown on bind mounts, UID/GID
impersonation).
* test(sandbox): gate the real-image smoke test behind the live marker
The default offline suite (make test = -m 'not live') must not depend on a
third-party registry: mark the smoke test live, probe the daemon inside the
test body (never at collection time), and allow pinning the image reference
via DEER_FLOW_SANDBOX_SMOKE_IMAGE for a dedicated integration job.
* test/docs: isolate DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS in tests; add table row; split custom-image guidance
_clear_hardening_env now clears the new knob so a developer shell or .env
preset cannot flip the default-path tests. CONFIGURATION.md gains the table
row, and the custom-image guidance becomes its own paragraph with the
no-new-privileges scope stated correctly (it does not mitigate the retained
CAP_SETUID/SETGID risk).
* test(sandbox): make the live smoke test diagnosable
300s readiness budget (cold pull + cold start must not be conflated with
broken capabilities) and dump the container's last 40 log lines on failure
so the next live run tells us whether the capability set is incomplete
(chown/useradd/su errors) or the services are merely slow.
* test(ci): align the smoke test with the 60s provider deadline; add a dedicated live smoke workflow
Single-source the readiness deadline as SANDBOX_LOCAL_PROVIDER_READY_TIMEOUT
(used by both provider paths and the smoke test) so the validation cannot
drift from the production contract again. New sandbox-image-smoke.yml runs
the live test on a dedicated job, with the image reference pinnable via the
SANDBOX_SMOKE_IMAGE repository variable (digest resolved and recorded in the
job summary when falling back to :latest).
* test(sandbox): pull the failing program's own logs on smoke failure
supervisord only surfaces exit codes in docker logs; nginx's stderr lands in
files inside the container. Dump supervisor program logs, nginx -t, and the
nginx error log on failure so the next run names the exact broken line.
* ci(sandbox): export an immutable repo@digest reference for the smoke run
docker pull once on the runner platform, resolve RepoDigests[0], and pass
that immutable reference to the test via GITHUB_ENV — the recorded and
executed images can no longer diverge when the tag moves, and platform
selection is left to the daemon instead of jq over the manifest index.
* fix(sandbox): add DAC_OVERRIDE — the root nginx master writes gem-owned logs
The image's root nginx master opens /var/log/nginx/{access,error}.log,
which belong to the gem user, for the container's lifetime; without
CAP_DAC_OVERRIDE it dies with 'open() failed (13: Permission denied)' on
every start (FATAL under supervisord) and readiness never arrives. Four
capabilities now: CHOWN/SETUID/SETGID for the entrypoint handoff plus this
runtime log-write need.
* feat(mcp): map request-scoped secrets to HTTP/SSE headers
`user_auth` binds a credential to a configured DeerFlow user, so a caller
that picks the credential per request — a multi-tenant gateway, a per-run
API key, one shared MCP server fronting several environments — had to
register one MCP server entry per credential.
Add a declarative `mcpServers.<server>.headers_from_context` block mapping
HTTP header names to keys of the run request's `config.context.secrets`
carrier. A new built-in interceptor resolves the mapping on every tool call
and rewrites those headers, mirroring `user_scoped_auth`. The config file
stores names only, never a credential, so the Gateway returns the block
unmasked.
Registered after OAuth and `user_auth` in the interceptor chain: the later
interceptor runs closer to the transport, and the value chosen for this one
request is the most specific, so it wins. Fail-closed by default — a mapped
key missing from the request raises a `ToolException` naming only that key,
because falling back to the server's discovery credential would send one
tenant's call under another tenant's authority. `on_missing: "passthrough"`
opts out.
Durable background tasks are excluded: `McpTaskToolCaller` drives status and
cancel polls after the Agent run ends, where no run context exists, so the
fail-closed interceptor would deny every poll. Those calls keep using
server-level credentials, and a server declaring both `headers_from_context`
and `task_toolsets` now logs a warning.
Also corrects the custom-interceptor example in docs/MCP_SERVER.md (and the
matching claim in skills/AGENTS.md), which read request secrets from
`langgraph.config.get_config()["context"]`. That key is `None` inside a tool
call — the run context rides the LangGraph runtime, not the RunnableConfig
propagated to child runnables — so interceptors written from that example
never saw a value. The example now reads `request.runtime`, and
tests/test_mcp_context_headers.py pins LangGraph's runtime-injection rule by
driving a real langchain-mcp-adapters tool through a real graph with the
ambient-runtime fallback disabled.
Closes#5005
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): resolve credential headers case-insensitively, carry them on durable submit
Review follow-ups on `headers_from_context`.
HTTP field names are case-insensitive, but every dict on the path to the wire
is not: `build_server_params` copies the operator's static `headers` spelling
verbatim, and langchain-mcp-adapters merges interceptor overrides into the
connection with a plain `{**connection_headers, **override_headers}` splat. A
static `authorization` and an injected `Authorization` therefore both reached
httpx as separate field lines, and a server reading the field with a
single-value accessor got the static discovery credential — inverting the
documented `headers` < `oauth` < `user_auth` < `headers_from_context`
precedence and running a per-request call under the shared credential.
Normalizing inside the interceptor cannot fix that on its own: the adapter
builds the request with `headers=None`, so an interceptor never sees the
connection's static headers and cannot displace them however it spells its own
key. A new `mcp/headers.py::apply_header_overrides` therefore drops any key
differing only in case and emits the spelling the connection already uses.
Applied to `headers_from_context`, `user_auth`, the OAuth interceptor, the
OAuth discovery-header write, and the durable-task connection merge, which all
carried the same collision. `headers_from_context.headers` now also rejects one
header mapped under two spellings at config load, in both the harness model and
the Gateway mirror.
Durable submit now carries the mapped headers, as docs/MCP_SERVER.md already
promised. `McpTaskToolCaller` disabled the interceptor for the whole caller, but
that caller serves submit as well as the polls, and submit is awaited inline
inside the Agent's tool call — where the run's LangGraph runtime is still the
ambient contextvar, so no secret has to be threaded through `TaskSubmitRequest`
or reach durable storage. The caller builds one chain and keeps a second view of
it without the context-headers interceptor; `call_tool` takes
`request_scoped_headers`, set only by `OrdinaryMcpTaskDriver.submit`. Status and
cancel keep server-level credentials, so background polls still cannot fail
closed, and the startup warning now describes the half it actually covers.
`_merge_preserving_secrets` restores masked extras inside `headers_from_context`
instead of writing the `***` sentinel back over the stored value, matching the
treatment `user_auth` extras and server-level extras already get; extras a PUT
omits carry over as well, while the declared mapping still replaces verbatim so
a round trip can remove an entry. `extra="allow"` plus name-based sensitivity
detection means the usual casualty is a name-valued key such as `tokenHeader`,
not only a credential.
The existing override test seeded the static header onto `request.headers`,
which production never does, so it modelled a merge that really happens one
layer down; the new tests drive a real adapter tool through a real connection
and assert on the headers the session is opened with, and the durable-submit
test runs through a real tool node with no runtime patching.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): reject case-insensitive duplicate static header names
* fix(mcp): preserve omitted headers_from_context fields on partial updates
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
ViewImageMiddleware injected the viewed-image message from before_model and
removed it again from after_model. before_model, model, and after_model are
separate graph nodes, so every view_image turn cost two extra nodes and two
state writes, and up to 20MB of base64 sat in two checkpoints for the duration
of the model call. A run interrupted in that window (user cancel, restart)
stranded the payload in history for good.
Inject from wrap_model_call instead, so the message lives only in
ModelRequest.messages and is never returned as a state update:
- before_model/after_model (and the async pair) are replaced by
wrap_model_call/awrap_model_call; _remove_image_context_messages and its
RemoveMessage bookkeeping go with them. The async hook keeps the existing
asyncio.to_thread offload for the file read and base64 encode.
- _should_inject_image_message gates on request.messages rather than state, so
the decision is made against what the model will actually see.
- _inject sweeps this middleware's own message out of the request before
rebuilding it. Dropping after_model also drops the cleanup it did on every
call, so without the sweep a payload stranded by an older interrupted run
would ride along in every later request for the life of the thread. Matching
requires both the reserved id prefix and the server-owned marker, and Gateway
strips that marker from client input, so a user message is never dropped.
Chain position is unchanged, and wrap_model_call nests first-registered
outermost, so TokenBudgetMiddleware still sees the image message and enforces
the input budget against it.
Checkpoint rows that already hold a stranded payload keep it on disk. It is
inert -- never sent to a provider, and strip_data_url_image_blocks keeps it off
the wire -- and reclaiming it would mean keeping the node this change removes.
tests/test_view_image_middleware.py is rewritten around the new hook (43
tests): sync/async at unit and graph level, the stranded sweep, and the
client-message protection. Docs: middleware chain entry 23, Vision Support, the
middleware-execution-flow hook matrix and diagrams, and the
strip_data_url_image_blocks docstring.
Adds scheduler.recursion_limit to config.yaml (default 1000, clamped by
max_recursion_limit) so scheduled background runs can use a different
recursion limit than the web UI. The value is read at dispatch time, so
a YAML edit applies to the next scheduled run without a Gateway restart.
Also logs a warning when the resolver falls back to the default or
clamps the configured value.
* feat: integrate MiniMax Code as an ACP agent
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* feat(mcp): add durable task runtime foundation
* fix(chart): sync embedded config version
* fix(mcp): isolate task polls during shutdown
* feat(mcp): track consecutive poll errors on mcp_tasks
poll_attempt_count grows on every claim (successful polls included), so it
cannot drive a failure backoff without misjudging normal long tasks. Add
consecutive_poll_error_count: incremented when a claim is released after a
poll error, reset to zero by any applied snapshot. The backoff/terminal
policy that consumes it lands with the first concrete driver.
* fix(mcp): harden durable task lifecycle
* feat(mcp): add ordinary durable task driver
* test(mcp): address durable task review feedback
* fix(mcp): preserve submit tool descriptions
* fix(mcp): bound remote task calls
* fix(mcp): bound persisted task payloads
* fix(mcp): preserve task tool error details
* fix(mcp): enforce durable task boundaries
* test(mcp): cover task config snapshot lifecycle
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: bound MCP server bring-up timeouts and exclude externalized tool outputs from delivery verification
Two related robustness fixes:
1. MCP server bring-up was unbounded. tool_call_timeout only covered
session.call_tool(); tool discovery (subprocess spawn + initialize +
tools/list) and persistent stdio session initialization could hang
forever, blocking agent construction (and on the Gateway event loop,
the whole process). Add a per-server session_init_timeout
(default DEFAULT_MCP_SESSION_INIT_TIMEOUT = 60s, null disables) that
bounds both discovery and pooled-session initialization. The session
pool's existing cancellation handling tears down a session stuck
mid-creation in its own task.
2. ToolOutputBudgetMiddleware externalizes oversized tool outputs into
outputs/.tool-results/ (configurable tool_output.storage_subdir). The
workspace-change scanner and run delivery verification counted those
files as produced artifacts, so any run that externalized a tool output
without also presenting a real artifact failed with
"Artifact delivery incomplete". Exclude TOOL_RESULTS_DIRNAME via a
shared constant (mirroring BROWSER_FRAMES_DIRNAME) and thread the
configured storage_subdir through snapshot capture so both
workspace-changes events and delivery verification stay clean.
* review: enforce single-segment tool_output.storage_subdir; document discovery-timeout cleanup
Address review feedback:
1. A custom tool_output.storage_subdir with a path separator (e.g.
cache/tool-results) silently no-oped the workspace-scanner exclusion:
os.walk yields one-segment dirnames, so a nested value never matched and
its files were counted as produced artifacts again. ToolOutputConfig now
validates storage_subdir as a single directory name (rejects separators,
.., absolute, empty) with tests, so the exclusion is always sound.
2. The discovery-timeout path now documents why cancellation is safe, mirroring
the session-init note: discovery runs inside the adapter's nested async
context managers, and stdio_client's finally terminates the process tree
(SIGTERM->SIGKILL on POSIX, process-tree on Windows), so a timed-out npx
subprocess and its children are reaped rather than accumulating.
* review: log session-init timeouts and align API response model default with runtime config
Address second-round review feedback:
1. A session-init timeout raised TimeoutError without any log, unlike the
discovery timeout which logs a WARNING. Wrap the bounded get_session in a
try/except that logs the timeout (server name + seconds) and re-raises, so
operators can diagnose tool-call failures caused by hung MCP sessions.
2. McpServerConfigResponse.session_init_timeout defaulted to None while
McpServerConfig defaults to 60s: a server created via PUT /api/mcp/config
without the field was persisted with null (no timeout) while the same
server created in the config file got 60s. Align the response-model default
to DEFAULT_MCP_SESSION_INIT_TIMEOUT so API-created and file-created servers
behave the same; an explicit null still opts out.
* review: narrow the discovery-timeout handler to the bounded wait_for path
The except TimeoutError clause covered both the bounded wait_for branch and
the bare discovery branch. With session_init_timeout opted out (None), a
TimeoutError raised by discovery itself would hit the %.1f format with None:
logging raises TypeError internally, the WARNING is silently dropped, and a
--- Logging error --- traceback goes to stderr.
Narrow the handler to wrap only the wait_for call, where the branch condition
guarantees the timeout value is not None. A discovery-internal TimeoutError on
the opted-out path now falls through to the generic failure handler and is
reported as 'tool discovery failed' with exc_info. Covered by a regression
test that asserts the skip is reported without any broken format.
* feat(channels): add Buzz (Nostr) channel connector
Adds a Buzz (https://github.com/block/buzz) channel so DeerFlow can join a
Nostr-relay workspace as a member: it answers @mentions in channels, replies
to DMs, and streams answers by editing one message in place.
* app/channels/buzz_nostr.py — pure NIP-01 helpers: canonical event ids,
BIP-340 signing/verification, chat/edit/auth builders, relay frames.
* app/channels/buzz.py — BuzzChannel: one NIP-42-authenticated websocket,
channel discovery (kind 39000) with one subscription per channel, live
membership tracking (44100/44101), per-channel replay watermarks, and
replies posted once then edited in place (kind 40003).
* app/channels/buzz_run_policy.py — same-thread serialization, mirroring
the Feishu precedent.
Inbound is gated in order: signature verification, self-drop, /connect
bind-and-return, pubkey allowlist, then mention / DM / mention-free /
thread-follow. Off by default; needs the new optional `buzz` extra
(coincurve, lazily imported), which detect_uv_extras resolves from
channels.buzz.enabled the same way it already handles channels.discord.
Two relay behaviours drove the design and are worth knowing when reviewing:
a global {"kinds":[9]} subscription receives nothing from buzz-relay and a
multi-value "#h" filter receives nothing either, so one REQ per channel is
required; and a single global `since` cursor skips quiet channels, so
watermarks are per channel.
Signed-off-by: Ajay R <ajayr@formbuddy.com>
* fix(channels): only publish assistant messages from the IM stream
`_accumulate_stream_text` decided what streamed `messages-tuple` payloads
become displayable text by rejecting ONLY payloads whose `type` contained
"tool", so it published everything else. DeerFlow writes hidden model
context into the messages channel as ordinary messages -- memory recall and
the rewritten user turn as hidden HumanMessages (DynamicContextMiddleware),
the `<durable_context_data>` block as another (DurableContextMiddleware) --
and LangGraph fans those state writes out on the messages stream, so they
reached every streaming IM channel as the assistant's reply.
Proved live on a Buzz relay: the connector published a `<memory>` fact block
and, in another run, a verbatim echo of the user's own inbound message.
Affects Feishu, Telegram, WeCom and Buzz; worst on Buzz, where each update
is an immutable public Nostr event that a corrective edit cannot unpublish.
Invert the filter to an allowlist of assistant message types. Two new pure
helpers keep it testable:
- `_stream_payload_type` resolves the type from both shapes the function
already handles: the `model_dump()` shape the gateway emits, and
LangChain's `to_json()` constructor shape whose own `type` is the literal
"constructor" and whose class name is the tail of the `id` path.
- `_is_assistant_stream_type` matches "ai"/"assistant" by PREFIX, not
substring -- ordinary words contain "ai" ("chain", "domain"), and a
substring test would admit a foreign type name by accident.
The bare-`str` branch is removed: an untyped payload cannot be attributed to
the assistant, nothing in DeerFlow produces one (serialize_messages_tuple
always emits `[message_dict, metadata]`), and a runtime that emitted raw text
deltas would emit hidden context the same way. Per-message-id buffering and
merging are unchanged.
Tests pin both directions, including multi-chunk merging across one message
id, so the allowlist cannot silently kill streaming, plus an end-to-end
`_handle_streaming_chat` test asserting the live payload never reaches an
outbound message.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Ajay R <ajayr@formbuddy.com>
* chore(helm): bump config_version to 33 in chart values and README
config.example.yaml moved to 33 for the buzz channel block; the chart's
embedded config example and its README copy track it (config_version only
drives the outdated-config warning, per scripts/check_config_version.sh).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Ajay R <ajayr@formbuddy.com>
---------
Signed-off-by: Ajay R <ajayr@formbuddy.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
