* fix(projects): check the project before staging shelf uploads
The shelf upload and save-from-thread routes staged the body under
users/{user_id}/projects/{project_id}/ before checking the project. A
missing, foreign, or archived id left an empty {project_id}/documents/
directory that the retention sweep keeps forever, and an id longer than
one path component failed with ENAMETOOLONG (500).
Both routes now require an owned, active project first and answer the
same 404 without writing. The shelf insert still re-checks under the
project row lock.
* test(projects): pin the shelf insert's re-check behind the early project check
With the early check in place, no route test reached the insert's
row-lock re-check any more. Archive the project between the check and
the insert and assert both routes still answer 404 and leave no staging
file.
* docs(changelog): reference #7040 for the shelf upload project check
* docs(changelog): drop the whitespace-only line after the zh #7040 entry
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The official DeepSeek Chat Completions endpoint rejects the generic
"minimal" effort token (issue #4514). The declared effort contract for
DeepSeek models now advertises the provider's real vocabulary
(low/medium/high/max/xhigh), defaults to "low", and aliases the generic
"minimal" preset onto "low" so the "flash" dialogue mode serializes a
value the endpoint accepts.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): exempt awk field arithmetic from the shell-variable path guard
Since #6930 every `$NAME/` in a host-bash command was treated as a
variable-prefixed host path unless both sides were digits, so legitimate
awk programs such as `awk '{print $NF/2}' data.txt` or
`awk '{print $sum/$n}' data.txt` on files under /mnt/user-data were
rejected with "Unsafe absolute paths".
Exempt a `$` only inside the single-quoted inline program of awk, gawk,
mawk or nawk (after `-F`, `-v` and `--`), where bash expands nothing and
awk reads `$NAME` as a field reference. awk string literals, option
values, input operands, double-quoted programs, text re-read by `sh -c`
or `eval`, heredocs, awk functions/aliases, unknown options such as
gawk's `--assign`, `-f` programs, and programs whose `"` may sit in a
regex or comment keep the strict scan.
Fixesbytedance/deer-flow#7021
* fix(sandbox): read awk string literals and comments before judging quotes
Review follow-up: the quote-reliability check gave up the awk exemption
for any `/` not preceded by an identifier, number or `]`, including a `/`
inside a plain string literal, so `awk '{n=split($0,p,"/"); print
$1/p[n]}'` was still rejected. Walk the program once with the same
string-literal tracker that computes the code spans: `/` and `#` inside
a literal are text, `#` outside one starts a comment to end of line, and
only a `/` outside a literal that cannot be division (regex literal)
forfeits the exemption.
* fix(sandbox): scan the awk exemption the way the shell parses the command
Review follow-up for the awk field-arithmetic exemption.
Bypasses closed: a backslash-newline let `aw\<nl>k() { eval "$1"; }`
define an awk function past the literal blocker regex, so any
continuation now forfeits the exemption; the word scanner emitted `)`,
`$(` and `>|` as command separators, so an `awk` that was only an
argument (`echo $(true) awk '...'`) looked like a command; `$'...'`,
comments and quotes nested in `"$(...)"` could desynchronise quote
pairing; the alias/function blocker only matched literal spellings and
ignored eval, source, hash, PATH, autoload and zsh's
`functions[awk]=` / `aliases[awk]=` arrays.
The scanner now follows bash: single and double quotes with escapes,
`#` comments at a word start, fd redirections, the usual separators,
and `$(...)` as a nested command context with fresh quoting. Anything
it does not model (backslash-newline, backticks, `$'...'`, heredocs, an
unmatched `)`, unbalanced quotes) returns no exemption. The command
word is found like `_validate_local_bash_shell_tokens` finds it:
after assignments, prefix keywords and wrappers, reduced to its
basename, inside every `$(...)`. A command that could redefine awk or
re-read quoted text gets no exemption at all, and the docstring says
the enumeration is best-effort.
False positives removed: `($2+$3)/2` and `length($0)/2` (a `/` after
`)` is division unless the `)` closes an if/while/for condition, and
the absolute-path scan is now exempt inside awk code too),
`/usr/bin/awk`, `LC_ALL=C awk`, `echo "$(awk ...)"`, `"<<"` inside an
awk string. The awk spans are computed lazily, only when a `$NAME/` or
absolute-path match needs them, and the division check scans backward
instead of re-slicing the program.
* fix(sandbox): exempt only single-segment absolute-path matches in awk code
Review follow-up. Inside awk code spans the absolute-path scan now skips
only a match with no second `/` (`/2`, `/n`, `/NR`), so a `/a/b` shape
is still rejected even if a span were ever misjudged; previously
`awk '{print ($1)/etc/passwd}'` and `awk 'BEGIN{x = 1 /etc/shadow}'`
were accepted.
`enable` joins the commands that withhold the exemption, since
`enable -f` can load a builtin under any name. The division check reads
the preceding `)` through the backward scan instead of re-slicing the
program for every `/`.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(mcp): fail durable tasks whose connection no longer resolves
A personal MCP runtime name is derived from the server's complete definition, and a deployment task server belongs to the Gateway startup snapshot. Editing, disabling, or deleting a personal server while a durable task is in flight therefore makes the persisted server name unresolvable. McpTaskToolCaller raised LookupError, which the service classifies as transient: status polls and cancellation retried forever, leaving the task stuck in working and impossible to cancel.
Raise a dedicated McpTaskConnectionUnavailableError instead, and terminalize it: status polls persist failed with the connection error and stop; a pending cancellation persists failed locally because the remote task can no longer be reached. Transient status/cancel errors keep their existing capped-backoff retry policy.
Tests: new service tests pin both terminal paths, and caller tests pin the exception for missing deployment and stale personal server names.
* fix(mcp): preserve LookupError compatibility for connection errors
McpTaskConnectionUnavailableError now subclasses LookupError instead of RuntimeError. The four existing personal-MCP assertions in test_personal_mcp.py still treat a missing or stale connection as a lookup failure; keeping that base preserves their behavior and avoids breaking callers that caught the previous type, while the task service still catches this exact type for permanent-failure classification.
Add a compatibility test and rerun the personal-MCP and task suites.
---------
Co-authored-by: Tong-bit-art <Tong-bit-art@users.noreply.github.com>
* feat: support client context capabilities
(cherry picked from commit b7c223f9415dcd545cbe05a5f5a6e47073a42d68)
[Totoro-qaq: resolved onto main. The persisted-reminder middleware hunks and their tests are dropped (delivery is redesigned in a follow-up); the RunCreateRequest description moved to app/gateway/run_models.py.]
* fix: address client context review comments
(cherry picked from commit bce6da6b819228eda58e887872720b6f9b5fab69)
[Totoro-qaq: middleware hunks dropped with the redesigned delivery.]
* feat(agents): deliver client context as a request-scoped data block
The cherry-picked #3106 put context.client into the persisted first-turn
reminder, with change detection and a "not provided" clear marker. Main
has since moved the date reminder to a SystemMessage, made request-scoped
context (the project block, #5443) request-only, and agents/AGENTS.md now
requires user-influenced values to ride the HumanMessage data channel,
never system text.
Render runtime.context["client"] on every model call in
DynamicContextMiddleware.wrap_model_call/awrap_model_call as one hidden
HumanMessage right after the leading SystemMessages, where DurableContext
puts its data. It is never a state update and never flagged
dynamic_context_reminder, so nothing is checkpointed, summarized or
replayed, and a run without a client gets no block. An unchanged client
keeps the previous request as the next request's prefix.
The renderer sanitizes again (direct graph callers bypass the Gateway),
html-escapes values and frames them with frame_untrusted_text: the same
--- BEGIN/END USER INPUT --- boundary the user's own turn and the MCP task
notification get. The lead prompt treats everything outside those markers
as framework data, so unframed values would rank above the user's own
text; framed, context.client can say nothing the same caller could not
already say in input.messages. client_context joins _BLOCKED_TAG_NAMES so
a forged block in user input or a tool result is escaped, and one static,
value-free sentence in the lead prompt names the block as untrusted hints,
never instructions, permissions or identity. The now unused empty-context
marker is removed.
* fix(gateway): log dropped context.client fields without values
A run request is never rejected over context.client: unknown fields,
nested values and malformed entries are silently dropped, the same
leniency as the other context keys. That left no signal when a custom
client sent something the Gateway ignored (review on #3106).
Route both the config.context.client and the body context.client paths
through one helper that emits a debug line whenever sanitizing changed
the value. Like the malformed client_timezone branch, the line carries no
values; unlike client_timezone, client.name and preference strings are
free caller-controlled text, so they are never logged, not even
sanitized. A clean client, an explicit null or an absent key logs
nothing.
Tests also pin the config-only path (sanitized before the early return)
and the end-to-end start_run admission: the sanitized client reaches
config["context"] and never configurable.
* fix(runtime): bound numeric client context preferences
String preferences are capped at 200 characters, but integers were kept
at any size. A JSON body may carry integers of up to 4300 digits, so 32 of
them rendered a block of about 138,000 characters, re-sent on every model
call of the run. In-process callers could also pass an int past the str()
digit limit, which failed the model call, or a number subclass with its
own __str__.
Keep integers within JavaScript's Number.MAX_SAFE_INTEGER, the range a
browser client can send exactly, and coerce numbers to plain int and
float.
* docs(api): describe client context as advisory request-scoped hints
The cherry-picked section described a hidden reminder that no longer
exists. Document what a custom client can rely on: the allow-list and its
limits (strings, the integer range, and only the first 32 entries of each
map), debug-level logging of dropped fields, runtime-only forwarding, and
delivery as a per-request user-role block, framed like user input, rather
than system text or thread history.
Only top-level context.client is documented. An explicit
config.context.client is sanitized for the run as well, but the run record
keeps the caller's config as sent, so it is not offered as an alternative.
It also answers the #3106 review: capabilities and preferences are
advisory hints, not enforcement, which belongs outside the model; name is
informational and never for authorization; the same object should be
sent on every run to keep the cached prefix stable; subagents and
embedded DeerFlowClient runs do not receive it, and server-started runs
(schedules, IM channels, GitHub agents, MCP notifications, extensions
unless the plugin passes one) carry none.
* fix(context): declare client delivery and keep client out of configurable
The request-only <client_context> block is model-visible, so
DynamicContextMiddleware now declares its delivery and the sanitizer's
limits in release_policy_parameters(); runs before and after this change
no longer share a policy fingerprint.
build_run_config copies body.config['configurable'] verbatim and
configurable is persisted with checkpoints, so a client placed there is
dropped instead of being stored raw. Client context stays runtime-only.
Document context.client in the README and in the middleware guide.
* fix(context): apply the PII redaction policy to the client block
DynamicContextMiddleware adds the <client_context> message inside the
outer PiiRedactionMiddleware, and the hidden message is not re-sanitized,
so with pii_redaction enabled a preference such as an email address
reached the model verbatim while the same address in the user turn was
redacted. Pass the rendered block through the shared redact_text seam
with this middleware's app_config.pii_redaction, and declare the setting
in release_policy_parameters().
* fix(context): keep client hints out of stored run config
---------
Co-authored-by: Blair Leng <25675774+BlairLeng@users.noreply.github.com>
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Crawl4AI's client assumed a JSON 200 was an object and indexed it. A reverse
proxy, a version-skewed server or an error document delivered with HTTP 200
can return a list or a bare scalar instead, and then:
- `data.get(...)` raised "'list' object has no attribute 'get'", and the
generic handler echoed that Python text to the agent;
- a non-string `markdown` field reached `.strip()` and raised the same way
one level in (["# heading"], 123, {"text": ...}).
Both now report "Crawl4AI returned an unexpected response format" and log
the unexpected type. This matches the sibling providers' contract -- tavily's
`_response_objects` returns a format error for a non-object payload -- and
the same containment the search providers already apply.
Measured: 8 new parameterised cases fail on the base revision (5 for the
payload, 3 for the markdown field); 39 tests in the file, 122 across the
crawl4ai/browserless/sanitization neighbourhood, ruff check/format clean.
Each guard was mutation-checked separately.
* fix(infoquest): reject invalid extracted fetch content
* fix(infoquest): address fetch response review feedback
Preserve reader_result precedence so malformed primary content is reported instead of silently masked by a secondary field. Include request URLs in invalid/empty content diagnostics and reject non-object JSON before field selection.
* test(infoquest): respect URL redaction in fetch diagnostics
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skills): strip a leading BOM from the SKILL.md content handed to the model
* fix(agents): hash the same BOM-free skill content the middleware injects
* docs(skills): state the BOM mechanism as it is, and sync the encoding convention
* test(skills): name the real BOM mechanism in the activation test docstring
* fix(ci): satisfy lint checks for skill BOM handling
---------
Co-authored-by: Jay0130-a <Jay0130-a@users.noreply.github.com>
Co-authored-by: Jay0130-a <309071492+Jay0130-a@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(memory): filter honcho search by category instead of stamping it on unfiltered results
HonchoMemoryManager.search sent an identical unfiltered query whether or
not a category was requested, then stamped the requested category on
every hit, so memory_search(category=...) returned unfiltered results
attested to be that category (e.g. a work fact labeled "preference")
— the opposite of the contract, where category filters before the
top_k slice and only facts with that exact category are returned.
Send the category as a real server-side metadata filter (Honcho applies
"metadata" filters in its ranked query, before the limit) and label
each result from the record's own metadata, falling back to "memory"
as before. HonchoClient.search gains a keyword-only filters argument
that is only put on the wire when set, so unscoped requests are
byte-identical to before.
* docs(memory): note honcho category filtering requires stored metadata
* fix(memory): drop out-of-range extraction confidence instead of failing the batch
A model-emitted confidence outside [0, 1] (a percentage such as 95, or 1.5)
passed extraction normalization, which only rejected non-finite values. The
value then reached storage, whose own validation raises for out-of-range
confidence, so one malformed field failed the whole update: the extraction
watermark never advanced and the same paid call repeated on every turn, while
every valid fact, summary and removal in the batch was lost each cycle.
Treat the range like the other malformed shapes in
_normalize_memory_update_fact: drop only that item and let the rest of the
update apply. Read paths already clamp stored corrupt values
(_coerce_source_confidence), so no read-side change is needed.
* fix(memory): bound the confidence drop and make it observable (review)
Review follow-ups on the out-of-range confidence fix:
- range-check integer confidence before float(), so a model emitting 10**400
is dropped instead of raising OverflowError and failing the batch;
- log the raw value when a fact is dropped, so a systematically
percentage-scale model is visible instead of silently losing every fact;
- document that a batch which also proposes factsToRemove still fails closed
when any new fact is dropped (replacementFactIndex positions shift), and
pin that boundary with a test. Values are documented as dropped, never
rescaled.
* fix(sandbox): search long lines for literal grep patterns
* test(sandbox): cover literal long lines across providers
* fix(sandbox): bound case-insensitive literal grep work
Replace repeated-prefix regex searches with KMP over single-character Unicode case matches. Keep local long-line literal results and existing regex and preview limits. Add a subprocess-deadline regression for the review counterexample and case-semantics coverage.
* docs(sandbox): keep grep guidance within inherited size limit
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skill-creator): preserve recorded token counts in benchmarks
Read timing.json token measurements independently of grading duration and retain recorded zero values. Keep the existing character-count fallback for missing measurements and add benchmark/report regression coverage.
* fix(skillscan): distinguish token metrics from credentials
Recognize typed token counts, numeric benchmark summaries, and signed deltas so skill-creator documentation passes deterministic review. Preserve credential names, string values, duplicate-key detection, and cloud-token findings; add regression coverage and document the boundary.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(models): tolerate non-mapping when_thinking_enabled.extra_body on the reasoning-contract path
* docs: keep frontend guidance below the hard size limit
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(workflows): stop idle agent waits from consuming worker turns
While a script step waited on a child Agent run, the supervisor sent an
empty response frame to the QuickJS worker every 20 ms and each round
trip counted toward the 10,000-turn cap, so any workflow whose Agent
calls kept it waiting for about 209 s failed with "Worker turn limit"
regardless of `timeout_seconds`.
The worker now reports when its job queue has drained, and the
supervisor waits on the pending calls locally while the worker is idle,
still re-checking cancellation and faults every 20 ms, without
exchanging frames. `timeout_seconds` is the only wall-clock bound again;
scripts that spin the job queue still hit the turn cap. The cap is a
module constant so tests can lower it.
Fixes bytedance/deer-flow#7022
* fix(workflows): default a frame without idle to the polling path
Review follow-up: read `idle` with `frame.get("idle", False)` so an
outcome-less frame that lacks the key degrades to the previous poll
behaviour instead of failing the run with a KeyError, and give the
cancel-latency regression test a 3 s budget so store writes and process
reaping on a loaded CI runner cannot flake it.
* fix(workflows): detect worker exit during idle Agent waits
Check worker exit in the existing local poll and clean up pending child runs. Adapt the work-frame budget regression from PR #7025 while retaining precise idle detection.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(searxng): contain malformed search payloads instead of crashing on them
`_search_page` indexed the decoded body directly:
data = resp.json()
return data.get("results") or []
so a self-hosted instance that answers HTTP 200 with something other than a
result page -- an HTML or plain-text error page from a reverse proxy or captive
portal -- reached the model as a raw Python error, or as an empty search:
{"results": "maintenance window"} -> {"error": "'str' object has no attribute 'get'"}
{"results": {"a": 1}} -> {"error": "'str' object has no attribute 'get'"}
{"results": [1, 2, 3]} -> {"error": "'int' object has no attribute 'get'"}
{"results": [None]} -> {"error": "'NoneType' object has no attribute 'get'"}
Worse, a single malformed entry in an otherwise valid page crashed the whole
call and discarded every good row:
{"results": [{"title": "ok", ...}, 7]} -> {"error": "'int' object has no attribute 'get'"}
This is the containment rule the sibling providers already follow (brave #6550,
groundroute #6603, tavily #6645, sofya #6719, serper #6779, serply #6796,
tencent-wsa #6799): a payload that is not a result page is a format error, not an
empty result, because "No results found" tells the agent the query legitimately
matched nothing and ends a research loop as "no hits".
The client now raises `SearxngResponseError` for a payload that is not an object,
a body that is not JSON, a `results` container that is not a list, and a
non-empty list holding no object entries, while still skipping non-object entries
in an otherwise valid page so the valid rows survive. `web_search_tool` reports
the provider's format error instead of leaking the exception text. Missing, null,
and empty `results` keep the existing "No results found" behavior.
Diagnostics log the container field name and its type, or the absence of usable
objects -- never the query, credentials, or payload values.
* test(searxng): pin the mid-walk format-error trade-off and fix the docstring
The paginating client aborts the walk and discards already-collected rows when
a later page comes back malformed, which is a different outcome from the
single-request siblings. Pin that trade-off with a test so the behaviour is a
deliberate decision rather than an accident, and reword the
_unexpected_format_error docstring: it promises the query is not echoed, but
the sibling providers all echo it back on purpose and only withhold the
payload.
* test(searxng): log skipped entries and pin the dedicated format-error handler
Two follow-ups from review, both about making the family contract observable
and pinned:
- Skipped non-object entries now log their index ("Skipping malformed SearXNG
result entry at results[1]"), matching the merged InfoQuest sibling. SearXNG
is the provider where drift is most expected -- a self-hosted instance serves
whatever sits behind its reverse proxy -- and a search spans up to
_MAX_PAGES requests, so rows could otherwise be shed with no trace at all.
- The dedicated `except SearxngResponseError` branch had no test that could
fail: the exception is a ValueError whose message is already the format-error
string, so the generic handler returns byte-identical JSON. The new test
asserts the generic branch's log line is absent, which is what actually
distinguishes the two paths.
* docs(searxng): name the empty result it actually returns, and pin the dedicated except
Two low findings from the follow-up review:
- The README said missing/null/empty `results` keep the existing "No results
found" response. That response does not exist in this provider: all three
produce a JSON empty list, and the string appears only in comments
explaining what must NOT be reported. The family convention distinguishes
(Tavily: "`[]` for search, Error: No results found for fetch"; InfoQuest:
"keep the existing empty result"), so this now reads "keep the existing
empty result (`[]`)".
- The client-level `except SearxngResponseError: raise` was unpinned: deleting
it left the whole file green, because the tool-level test mocks the client
and the payload tests only assert the raise type. The observable regression
is the generic branch logging "An unexpected error occurred" for every
contained page, so the existing payload test now asserts that line is absent
from caplog.
Mutation check: removing the clause now fails 8 parameterised cases (was 33
green); with the clause in place the suite is 33 passed and 91 across the two
provider files, ruff check/format clean.
run_agent() only refreshed runs.model_name when the request named a model,
and it read the effective model from agent.metadata, which compiled
LangGraph graphs do not have. The default model (no model_name requested)
and fallback models were therefore never persisted. Use the effective
model that lead-agent assembly already resolves, and write only when it
differs from the stored value.
Supersedes #3110. Fixes#3109.
Co-authored-by: Yi Tang <6054101+yitang@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(channels): share encrypted runtime channel config across Gateway instances
IM bot credentials entered in the web UI lived in each host's plaintext
channels/runtime-config.json, and the router cached the merged config on
app.state for the life of the process. With several Gateway instances a
save or disconnect served by an instance that does not hold the
provider's leader lease never reached the transport holder: it kept the
old credentials until it next started the channel, and a disconnect
could leave the old bot connected.
- New shared table channel_runtime_configs (migration 0044): one row per
provider with the config block encrypted under DEER_FLOW_CREDENTIALS_KEY,
a plaintext disconnect tombstone, and a version bumped by every save.
- SqlChannelRuntimeConfigStore replaces the JSON file for sqlite/postgres
(memory keeps the file); it is synchronous on app_sync_sqlalchemy_url
because its callers already run in worker threads.
- The router re-reads the store on every request instead of caching the
merged config, and refuses (503) to save credentials without a key.
- The leader-lease supervisor compares the versions on each renewal tick
and restarts or stops the holder's transport when a provider's config
changed; a takeover reads a disconnect instead of reviving cached
credentials.
- The first start imports an existing runtime-config.json (rows already
in the table win) and deletes the plaintext file.
* docs(changelog): reference #6988 for the shared runtime channel config
* ci: rerun checks after #6999 fixed the frontend guidance budget on main
* fix(channels): start a held transport that is down when its runtime config changes
When the lease holder's transport was down (a start that failed on the
old credentials), applying a peer-saved config only updated the cached
config, and the transport started on the next lease round: two renewal
ticks instead of the one the docs promise. The apply step now starts it
on the held lease right away, with the lease round's backoff bookkeeping.
* fix(browser): never replace an existing output file with browser_screenshot
browser_screenshot published straight onto outputs/<name> through the
replacing atomic publisher, so a screenshot saved as report.png silently
replaced the user's own report.png, an earlier capture, or another
default-named capture taken in the same second (the default stem has
second resolution). The sibling web_capture tool already keeps existing
files and takes a -1, -2, ... suffix.
Stage the complete capture under a hidden sibling and hard-link it onto
the first free candidate name. The link is the whole no-overwrite guard:
it fails with FileExistsError when the name exists as anything, so no
name is checked before it is claimed and a concurrent writer cannot be
replaced; the final name never shows a partial file. Staging shares the
outputs directory, so the link is same-filesystem, and like Gateway
uploads other link failures propagate instead of falling back to
replacement. New files keep the publisher's normal creation mode.
Hidden per-action progress frames keep the replacing _write_screenshot:
their names carry microsecond timestamps in a server-owned directory.
* docs(changelog): reference #6913 for the browser_screenshot no-overwrite fix
* fix(browser): refuse a staged screenshot swapped before it is linked
The sandbox can write outputs/, so it can see the hidden staged capture
and replace it with a symlink before the hard link. os.link follows
symlinks by default, which would publish a hard link to the host file as
the screenshot artifact (the replacing publisher this PR moved away from
never followed one).
Stage the capture with an exclusively created descriptor and keep its
identity from fstat, link without following symlinks where the platform
supports it (decided once at import), and require the published name to
be that inode; anything else is removed and the capture fails, like
Gateway uploads' identity check. A failed staging write still leaves no
files behind.
* docs(readme): note that browser_screenshot never replaces an output file
* fix(browser): keep the staged screenshot open until its link is verified
Linux filesystems reuse a freed inode number at once. Closing the staged
capture before linking let a file swapped in at the staged name take the
same number and pass the samestat check, so the symlink and planted-file
swap tests failed on Linux CI while passing on macOS. Hold the staged
descriptor through the link and identity check, as Gateway uploads do,
and require the published name to be a regular file.
* test(browser): race the peer claim against the real publication step
The claimed-name test patched _publish_bytes_atomically, which the
screenshot path no longer calls, so its hook never fired and the race
assertions passed vacuously. Plant the peer file right before os.link or
os.replace lands bytes on a visible name, and assert that the claim
happened, the capture moved to report-1.png, and the peer file survived.
* refactor(capture): share one output-name candidate sequence
browser_screenshot hand-copied web_capture's probe bound and suffix
order, and the copies had already drifted: Browserless retried a held
timestamped name five times, the screenshot tool once. Move the bound,
the retry count and the candidate order into utils/output_names.py and
have both tools iterate it, so the naming cannot drift again. Browserless
behavior is unchanged; its tests now patch the shared module.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(tencent_wsa): fall back when digit-class max_results cannot parse
str.isdigit() accepts values like "²" and "1²" that int() rejects, so a
configured max_results could crash web_search_tool before the HTTP call.
Parse strings with int() inside try/except (matching sibling providers)
while still rejecting bools and non-integral floats.
Fixes#6921
* docs(tencent_wsa): clarify that all float max_results fall back to default
- test_local_dev_loopback_bind: the stale-render test compared Git Bash's
MSYS path (/tmp/...) against a native path; route it through
_normalize_rendered_path like its sibling tests.
- test_project_documents_repo: TestShelfSnapshot relied on back-to-back
inserts getting increasing updated_at, which ties on Windows' ~15.6 ms
clock and falls through to the id tie-break. Pin updated_at explicitly.
- test_multi_worker_run_ownership: the transient-renewal test compared two
immediate lease timestamps with a strict >; start from a deadline inside
the lease window, as test_scheduled_goal_handoff already does.
* fix(infoquest): screen result URLs through the shared public-URL owner
Three providers already delegate result-URL screening to
community/url_safety.safe_public_result_url; infoquest registers the
image_search tool too but imported the owner zero times, so organic,
news and image rows all handed raw provider URLs to the model. A row the
owner rejects is now dropped rather than raised, keeping the containment
rule #6932 established for this client.
tests: test_infoquest_result_url_safety.py (new, red on main),
test_image_result_url_safety.py (infoquest added to the pinned modules).
* fix(infoquest): reject unscreened search response fallbacks
---------
Co-authored-by: ZHOU YANG <236804849+Zhouyang66639@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(streaming): retire memory buffers and wake existing subscribers
* fix(streaming): preserve end for fully consumed retired streams
* docs: keep frontend guidance within its hard budget
* fix(streaming): report gaps when retirement wins initial attachment
Cancel event and progress writes before force-close waits, bound progress settlement, and supervise cancellation-resistant reporters until their late result is observed.
Fixes#6986
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* fix(tests): pass Windows base system env to isolated pytest child
The live-test policy regression tests spawn a pytest child with a minimal,
from-scratch env. On Windows that env lacks SystemRoot/TEMP/USERPROFILE, so
the child dies during import asyncio (WinError 10106, winsock provider init)
and tempfile cannot resolve a directory, making the tests fail for
environment reasons instead of testing the opt-in policy.
Extract the env assembly into _child_pytest_env() and pass through a small
allowlist of Windows base system variables (SystemRoot, windir, COMSPEC,
PATHEXT, TEMP, TMP, USERPROFILE) when present. These carry no credentials,
so the isolation contract (no API keys, no real .env) is unchanged, and
POSIX behavior is byte-identical. A new unit test pins the contract: only
the allowlisted base vars may pass through, and SystemRoot must be present
on Windows.
* test(live-policy): seed Windows base env vars in the bootability contract
The contract test asserted TEMP/TMP pass-through without seeding them, so
on Windows hosts that do not export those optional variables (CPython then
falls back to %USERPROFILE%\AppData\Local\Temp) it failed on host
configuration rather than on the helper's contract.
Seed the representative base variables with monkeypatch and assert the
forwarded set exactly, so the isolation contract is deterministic on every
host. Red/green evidence: with TEMP/TMP stripped from the environment the
old assertion fails (extra = {COMSPEC, PATHEXT, SystemRoot, USERPROFILE,
windir}) while the seeded assertion passes.
Addresses the P3 finding in the review of PR #6439.
* ci(tests): cover Windows live-test policy
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skills): let skill_manage lock waiters wake across event loops
The sync tool wrapper runs every call under its own asyncio.run, and
ToolNode runs same-turn calls on separate threads. The per-(user, skill)
asyncio.Lock bound itself to the first contending loop, so a release from
another thread never woke that waiter, and a third caller failed with
"bound to a different event loop".
Reuse the read-before-write gate lock: each waiter parks on its own
loop without holding an executor thread, and release wakes every waiter
to re-contend, so a cancelled waiter cannot strand the skill. Keep the
gate's waiters in arrival order, so same-turn calls on one skill still
take the lock in the order they reached it, as with asyncio.Lock. Guard
the registry lookup, which the sync path runs on several threads.
Fixes#5846
* fix(skills): hand the skill lock to the oldest waiter
Reusing the read-before-write gate let a later same-skill call overtake
one already waiting. Release unlocked the gate and woke every waiter, so
a call whose storage lookup finished in that same loop step took the
free lock first. asyncio.Lock on main never lets a new caller pass a
queued one.
Give skill_manage its own small lock. Release hands the lock straight to
the oldest waiter and keeps it locked, so a later caller queues behind
it. Each waiter still parks on its own future and wakes on its own loop,
so waiting holds no executor thread. Claiming the future settles a race
with its cancellation, and a waiter cancelled after the handoff passes
the lock on. Releasing an unlocked skill lock raises, as with
asyncio.Lock. read_before_write_middleware.py is back to main.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* fix(mcp): consume resolved notification snapshots instead of launching empty events
A tracking-degraded snapshot recorded by a failed poll can resolve before any
notification worker claims it: the next poll succeeds, or an answered
input_required prompt returns the task to working. _record_event_if_changed
left that pending snapshot in place when the event disappeared, so
claim_notification_work rebuilt it as None and still claimed it, and the
service launched an empty notification run (dispatch_event or {}).
Consume such snapshots at claim time instead: advance notified_version, return
the status to none, clear the claim, and skip the row. The same guard consumes
rows left without a dispatch event by pre-fix claims, which would otherwise
relaunch empty notifications on every retry until dead-lettering.
* fix(mcp): clear consumed snapshot state for the review follow-ups
Review feedback on the claim-time consume guard:
- clear event_fingerprint when consuming, so an identical later attention
event is recorded again instead of being suppressed by a snapshot that was
never delivered;
- clear stale notification_error/attempt/dispatch state so a status
projection cannot show a notification that no longer exists;
- cover the reclaimed expired-claim shape (claim, lease expiry, resolve,
reclaim) and the identical-event sequence with regressions.
* docs(persistence): document the notification consume rule
mcp/AGENTS.md's effective chain has only 85 bytes of headroom on main (98,219 / 98,304), so the rule moves to persistence/AGENTS.md, which owns the mcp_tasks storage contract. No behavior change.
* fix(mcp): fence the notification consume against newer events (review)
Two P2 findings on the claim-time consume:
- the consume acknowledged the latest event version from a stale read, so on
SQLite a completion recorded after the claim's SELECT was overwritten with
status none and lost forever. The consume is now a conditional UPDATE fenced
on the event version and phase that were read; a newer event makes it a
no-op and stays claimable;
- a legacy launching/dispatched row whose dispatch payload is empty but whose
event version moved on consumed that newer event. Such rows are now rebuilt
and claimed; only a generation with nothing newer is consumed.
Regressions: a completion recorded while a claim is paused after its SELECT,
and legacy launching/dispatched rows with and without a newer completion.
* fix(sandbox): let local file tools read the /mnt/user-data root
The local provider maps the aggregate /mnt/user-data root so listing it
matches AIO, but the tool layer only accepted paths below it: ls, glob
and grep on the root answered "Permission denied" (with a trailing
slash, from the resolved-path "path traversal detected" check).
validate_local_tool_path now accepts the bare root for reads, and the
user-data resolver accepts the shared parent of workspace/uploads/
outputs itself when the caller opts in. Only ls and the shared read
path opt in; writes to the root and files placed directly in it stay
rejected.
* docs(changelog): reference #6991 for the /mnt/user-data root fix
* fix(sandbox): list the /mnt/user-data root through its allowed dirs only
The first version let glob, grep and ls start at the user-data root
after validating only the root itself. The walkers then descended into
everything under the host user-data/ directory, so a file or directory
stored directly under it was listed and searched although read_file
rejects it.
ls now lists the root one allowed directory at a time (workspace,
uploads and outputs, each one level deep) through the sandbox's own
list_dir, so the host root is never walked and list_dir's existing
per-directory containment check still filters symlinks. The resolver
is back to main's behavior, so glob, grep and read_file keep rejecting
the root.
