wait_for_sandbox_ready polls a sandbox's health endpoint until ready or a
timeout-second budget elapses, but measured that budget on the wall clock
(start_time = time.time(), while time.time() - start_time < timeout). A
backward wall-clock step (NTP correction, VM resume, manual change) while
polling keeps the loop running far past timeout -- and this runs on the
sandbox acquisition path, so it pins the acquiring thread instead of giving
up -- while a forward jump ends the wait early and destroys a sandbox that
was about to become ready.
The loop also only checked the deadline at the top and then blocked on a
fixed 5s request timeout plus a fixed 1s sleep, so a single slow probe could
overshoot timeout by up to ~6s; on the 5s acquisition path (local_backend)
that roughly doubles the budget.
Rework the wait to a monotonic deadline that clamps both the per-probe
request timeout and the inter-probe sleep to the remaining budget
(min(5.0, remaining) / min(1.0, remaining)), making the sync twin a faithful
mirror of wait_for_sandbox_ready_async, which already does this on
loop.time().
Add two deterministic regressions: one advances a scripted monotonic clock
past the deadline while a fake wall clock steps backward (red on the old
wall-clock loop), and one asserts the per-probe timeout and sleep are clamped
to the remaining budget (red on the old fixed 5s/1s values).
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* fix(view-image): bound re-injected image context with per-request budgets
`viewed_images` only grows (#6825): every model call that follows a
`view_image` round re-encodes and re-sends every entry ever viewed, so a
long PDF-inspection thread pushes the request body past provider limits
(e.g. Anthropic's 32 MB) even though each single image is capped.
Admission now runs newest-first against two knobs on ViewImageMiddleware:
- `max_injected_images` (default 10)
- `max_injected_total_chars` (default 32M characters, so a single
max-size image stays injectable: 20 MiB source -> ~27 MB base64)
Over-budget entries keep their text bullet with a placeholder ("call
view_image again to re-view this image") and are never read from disk or
sandbox, bounding both the request size and the middleware's per-call
work. Existing threads with small image counts and sizes are unaffected.
Refs #6825
* fix(view-image): make re-viewing an omitted image actually rescue it
The placeholder told the model to "call view_image again to re-view"
an over-budget image, but the recovery advice could not work:
merge_viewed_images merged with `{**existing, **new}`, and updating an
existing key in Python preserves its original insertion position. The
admission pass walks `reversed(viewed_images.items())` purely by
position, so the re-viewed entry stayed at its old (oldest) slot, was
admitted last, and was omitted again — a wasted tool round.
merge_viewed_images now re-inserts updated keys at the end, so dict
order tracks recency and a re-viewed image is admitted first on the
next injection. Also fix the `_create_image_details_message` docstring,
which said entries are "admitted oldest-first" while admission runs
newest-first (content blocks are emitted oldest-first; the admission
decision is not).
Wiring the budget knobs into config.yaml stays a follow-up: both
production instantiation sites construct ViewImageMiddleware() with no
arguments today, and this PR keeps the surface minimal.
Tests: reducer unit tests for re-insertion order plus an end-to-end
regression that re-views an omitted image and asserts it is admitted
on the next injection (#6829 review).
* fix(mcp): keep resolved credentials out of discovery failure logs
A failed per-server MCP discovery logged the raw exception message and traceback. ExtensionsConfig resolves $VAR values before validation, so a discovery error can embed a resolved-credential request URL (for example an httpx error carrying a URL with userinfo). Log only the server name and the exception type instead, at both the per-server and whole-phase handlers, and pin it with a caplog regression test using a simulated credential.
* test(mcp): assert the exact retained server name when redacting logs
The redaction regression test asserted a bare "A" in the captured log, which any unrelated A would satisfy. Assert the full "server 'A'" fragment so the test actually proves the offending server name is preserved while the credential is not.
* test(mcp): assert structured discovery warning arguments
The redaction change made the per-server discovery warning a parameterized log call, so asserting the server name inside the raw format string no longer matches. Assert the structured arguments instead: the server name, the exception type, the fully formatted message, and the absence of exc_info.
The LLM error-handling middleware's circuit breaker stored its recovery
deadline as time.time() + recovery_timeout_sec and fast-failed while
time.time() < _circuit_open_until. That is a duration deadline measured on
the wall clock, so a backward wall-clock step (NTP correction, VM resume,
manual change) while the circuit is open holds the fast-fail window open far
past recovery_timeout_sec -- shedding every LLM call until the clock catches
up -- while a forward jump ends it early.
Measure the recovery window with time.monotonic() instead, matching the rest
of the harness (task_tool deadlines, batch lease renewal, guardrail TTLs,
secret-file settle windows, journal latency). The clock is held on the
instance so tests can inject a deterministic one. The adjacent Retry-After
math stays on time.time(): it computes target.timestamp() - time.time() for
an absolute HTTP-date, which is genuinely wall-clock.
Add two regression tests (an injected-clock recovery test and a backward
wall-clock-step test) that go red without the fix, and update the two
existing recovery tests to drive the breaker through the same injectable
clock seam instead of patching time.time.
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* fix(loop-detection): let a warned agent save collected results before its final answer
The warn-level loop messages told the agent to "Stop calling tools and
produce your final answer now." Subagents read this literally: one whose
task names an output file stops all tool calls, including the single
write that delivers its result, and finishes as "completed" with nothing
on disk.
Only the two warn-level texts change. They now ask the agent to stop
repeating the looping call and wrap up, first saving what it has
collected through a file-writing tool if its task asks for a file. The
per-tool warning names the looping tool. Thresholds and the hard-stop
path (which strips tool calls) are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(loop-detection): bound final-save recovery after warnings
* docs(middleware): keep inherited guidance within CI budget
---------
Co-authored-by: huangzz <110116@nd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(llm-error): retry streamed 200-OK responses that carry no generation chunk
The non-streaming empty-generations glitch (IndexError) is already
classified as transient and retried. The streaming path reports the same
upstream glitch as ValueError("No generations found in stream.") from
langchain_core's generate_from_stream; it fell through to "generic" and
failed after one attempt regardless of llm_call.retry_max_attempts.
Classify that ValueError as transient so it uses the configured retry
budget. Unrelated ValueErrors stay generic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(llm-error): keep comments adjacent to their constants
---------
Co-authored-by: huangzz <110116@nd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(browser): publish captured screenshots and fetches atomically
_write_screenshot and _write_capture_output wrote the final path with a truncating
write and then returned its name, so a failure part-way through left a partial
capture that the caller reported as saved - and the leftovers then made the dedupe
helper pick a new name for the next capture. Publish through a temporary file plus
os.replace.
* fix(browser): preserve permissions on atomic capture publication
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): decode cmd.exe output with the ANSI code page under UTF-8 mode
The cmd.exe branch left its pipe decoder to the `encoding=None` default of
`_run_windows_command`, which follows Python text-mode semantics via
`locale.getpreferredencoding(False)`. Under Python UTF-8 mode
(`PYTHONUTF8=1`, `-X utf8`, the Windows installer's UTF-8 opt-in, and the
default once PEP 686 lands) that call reports `"utf-8"`, while cmd.exe and
the children it spawns keep writing the host ANSI code page - so every
non-ASCII byte in cmd output collapses to U+FFFD, with no fallback path.
Measured on a real zh-CN host under `PYTHONUTF8=1` (`getpreferredencoding`
-> `utf-8`, `getencoding` -> `cp936`):
- through the `encoding=None` default: `'��� �����Z
'`
- pinned to the ANSI code page: `'你好 日本語
'`
This branch pins the cmd.exe pipe decoder to `locale.getencoding()`
(3.11+, repo floor is 3.12), the same selection the MSYS and PowerShell
branches already use for their fallback (#6727, #6548).
## Scope note
The `encoding=None` default itself is deliberately unchanged: it mirrors
Python's own `text=True` semantics and is pinned by
`test_windows_capture_matches_subprocess_text_mode_encoding`. Only the
cmd.exe branch - whose output is known to be the ANSI code page - overrides
it, the same way the MSYS and PowerShell branches already pin their own
decoders.
Follow-up to willem-bd's review note on #6727 (inline comment 4247281255)
flagging this path.
## Testing
- new (POSIX-runnable, mocked): the cmd branch passes
`encoding=locale.getencoding()` to `_run_windows_command`, verified under
a simulated UTF-8-mode host (`getpreferredencoding` -> `utf-8`,
`getencoding` -> `cp936`).
- updated: `test_execute_command_uses_cmd_command_mode_on_windows` now also
pins the new `encoding` kwarg.
- new (Windows; skipped when the console output CP does not match the ANSI
page - sibling tests can change it process-wide): real `cmd.exe /c echo`
roundtrip through `execute_command` returns the text intact (passes on a
zh-CN host when run standalone).
- full `tests/test_local_sandbox_encoding.py`: 40 passed, 2 skipped under
both `PYTHONUTF8=0` and `PYTHONUTF8=1` (Windows 11, Python 3.13.12, repo
venv); the text-mode contract test stays green in all three parametrized
modes (`locale-code-page`, `PYTHONUTF8`, `-X-utf8`). Red run against
unmodified main: both new/updated cmd-branch tests fail (`KeyError:
'encoding'`).
- `ruff check` + `ruff format --check` clean on both files.
* fix(sandbox): use the cmd console output code page
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(runtime): preserve canonical delta-history order in cache
CachedHistorySaver composed a parent checkpoint's channel writes from
CheckpointTuple.pending_writes in storage order, but the saver's
authoritative get_delta_channel_history orders writes by (task_id, idx).
On the in-memory backend pending_writes follows task-arrival order, so a
fan-out super-step (several parallel tasks writing one channel in a single
checkpoint) was materialized in a different order with the cache enabled
than with the bare saver, breaking the cache's transparency invariant.
Sort a checkpoint's channel writes by task_id in _channel_writes, restoring
the canonical (task_id, idx) order while preserving each task's internal idx
sequence. The sort is a no-op on sqlite and postgres, whose pending_writes
already arrive ORDER BY task_id, idx.
Add async and sync regression tests over a real InMemorySaver seeded with a
fan-out checkpoint whose writes arrive out of task_id order; both diverge
from the inner saver's native walk without the fix and match with it.
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* docs(runtime): clarify cached saver ordering contract
---------
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): add channels.leader_lease and gate it under multi-process signals
IM channels other than GitHub connect outbound to their platform. With more
than one Gateway instance each one opens that connection with the same bot
credentials: Telegram rejects the second poller with 409, Discord answers every
message twice and per-process ChannelManager state diverges. The multi-process
startup gate said nothing about channels; the only guard was a comment in the
Helm values.
Add the `channels.leader_lease` section (`enabled: auto|true|false`,
`lease_seconds`, `grace_seconds`; startup-only, registered in the reload
boundary) that the per-provider leader lease will read, validated by a Pydantic
model from the non-schema `channels` mapping. The gate now refuses
`enabled: false` under a multi-process signal while a non-GitHub provider is
enabled under `channels.*` or connectable through `channel_connections`, naming
the providers and the rollback; a malformed block is refused on every topology.
`auto` follows the multi-process signal, so a single instance is unchanged.
ChannelService strips the block before its per-provider loops and exposes it
as `leader_lease`; `ensure_ready_channels` only visits registered providers.
The two-Gateway harness overlay leaves the block alone. config_version 60 in
the example, the Helm chart and its README; docs replace the comment-only
"keep 1 replica" guidance with the gate.
* docs(changelog): link #6746
* fix(channels): stop recommending the unimplemented leader lease as a remedy
Review on #6746: the refusal told operators to switch channels.leader_lease
from false to auto/true "so one instance owns each provider's connection", but
the transport arbitration ships separately, so that change only bypassed the
clause and every worker still started the same bot.
The refusal now offers only the remedies that work today (a single instance, or
disabling the channels) and states that auto/true are reserved for the lease
that ships separately. A regression helper in the gate tests asserts the
remedies start with the rollback and that auto/true are never offered. The
config example, Helm values/README, CONFIGURATION.md, README, the deployment
guide (en/zh), the model docstrings and both changelogs describe auto/true as
reserved settings rather than an ownership guarantee.
* fix(channels): warn on an enabled unknown channel key; reword the changelog entry
Review on #6746: the registry guard in ensure_ready_channels skipped a
misspelled provider key (channels.telegam) silently, where it previously
reached _start_channel and logged 'Unknown channel type'. An enabled dict
block that is neither a registered provider nor leader_lease now logs a
warning that lists the registry.
The changelog entry claimed the chart and docs drop the "keep 1 replica"
guidance; they keep it (auto/true do not make several replicas safe yet) and
only stop relying on the comment alone.
* fix(dev): disable UI channel providers in multi-instance overlay
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(memory): read the complete scope in the substring search fallback
_search_substring iterated list_facts without paginating, so the
adapter-less fallback only ever saw the default page of 100 facts.
With 150 matching facts in a scope (max_facts allows up to 500),
memory_search returned 100 results for top_k=150 and silently dropped
every match past the first page; DeerMem.search keeps these results
whenever they are non-empty, so its uncapped fallback never ran. Read
the complete scope instead, as the scoped index rebuild already does.
* test(memory): drop in-page fact-100 from the past-page assertion
fact-100 was the last id inside list_facts' old first page
(``matched[0:100]`` in insertion order), so its presence does not
distinguish the fixed search from the pre-fix page; fact-101 and
fact-150 (plus the ``len(results) == 150`` check) carry the regression
signal. Keep a boundary note so the remaining ids are not misread.
* fix(memory): keep non-core fact categories instead of rewriting to other
_normalize_category rewrote any category outside CORE_CATEGORIES to
"other" and parked the original label in a write-only
categoryExtension field, so memory_add(category="personal") -- an
example label in the tool's own docstring -- persisted as "other" and
memory_search(category="personal") ("Only facts with this exact category
are returned") never matched it. get_memory documents kept reporting
"other" as well.
Keep the caller's label as the stored category so it round-trips
through search, documents, and the retrieval index.
* fix(memory): finish migrations interrupted across the category change
A pre-native-category Gateway committed a per-agent legacy JSON migration
as category="other" + categoryExtension=<label> and could stop before
deleting the legacy JSON (crash / failed unlink). The retained JSON holds
the label natively, so _migrate_locked compares "personal" against its
own committed "other"/"personal" alias, raises MemoryStorageCorruption,
and fails every subsequent read, reload and migration retry for that
agent (HTTP 500 at the Gateway) -- a recoverable state the previous
build recovered from.
Fold the legacy alias into _migration_equivalent's category comparison
("other"/<label> equals <label>/no-extension) so retries recognize and
complete recovery, while a genuinely different label still stops the
migration with the legacy file preserved.
Also drop a stale categoryExtension when a fact is normalized away from
"other": _normalize_fact deep-copies unknown fields through, so a legacy
fact updated to its native category kept surfacing the leftover
extension, contradicting the Gateway contract that the field is only
set "when category is 'other'".
Both regressions are pinned by replaying the historical writer in tests.
---------
Co-authored-by: BlueX888 <BlueX888@users.noreply.github.com>
* docs(agents): trim restated rationale to restore AG002 chain headroom
The agents/middlewares AGENTS.md chain crossed the 96 KiB AG002 hard
limit at 3f7a0769 (98441 > 98304 bytes); that push produced no lint-check
run, and later pushes compare equal to their base, so main stayed green
while a full-worktree check_agent_guidance run reports an error. Trim
restated rationale:
- backend/AGENTS.md: the Feishu button-card sentence restated semantics
app/channels/AGENTS.md already owns (originator-validated, one-time,
fail-closed clicks); keep the flag visibility and point at the module
guide.
- agents/middlewares/AGENTS.md: drop the membership-probe rationale
duplicated by the model_response.py paragraph, the ToolOutputBudget
ordering narration, and the lead-only append note restated by the
lead-only middlewares heading.
Chain: 98441 -> 98198 bytes (106 under the hard limit).
* docs(channels): own the Feishu card fail-closed invariant in the module guide
Review follow-up on #6759: the backend/AGENTS.md trim dropped two things
app/channels/AGENTS.md item 7 does not own - the fail-closed-across-workers
guarantee and the owning file's name - so the invariant left every
instruction chain and the pointer dead-ended one level above the
implementation. State both in item 7, paid by dropping the therefore-clause
restated by the single-active-owner invariant above it (the partial unique
index and the losing-writer retry already imply deterministic resolution).
channels/AGENTS.md: 49134 -> 49129 bytes (AG001 headroom 18 -> 23).
---------
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(client): convert the captured upload source
* test(uploads): cover ACL sources and scope capture failures
* fix(uploads): preserve concurrent uploads from one captured source
Combine the embedded client's source capture with no-replace publication for
first-time filenames and Markdown companions. Hold one private capture across
collision retries, retain original descriptor metadata, and keep replacement
behavior for names present when the call starts.
Port concurrent-upload and staging-publication regressions from PR #6687.
Extend source-edit/removal and real XLSX cases to both publication paths, and
cover edits or removal during two filename collisions in ordinary and event-loop
calls. Document capture, retry, publication, and cleanup contracts.
Validation: related suites 469 passed, 24 skipped; full offline suite 28,788
passed, 488 skipped; blocking-I/O suite 245 passed, 1 skipped; Ruff, guidance,
and diff checks passed. Nine race/collision cases failed on the #6632 baseline.
Co-authored-by: haotianqi666 <250384991+haotianqi666@users.noreply.github.com>
* docs: keep agent guidance within CI budgets
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: haotianqi666 <250384991+haotianqi666@users.noreply.github.com>
* fix(sandbox): decode PowerShell native-child output with the locale fallback
Fixes#6646
## What
The Windows PowerShell branch of `LocalSandbox.execute_command` passed `encoding="utf-8"` to `_run_windows_command` with no `fallback_encoding`. The guarded console-UTF-8 preamble covers PowerShell's *own* output, but native children it runs (`python.exe`, CLI shims) inherit the pipe and keep writing the host code page, so the pipe decoder's `errors="replace"` turned every such byte into U+FFFD. This passes `fallback_encoding=locale.getpreferredencoding(False)` there, mirroring what the MSYS branch above already applies (#6548).
## Why
On Windows hosts whose ANSI code page is not UTF-8 (GBK on zh-CN hosts, cp932, cp949, cp1251, ...), non-ASCII tool results printed by native children reach the model garbled (#6646). PowerShell's own output stays correct because the preamble pins it to UTF-8, so the fallback only engages when a buffer is not valid UTF-8 — exactly the case where a native child wrote host-code-page bytes. Pure-ASCII and PowerShell-emitted UTF-8 buffers never touch the fallback, matching the tradeoff the MSYS branch already accepts.
## How checked
Reproduced on a real zh-CN Windows host (ANSI code page 936):
- Red on `main` (3f7a0769): a native `python.exe` child printing CJK text through `powershell.exe` / `pwsh.exe` returned `'\ufffd\ufffd\ufffd \ufffd\u0571\ufffd\ufffdZ\n'` — the new regression test failed on both shells:
```
FAILED tests/test_local_sandbox_encoding.py::test_windows_powershell_native_child_cjk_roundtrip[powershell.exe]
FAILED tests/test_local_sandbox_encoding.py::test_windows_powershell_native_child_cjk_roundtrip[pwsh.exe]
```
- Green with the fix: both pass, and the child's text round-trips exactly (`'\u4f60\u597d \u65e5\u672c\u8a9e\n'`).
- `tests/test_local_sandbox_encoding.py`: 34 passed, 1 skipped (Windows 11, CPython 3.13, `uv sync --frozen` environment). The two pre-existing PowerShell unit tests were extended so their `_run_windows_command` fakes accept the new `fallback_encoding` kwarg and assert it is the locale code page.
- `ruff check` and `ruff format --check` pass on both changed files.
* fix(sandbox): decode fallback must survive Python UTF-8 mode and mid-character captures
Follow-up to review feedback on #6727.
- Use ``locale.getencoding()`` (3.11+, repo floor is 3.12) instead of
``locale.getpreferredencoding(False)`` for both fallback call sites
(MSYS and the native-child guard): ``getpreferredencoding`` reports
``"utf-8"`` whenever the Gateway runs under Python UTF-8 mode
(``PYTHONUTF8=1``, ``-X utf8``), silently disabling the fallback and
leaving #6646 unfixed on those hosts. ``getencoding()`` deliberately
ignores UTF-8 mode and returns the code page the child's CRT used.
- After a failed strict UTF-8 decode, retry once without a trailing
incomplete multi-byte sequence, so a capture cut mid-character (the
10MB capture-cap boundary, or a timeout kill interrupting the child
mid-write) cannot cascade the whole buffer to the locale fallback.
Complete non-UTF-8 buffers still fail the retried attempt and reach
the fallback untouched.
- Tests: the Windows roundtrip probes now run under ``PYTHONUTF8=1``
with ``locale.getencoding`` pinned, so a green run proves the fallback
source ignores UTF-8 mode; added POSIX-runnable unit tests for the
cap-split and timeout-kill partial tails.
Test results (Windows host, ACP=936, repo venv via ``uv sync --frozen``):
``tests/test_local_sandbox_encoding.py`` 36 passed, 1 skipped, both with
``PYTHONUTF8=0`` and ``PYTHONUTF8=1``; ``ruff check`` and
``ruff format --check`` clean on both files.
* fix(sandbox): trim only on cut captures, preserving complete locale tails
Willem-bd's follow-up review pinned a real P2: on a cp1252 host the MSYS
branch receives b'caf\xe9' (a complete e-acute), the trailing E9 is trimmed
as an incomplete UTF-8 lead, and the ASCII prefix is accepted as valid UTF-8
- so the locale fallback never runs and the byte is silently dropped. No
capture cap or timeout is needed to trigger it.
Fix: the strict-retry trim is now authorized only when the capture may
actually have been cut mid-character - a byte-cap cut (signalled by the
capture's own truncated state) or a timeout kill (new mark_split_tail(),
called from _run_windows_command right after the process tree is killed). A
complete buffer, even one ending in a byte shaped like a bare UTF-8 lead,
now goes to the fallback with every byte intact.
Tests:
- new: cp1252 complete-tail roundtrip (red on b62ef75: returned 'caf').
- new: the same partial-shaped tail without a cut signal must fall back
intact instead of being accepted as a truncated UTF-8 prefix.
- new: POSIX-runnable _run_windows_command timeout integration test - the
child writes a partial UTF-8 sequence, hangs, is killed at the 1s
timeout; the trimmed prefix survives (this goes red if the kill path
forgets to authorize the trim, since the whole buffer cascades to the
fallback).
- updated: the timeout-kill unit test now marks the capture explicitly,
mirroring the real call site.
Evidence: Windows 11, Python 3.13.12 (repo venv), pytest
tests/test_local_sandbox_encoding.py -> 39 passed, 1 skipped under both
PYTHONUTF8=0 and PYTHONUTF8=1; ruff check + ruff format --check clean.
Red run against b62ef75: the three new/updated tests fail exactly as
described above.
* fix(sofya): report a format error for results with no usable objects
`_response_results` skipped non-object entries but returned the filtered
list even when it was empty, so a non-empty `results` array holding no
objects fell through to "No results found" -- telling the agent the query
legitimately matched nothing instead of surfacing the malformed payload.
Treat a non-empty list with no object entries as malformed, matching the
containment rule #6603 documents for this provider family (brave #6550,
groundroute #6603, tavily #6645, webz). Both `web_search` and
`web_fetch` share this helper, so both now report the provider's format
error. Missing, null, and empty `results` keep returning "No results
found", and mixed lists still keep their valid entries in order.
* test(sofya): cover mixed fetch result entries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(uploads): convert the same source version the upload saved (#6600)
DeerFlowClient.upload_files copied the caller's file into the uploads
directory, then reopened the original path for Markdown conversion. A
save landing between the two reads left the saved upload at version A
while the companion registered version B, and resolve_companion still
accepted the mismatched Markdown.
Snapshot convertible sources into a private temp dir up front and run
both the upload copy and the conversion from that one read. The snapshot
keeps the #5611 invariant (conversion never reopens a name inside the
sandbox-writable uploads dir), preserves metadata via copy2, and keeps
the re-upload SameFileError rejection through an explicit same-file
probe before the snapshot.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(uploads): pin the snapshot copy to one held source descriptor
Review on #6621 found two gaps in the snapshot step: stat-by-name then
copy2-by-name split identity from content across two path resolutions,
and copy2 drops ACLs so a source readable only via ACL made the private
snapshot unreadable. Open the source once, fstat it for the same-file
rejection, and copyfileobj out of the held fd; restore the mtime from
that same stat so the upload keeps the source's timestamps.
Tests: the version-parity regression's injection now also fires when a
future regression reads the caller's path during conversion, and a real
macOS ACL case (mode 000 + read grant) uploads and converts cleanly.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* test: catch upload conversion source regressions
---------
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skillscan): match fork bombs by shape, not spelling
`resource-fork-bomb` and the fork-bomb branch of `shell-destructive-command`
only matched the canonical `:(){ :|:& };:` rendering, so renamed or respaced
payloads produced no findings and passed the blocking gate.
Match the general shape instead - a shell function whose body recurses through
a pipe into a background job - keeping the pre-LLM static gate at least as
wide as the runtime sandbox audit middleware it backstops.
Closes#6651.
* fix(skillscan): bound the fork-bomb pattern and require self-recursion
Addresses the three review findings on #6689:
- Bound every quantifier in the shared fork-bomb pattern so a whitespace-free
blob costs O(n) instead of O(n^2) (measured ~0.15s at 200KB; the previous
spelling took ~5.5s at just 50KB).
- Require the function name to recur after the pipe (backreference), so a
benign pipeline such as `backup() { rsync -a src/ | logger & }` no longer
blocks as a CRITICAL finding.
- Keep both gates byte-identical again: skillscan's _FORK_BOMB_PATTERN and the
sandbox_audit_middleware fork-bomb entry, pinned by a parity test.
- The code comment now describes exactly what the pattern checks; pipe-less
self-backgrounding recursion and the spaced `|&` spelling are documented as
out of scope.
- New regression tests: near-miss pipelines in both suites, the parity
invariant, and a 200KB single-token scan-time ceiling.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(acp): resolve an absolute Windows agent command before spawning
A command configured as an absolute path without an extension -- npm's global
shim layout, e.g. C:\npm\mcode beside mcode.cmd -- was handed to the ACP SDK
unchanged and always failed: create_subprocess_exec applies no PATHEXT, so it
ran npm's extensionless shell script and died with WinError 193 (or raised
FileNotFoundError when only the .cmd exists). The separator guard from #6598
was too broad: only relative paths need to stay with the spawn, because their
meaning is owned by the ACP-workspace cwd, while an absolute path does not
depend on the cwd.
Absolute paths now go through the same shutil.which lookup as bare names: the
lookup searches the command's own directory and probes PATHEXT, and the
configured value is kept when nothing matches so the not-found remediation
still fires. Relative paths (bin/agent, ./bin/agent) still go to the spawn
unchanged.
Docs: README.md and backend/packages/harness/deerflow/tools/AGENTS.md now
describe the bare/absolute vs relative split.
AI disclosure: this change was drafted with AI assistance at my request, under
my direction and review.
* fix(acp): keep already-absolute lookup results unchanged
Review feedback on #6656 (willem-bd): os.path.abspath collapsed a `..` that
follows a symlink, so an absolute POSIX command like
`/srv/current/../shared/agent` (with `current -> releases/v1`) was rewritten to
`/srv/shared/agent` -- a path that either does not exist or names a different
launcher. The anchoring to the Gateway cwd exists only for relative PATH
lookup results, where shutil.which cannot know the cwd the spawn will use; an
already-absolute result needs no rewrite.
Regressions: a platform-independent stub-lookup test pinning the literal
parent-segment path, and a POSIX-only test that builds the symlink layout and
asserts the configured command is returned unchanged.
* fix(runtime): mark cancel takeovers with the orphan-recovery stop reason
A cross-worker cancel that takes over an expired-lease run writes error but
left stop_reason unset. `orphan_recovered` is the only signal that lets a
store-only SSE/`wait` consumer synthesize END after a heartbeat, because the
producer is known to be gone. Without it, a subscriber that outlives the
crashed owner waits forever for an END no surviving worker publishes; periodic
reconciliation ignores the row because it is already terminal.
Mark the takeover like periodic orphan recovery: claim_for_takeover already
accepts the stop reason and the reconcile path already passes it. The new
tests pin the takeover row's marker and prove a real takeover wakes a waiting
consumer; three existing takeover test doubles now forward the store argument.
* docs(gateway): keep guidance within the 48KB budget
The takeover stop-reason sentence pushed backend/app/gateway/AGENTS.md over its 48KB hard limit, failing agent-guidance and the backend guidance-budget test. Trim the sentence; the rationale stays in the manager.py comment.
* fix(gateway): complete streams after scheduler orphan recovery
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): stop resolving sandbox and ACP env values a second time
AppConfig.resolve_env_variables already replaces every $VAR when
config.yaml loads. The AIO, E2B and OpenSandbox providers and the ACP
tool then resolved the result again, so a secret that itself starts with
$ (a bcrypt hash, a generated password) was looked up as a variable name
and reached the sandbox or agent as "" -- or as an unrelated host
variable's value when that name existed.
Pass the loaded values through unchanged, as BoxLite and the local
provider already do. The removed helpers' str() coercion was dead: both
fields are typed dict[str, str] and reject non-strings at load.
* docs(changelog): reference #6673
* docs(config): pin when sandbox and ACP env values are resolved
ACPAgentConfig.env and the SandboxConfig docstring said $ values are
resolved from the host environment without saying when, which reads as
"at spawn time" and invites a consumer-side second pass again. Both now
say it happens when config is loaded, matching SandboxConfig.environment.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(lark): publish the skill file and manifest atomically
_write_skill_file and _write_manifest wrote their final paths with a truncating
write, while the runtime files in the same module already go through
_publish_bytes_atomically. A partial skill file or manifest is unparseable, and
the manifest carries the content hash the install is verified against, so a
half-written one cannot be repaired. Route both through the existing helper.
* fix(lark): address skill publication review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(agents): skip content blocks with unhashable type values
has_visible_content and ParentContextSnapshot.from_state classified a
content block's type with set membership (`in {"text", "output_text"}`
plus the _MEDIA_BLOCK_TYPES frozenset). `type: []` / `type: {}` are
legal JSON from a provider, and hashing an unhashable value raises
TypeError instead of skipping the block:
- has_visible_content runs on every model response through the
llm-error-handling (_raise_for_empty_response), terminal-response and
model-length middlewares, so one malformed block turned a completed
response into a failed run instead of a skipped block;
- from_state builds the history every background subagent dispatch is
seeded from, so the same block failed the dispatch.
Probe the type by equality with tuples so a non-string type simply fails
the probe, mirroring #6614's fix for extract_response_text and the
isinstance guard clarification already uses for form field types. The
media constant becomes a tuple too, so the second probe in the same loop
cannot move the crash one line down.
Tests pin the RED-to-GREEN cycle at both call sites: unit-level
has_visible_content, middleware-level empty-response classification,
and the dispatch snapshot across human/ai/tool messages.
Fixes#6663
Signed-off-by: yetuge <2219677952@qq.com>
* fix(agents): warn when a snapshot skips a non-string-typed block
Review follow-up on #6664: with the malformed block now skipped
instead of raising, a provider that systematically emits non-string
types would degrade the child's history with nothing in the logs to
explain it. The elif chain cannot log on fall-through —
reasoning/signature and tool-use exclusions are deliberate and must
stay quiet — so the signal keys on the type not being a str, matching
the guard the review suggested.
has_visible_content keeps its silence on purpose: the empty-response
retry already surfaces the symptom there.
Fixes#6663
Signed-off-by: yetuge <2219677952@qq.com>
---------
Signed-off-by: yetuge <2219677952@qq.com>
* fix(tavily): contain malformed web search and fetch payloads
web_search and web_fetch normalized the SDK's decoded 200 body without
validating its shape: a `results`/`failed_results` container of the wrong
type, a non-object entry, or a missing leaf field escaped as a raw
TypeError/KeyError/AttributeError instead of the structured format error
brave, serper, serply, sofya, tencent_wsa, unbrowse and groundroute return.
Validate the container with one shared helper, skip non-object entries while
preserving valid objects in order, normalize missing leaf fields to empty
text, and coerce a non-string raw_content in web_fetch before truncating.
Diagnostics carry only the malformed container's name and type, never the
query, URL, key or payload values.
backend/tests/test_tavily_response_shapes.py drives the real tools and SDK
over an offline HTTPX MockTransport; it fails 32 of 36 cases on main and
passes with this change, while test_tavily_tools.py stays green.
* docs(readme): note Tavily payload containment
Follow the Brave and GroundRoute README entries in the setup wizard step; the containment contract itself lives in community/tavily/AGENTS.md.
* fix(tavily): normalize null search fields and test mixed fetch lists
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
_write_lark_cli_runtime_from_archive and _write_lark_cli_sandbox_launcher wrote
their file with a truncating open and chmod 0o755 afterwards, so a crash mid-write
left an executable fragment behind - for the extracted binary that file can be
large. Publish through a temporary file plus os.replace, as the flow-state writer
in the same module already does.
* fix(tools): publish the default agent's SOUL.md atomically
setup_agent persisted the default agent's global SOUL.md with a
truncating write_text. The open truncates the previous file before the
new content lands, so an interrupted setup — process kill, disk full,
an exception while flushing — leaves a truncated persona file, and the
prompt builder reads SOUL.md on every run, so the agent then silently
runs with part of its instructions missing.
The same file is already published atomically everywhere else: the
custom-agent store stages it to a temporary and commits with
os.replace, and #6618 replaced the identical truncating pattern for
the Lark CLI runtime files the same day. The default agent's copy was
the one remaining write of this file that could tear.
Publish through a staged temporary plus os.replace, mirroring the
store's _stage_temp commit, and clean the temporary up when the
publish fails so no partial artifact survives.
Tests: a failed publish (os.replace raising) keeps the previous
SOUL.md byte-for-byte and leaves no temporary behind, a control pins
that the old truncating write loses the content in the same failure,
and the happy path writes the full file.
Fixes#6667
Signed-off-by: yetuge <2219677952@qq.com>
* fix(tools): keep SOUL.md's mode and clean the temporary on staging failure
Review follow-up on #6668, both non-blocking suggestions that guard the
publish itself:
- NamedTemporaryFile creates the temporary with mode 0600, so the first
publish made the global SOUL.md 0600 (the old write_text produced 0644
& ~umask) and every re-publish clobbered a mode an operator had set.
Stat the target and chmod the temporary to that mode (0644 for a first
publish) before the replace — the order #6618 codified for the Lark
runtime files and atomic_write_extensions_config already follows.
- A failure inside the staging block (handle.write or the implicit
flush/close, e.g. ENOSPC) raised before the try started, orphaning the
delete=False temporary in base_dir. One try/finally now spans staging
and publishing, so neither phase can leave a temporary behind.
The exact POSIX mode semantics were verified outside the Windows host
(WSL Ubuntu: mkstemp is 0600, first publish is 0644, a re-publish keeps
an operator-set 0640, and both failure phases leave no temporary); the
exact-mode tests are therefore POSIX-gated, with a cross-platform test
pinning that the chmod lands on the temporary before the replace and a
staging-failure test that is red on the previous head.
The shared-helper suggestion is left as a follow-up: folding six
independent publishers into one helper is a cross-module refactor, not
part of this bug fix.
Fixes#6667
Signed-off-by: yetuge <2219677952@qq.com>
---------
Signed-off-by: yetuge <2219677952@qq.com>
* fix(tui): keep a refused message in the composer
on_composer_input_submitted cleared the composer before _send_to_agent
decided whether it could start a run. A prompt typed during an active run,
while an interrupted run on the same thread was still stopping, or when the
worker failed to start was replaced by a notice and survived only in input
history.
_send_to_agent now reports whether it started a run, and the submit handler
puts the exact submitted text back with the cursor at its end, so Enter
resends it. Accepted messages and local slash commands clear the composer as
before.
* fix(tui): keep a launch message that fails to start in the composer
on_mount sent the launch message through _send_to_agent and ignored the
result, so a worker start failure lost the message: it never reached the
composer or input history. Route both the launch message and composer
submissions through one _restore_draft helper.
* docs(changelog): reference #6666 for the TUI refused-message fix
* fix(tui): keep the palette closed over a restored slash draft
Restoring a refused bare skill activation such as /tdd posted
TextArea.Changed, which reopened the command palette, so Enter filled the
skill instead of resending it. Remember the restored draft and keep the
palette closed until it is edited. Restore the history copy of the text too,
so a trimmed slash command matches what Up recalls.
* fix(tui): keep every word of an unquoted --print/--json message
--print and --json took a single optional value and never read the
positional words, so `deerflow --print summarize this repo` sent only
"summarize", `deerflow summarize this repo --print` exited 2 asking for a
message on a terminal, and with piped stdin the typed words were ignored.
Both flags now take every following word and fall back to words given
before them, as --cli does; a typed message wins over piped stdin. A
message split around the flag is refused with exit 2, because argparse
loses the word order between the two parts.
* docs(changelog): reference #6636
* fix(tui): refuse --print and --json together
Only one headless mode runs, so the other flag's words were dropped:
'--print hello --json world' ran with 'hello', and '--print --json hello'
asked for a message. Put both flags in a mutually exclusive group so argparse
exits 2 for every combination.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(community): add You.com web search provider
Adds You.com as an opt-in community `web_search` engine, mirroring the
existing community engines (tavily, brave, exa, firecrawl, groundroute).
You.com runs its own web index and returns extracted page snippets per
hit rather than a single meta description, so a research loop gets usable
context without a follow-up fetch.
The provider picks its endpoint by whether a key is configured, so it
works out of the box and scales up when the operator adds one:
- no key -> /v1/agents/search, keyless free tier
- YDC_API_KEY -> /v1/search with X-API-Key, full Search API
Endpoint and headers are always chosen together; the keyless endpoint
rejects an auth header, so a stale key can never be sent there. Both
return the same body shape, so one normalizer covers both.
httpx only, no new dependency and no pyproject extra, so uv.lock is
untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(community/youcom): review follow-ups — drop stray .pr-body.md, guard OverflowError, add 429 hint
* fix(doctor): make the You.com free-provider detail true with or without YDC_API_KEY
The doctor detail returned before any key check, so operators who had
set YDC_API_KEY still saw 'set YDC_API_KEY for the full Search API'.
Reworded to match the sibling free-provider entries and stay accurate in
both states.
* fix(search): sweep youcom max_results onto the shared owner
The single-owner gate that landed with #6563 flags this branch's private
_coerce_max_results as a newly hand-rolled copy, so backend-unit-tests
now fails after merging main. Follow the coordination note from review:
delegate to deerflow.community.search_max_results.coerce_max_results,
which owns the rejection profile (booleans, non-integral floats,
OverflowError), and keep only the 1-100 mirror of You.com's server-side
count clamp on top — clamping policy stays per-provider.
The .inf regression test now goes through the public tool path and also
pins the bool and non-integral-float shapes of the shared bar.
* docs(tools): use the bare you.com platform link in config and docs
Match every other provider entry: no other provider in config.example.yaml
or the en/zh tools.mdx uses campaign-tagged URLs, so drop the utm params
from the three you.com/platform links. Integration attribution stays in
the identifying User-Agent sent to You.com hosts.
* test(youcom): cover shared max_results ownership and bounds
Register You.com among the shared-owner providers and assert default handling and the 100-result cap through outbound request parameters.
---------
Co-authored-by: Brian Sparker <brainsparker@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: valueadd-lgtm <339269770+valueadd-lgtm@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(mcp): preserve server binding history across gateway workers
A peer Gateway worker could keep a stale pooled session after another worker deleted and identically re-added a stdio server, because the final effective MCP slice matched the revision it had already applied.
Record a per-server lifecycle generation in the shared extensions config and consume it in the reader, discovery claim, and committed handoff so an observed generation change force-rebinds only the affected server. Legacy configs derive a stable in-memory token; deletions leave permanent tombstones.
* fix(mcp): fail closed on an invalid mcpLifecycle instead of retrying
An unverifiable mcpLifecycle made initialize_mcp_tools() unpublishable, so get_cached_mcp_tools() retried discovery forever and could keep respawning stdio servers. Abort discovery on an invalid ledger after conservatively retiring local MCP state; a fixed config recovers on the next call.
Also documents the rolling-upgrade prerequisite: cross-worker ABA detection requires configuration writes to go through a lifecycle-aware writer.
* refactor(mcp): share stdio fingerprinting across lifecycle paths
Reuse lifecycle._stdio_fingerprints from the applied-revision builder so the connection-identity rule has a single implementation, and move the unusable-server diagnostic into the shared helper.
Add a regression proving the Gateway Skills writer preserves mcpLifecycle byte-for-value, and document that DeerFlowClient.update_mcp_config can raise RuntimeError after the config was committed.
* docs(mcp): condense lifecycle guidance under the AGENTS chain limit
The effective mcp/AGENTS.md chain was 15 bytes under the 98,304-byte CI hard limit, so the standalone cross-worker lifecycle bullet pushed it over. Fold the mcpLifecycle and rolling-upgrade facts into the existing selective-reconciliation bullet and condense it, keeping the chain under the limit while preserving the contract.
* docs: trim inherited agent guidance below CI size limit
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>