`_parse_results` skipped malformed page entries but returned the collected
list even when it was empty, so a non-empty `Pages` container holding no
usable page fell through to "No results found". That tells the agent the
query legitimately matched nothing instead of surfacing the malformed
payload -- the failure mode the sibling providers already contain (brave
#6550, groundroute #6603, tavily #6645, sofya #6719), and the rule the
tavily changelog entry already lists `tencent_wsa` as applying.
A missing or null `Pages` stays an empty search, a non-list container keeps
the existing structured format error, and individual malformed entries in an
otherwise valid container are still skipped so the valid rows survive. Only
the non-empty-container-with-no-usable-page case now returns the provider's
format error and logs the container, never the query or payload values.
RED/GREEN: the 5 new cases fail on main with "No results found" and pass with
the fix (stash-compared source-only); test_tencent_wsa_tools.py 33 passed,
the five provider suites 281 passed, ruff check and format clean.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): arbitrate IM transports with a per-provider leader lease
Every Gateway instance connected every enabled IM platform with the same
bot credentials, so channels could not run on more than one instance
(Telegram 409 storms, Discord answering twice, scattered per-process
state). channels.leader_lease was reserved by #6746; this implements it.
- channel_leases table (migration 0041): one row per provider. Acquire is
one dialect-native upsert decided against the database clock and bumps
a fencing epoch; renew extends only a live lease at the holder's epoch;
release sets the expiry to 0 so a peer takes over on its next tick.
- app/channels/leadership.py: the holder's local monotonic deadline
(send time of the last confirmed renewal + lease_seconds); holds() turns
false one renewal interval early, so the transport stops at least
grace_seconds before a peer may connect. DB calls are bounded.
- ChannelService: start fence in _start_channel (no DB call) behind every
start path; fixed-rate supervisor renews/acquires and runs transport
starts/stops as per-provider tasks; takeover reloads the provider's
saved config; failed starts back off; shutdown releases only leases
whose transport stopped. get_status reports leader/leader_here and the
holder's transport state on every instance.
- Notification worker leaves rows of providers another instance holds.
- Gate: the enabled=false refusal now recommends auto/true; an explicit
true with database.backend memory is refused when a channel is enabled.
- Docs, Helm, deployment guides, AGENTS.md and CHANGELOG updated.
* docs(changelog): link the channel leader lease entry to #6843
* fix(channels): see peer-held leases for providers absent from local config
Review on #6843: a provider connected from the web UI on a peer never
enters this instance's channel config, so the notification worker here
still claimed its rows (and parked them for 900s) and GET /api/channels/
reported no leader for it. Every supervision round, and startup, now also
reads the whole lease table once (bounded), and notification exclusions
and status use those observations: a provider whose live lease a peer
holds is left to it even without local config.
Running that read concurrently with renewals exposed that, through
SQLAlchemy's aiosqlite adapter, an UPDATE ... RETURNING racing a read on
another connection can hand back one of its own bind values instead of
the column (stored rows stay correct; plain sqlite3 and raw aiosqlite do
not show it). A lease must never take such a row for a confirmed
renewal, so acquire/renew/release now decide by affected-row count and
acquire reads its epoch back in the same transaction.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): append the Exit Code: 124 marker on E2B/BoxLite bash timeouts
When a bash command exceeds its timeout on the E2B or BoxLite providers,
the SDK raises (TimeoutException / TimeoutError) and the generic except
returned a bare "Error: <message>" with no exit marker; a bridge-future
timeout on BoxLite stringifies to an empty message, so the model saw just
"Error: ". Local, AIO, and Tenki all end timed-out output with the
"Exit Code: 124" marker that _truncate_bash_output preserves and the
acceptance-checklist harvest parses to recover the real shell status.
Catch the timeout exceptions ahead of the generic handler in both
providers and return the Tenki-style "Error: command timed out" +
"Exit Code: 124" ending. With #6782 forwarding bash_command_timeout to
every provider, this path is now a first-class configured behavior.
OpenSandbox routes timeouts through the same generic except, but its SDK
is an optional extra whose timeout behavior I could not verify locally,
so it is left out of this change.
Fixes#6847
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* test(boxlite): clarify bridge timeout coverage and SDK failure limits
---------
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(persistence): retry feedback and conversation upserts that lose a race
FeedbackRepository.upsert and ChannelConnectionRepository.set_thread_id
looked up the row and inserted it when missing, with no conflict
handling. Two first writers for one key both missed the lookup, and the
second INSERT violated uq_feedback_thread_run_user or
uq_channel_conversation_connection_external: a 500 for the second of two
concurrent first ratings, and a failed message when a conversation's
first message raced /new.
Both now catch IntegrityError, roll back and retry, so the losing writer
re-reads the committed row and updates it, as upsert_connection already
does. A retry rather than ON CONFLICT keeps the NULL-user feedback path
updating its single row, since a NULL key never conflicts.
* docs(changelog): reference #6852 for the upsert race fix
* test(persistence): pin upsert retry exhaustion and share the miss-select double
Both retry loops now have a test that makes every SELECT miss against a
pre-seeded row, so all attempts hit the unique key: the original
IntegrityError must propagate after exactly _UPSERT_MAX_ATTEMPTS
attempts and the stored row must stay unchanged.
The "first SELECT misses" session double lived in four copies (one per
retry test, including upsert_connection's). It now lives once in
tests/_repository_race_helpers.py, with a misses count so the same
double drives both the single lost race and exhaustion.
* feat(channels): control IM notifications and add WeChat/QQ delivery
* fix(scheduler): skip notification preference locks without recipients
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): parse Slack and DingTalk allowed_users fail-closed
Slack and DingTalk still built their allowlists with their own
_normalize_allowed_users: every entry went through str() without
stripping, only list/tuple/set counted as collections, and the gate was
`if self._allowed_users and ...`. A configured list whose entries are all
blank, such as `allowed_users: [$SLACK_USER_ID]` with the variable set to
an empty string, became an empty set and let every sender through.
" U123 " never matched U123, and a frozenset was stringified into one ID.
Parse both with the shared parse_allowed_users helper that Telegram,
WeChat and QQ use, and gate on `is not None`: unset, null, empty
collections and a blank string allow everyone; a scalar is one entry;
strings are stripped and integers keep matching their string form, as
before; booleans, floats, None and mappings are rejected (a mapping as a
whole); a configured value with no valid ID denies every sender and logs
an error. A scalar containing commas or whitespace warns that it stays
one literal ID.
* refactor(channels): share the Slack and DingTalk allowlist coercer and separator warning
Slack and DingTalk had byte-identical ID coercers and their own copies of
the scalar-separator warning. Move the coercer into allowed_users.py as
parse_string_or_int_user_id and opt both channels into
warn_on_scalar_separators, as QQ and WeChat do. Log messages are unchanged.
Pin that a separator-only string such as "," stays one literal ID
(denying everyone else) and still warns.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): merge runtime channel config saves across Gateway workers
Each Gateway worker builds its own ChannelRuntimeConfigStore, loads
runtime-config.json once, and rewrote the whole file from that copy on
every save. With two workers, a bot saved on one worker was dropped when
another bot was saved on the other, and a disconnected bot came back.
Every save, disconnect and removal now rereads the file while holding the
thread lock and an advisory sidecar file lock shared across processes,
changes only its own provider, and saves. The served snapshot is
refreshed only after a successful save, and an unreadable file still
falls back to this process's copy as before.
* docs(changelog): reference #6844 for the runtime channel config fix
* fix(channels): normalize the runtime config lock path like extensions_config
The sidecar lock was derived from the store path as given, so a worker
configured with a symlinked spelling of runtime-config.json took a
different lock file and no longer excluded peers using the real path.
Resolve a symlinked path, and make any other path absolute, before
deriving the sidecar, as extensions_config_file_lock already does.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(agents): bound viewed image context per model request
Keep recent image payloads within a count and byte budget, and move re-viewed paths to the newest position. Add focused regressions for budget enforcement and state ordering.
* test(agents): verify cumulative image budgets and re-view recovery
* fix(agents): clarify image omission and read-attempt limits
* docs(middleware): shorten image context guidance for CI budget
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): restore saved thread files when creating E2B sandboxes
After an E2B sandbox expires between turns, a replacement starts without
the thread files already saved by the Gateway, so later reads fail.
Restore owner-scoped workspace, uploads, outputs and ACP files before a
new sandbox becomes available. Keep live sandbox reuse unchanged, share
the existing upload budget, and discard incomplete sandboxes on restore
failure. Skip links and unpublished temporary files during restoration.
Add offline regressions for release, expiry and recreation, sync and async
acquisition, owner isolation, live reuse, upload limits and failure cleanup.
* fix(sandbox): make thread restore failures actionable
* fix(sandbox): use neutral upload limit diagnostics
---------
Co-authored-by: conghua1013 <16472796+conghua1013@users.noreply.github.com>
* fix(subagents): release batch executions the worker stops supervising
_execute_item cleaned up its execution in a finally block, but the
executor registry only removes terminal entries. When stop() cancelled
an item task, the subagent was still running at that point: the
cancellation request only signals it, so cleanup kept the entry, and
nothing removed it once the run turned cancelled. Each in-flight item
left its SubagentResult, transcript included, in the process-wide
registry for every SDK SubagentRuntime start/stop cycle.
A repository error while polling took the same exit without even
requesting cancellation, so the item was failed or requeued while its
subagent kept running unsupervised, and a retry could run it twice.
On either exit, cancel a run that is not terminal yet and remove its
entry from the persistent subagent loop once it stops. Cancellation is
cooperative, so the wait is bounded by the item's lease, after which
another worker owns the item and the entry is removed regardless.
* docs(changelog): reference #6785
* fix(subagents): log a scheduled batch release that fails
A concurrent future never reports an exception nobody retrieved, so a
release that raised on the subagent loop leaked its registry entry with
no log trace. Log it at error level with the execution id, as the task
tool's deferred cleanup already does.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(mindie): degrade unserializable tool args to str() in XML replay
The XML parser's literal_eval fallback puts values json.dumps rejects
(e.g. Python sets from set-literal arguments) into AIMessage tool_calls
args. Replaying that message from history hit the unguarded json.dumps
in _fix_messages' args join, raising TypeError and failing the whole
request — and every later turn, since the poisoned message rides the
history. Mirror the tool-result json block handling already in the same
function: degrade the value to str() so one unserializable argument
cannot fail the request.
* fix(mindie): reject non-finite floats in literal sets and tuples
* fix(mindie): validate literal dictionary keys recursively
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): surface missing-file AIO reads as actionable errors
When read_file targets a nonexistent file, the AIO server returns an error
envelope with data: {}, which the agent-sandbox SDK cannot parse into its
required FileReadResult fields. The resulting pydantic ValidationError text
became the model-visible tool result ("Error: 2 validation errors for
ResponseFileReadResult ..."), so the agent could not tell the file simply
does not exist and kept retrying or "fixing" nonexistent problems.
Catch ValidationError in AioSandbox.read_file and return an explicit
"Error: file not found or unreadable: <path>" instead. The branch is logged
at debug level: a missing file is an expected outcome, since
ReadBeforeWriteMiddleware probes every new-file write before allowing it, and
the previous error-level logging produced noisy gateway logs on that benign
path. Other exceptions keep the error log and the generic error string.
Fixesbytedance/deer-flow#6833
* docs(aio): align regression test reference style
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(models): tolerate null template keys in the legacy thinking disable path
`ModelConfig.when_thinking_enabled` is a plain `dict | None` with no
per-key validation, so a template key left blank in config.yaml
(`extra_body:`, `thinking:`) loads as an explicit None. The legacy
disable path read those keys with `effective_wte.get(key, {})`, whose
default only applies to an absent key — so it called `.get` on None and
every create_chat_model for the profile crashed with
AttributeError: 'NoneType' object has no attribute 'get', on the
default thinking_enabled=False path (title generation, summarization,
subagent and lead-agent builds).
Guard the three reads with the same `or {}` idiom the sibling accesses
in this module already use (`_infer_dialect`, the contract path), so a
null value falls through the dialect heuristics like an absent one and
the legacy path matches the contract path, which already tolerates the
same profile shape.
* fix(models): tolerate null nested thinking templates
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(mcp): add per-server discovery cache scaffolding
Introduce the per-server tool cache entry, canonical parsed-config snapshots on the applied revision, and the shared _binding_matches reuse predicate. Identity reuse is object-identity based on the pool's current binding (no value equality); snapshots and ownership handles stay out of repr. No selection, eviction or publication is wired yet.
* feat(mcp): selectively evict stale per-server tool cache entries
Evict per-server discovery results during selective reconciliation, after the new binding epoch is installed, so unchanged servers keep their exact entry objects while changed, removed or re-bound servers (including lifecycle-token rotations) are dropped. Delegate the binding verdict to _binding_matches so reuse and eviction cannot diverge.
Clear _server_tool_cache on every full reset, and count retained entries in the reconciliation state predicates so a lingering entry can never let a later config change be silently skipped.
* feat(mcp): reuse unchanged per-server results during initialization
Initialize MCP tools from per-server discovery instead of a single full pass: select reusable entries under the claim lock, discover only the missing servers with get_mcp_tools_by_server, then publish atomically after re-validating every candidate binding against the pool's current binding via _binding_matches. Failed servers stay absent (never cached as empty), and superseded or partially bound candidates are discarded whole. Test fixtures now drive the grouped entry point.
* fix(mcp): close T4 review gaps in per-server cache publication
Address four findings from the adversarial review: treat retained entries as unverifiable when the applied baseline is None so an interceptor change still forces a full reset; count retained entries in the claim-time shared-reset predicate; refuse to publish a false-success empty cache when every enabled server failed discovery; and hoist the lazy imports inside the initializer's try/finally so an import failure releases the initialization claim.
* fix(mcp): sanitize cache failure logging and document per-server cache
The outer lazy-initialization handler now logs only the exception type, never the raw message or traceback, so a discovery error cannot print a resolved $VAR credential. The committed-writer planner mirrors the reader's no-baseline guard so retained entries are never reused under an unknown interceptor chain. Document the per-server tool cache (reuse, invalidation, failure handling, remote tool changes) in MCP_SERVER.md and mcp/AGENTS.md, and drop phase-staging wording from the new code and tests.
* docs(mcp): keep the agent-guidance chain under its hard limit
F2's cache notes pushed the effective AGENTS.md instruction chain 905 bytes past its 98304-byte hard limit, failing the agent-guidance CI job. Compress the selective-reconciliation bullet to its invariants (the full cache behaviour lives in docs/MCP_SERVER.md), and reword the total-failure comment so it no longer implies that build-invalid connections were attempted.
* fix(mcp): fence cached tools by interceptor chain
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(uploads): stop Windows replacement uploads from being skipped by the multi-link guard
NTFS never decrements st_nlink when one of a file's names is removed, so
every upload published through os.link(staging, dest) + unlink keeps
reading st_nlink == 2 on Windows. The non-exclusive replacement guards
then reject that destination forever, and DeerFlowClient.upload_files
turns the UnsafeUploadPathError into a log line - a replacement upload
is silently skipped (Closes#6820).
- Add _replacement_destination_is_hardlinked: POSIX-authoritative
st_nlink > 1, skipped on Windows, used only on the not exclusive
replacement path. The exclusive (first-claim) path keeps the strict
count check, as does the POSIX O_NOFOLLOW branch.
- Decide "published alias" in cleanup_stale_upload_staging_files by
inode identity against siblings (st_dev/st_ino via os.lstat) instead
of the link count; DirEntry.stat reports zeros for both on Windows,
matching the existing _staging_entry_stat note.
- Guard the POSIX-only st_nlink assertions in the affected tests and
mark the three inbound-attachment mode-bit tests POSIX-only, mirroring
the existing skipif in the same class.
* fix(uploads): restore the exclusive post-open link guard and harden the staging sweep
Address the three review findings on 5353715:
- Restore the strict post-open st_nlink != 1 check for exclusive=True
on the no-O_NOFOLLOW (Windows fallback) branch, which the previous
commit dropped there against what the PR description claimed. Only
the non-exclusive replacement path defers to the Windows-aware guard.
- Never match a degenerate st_ino == 0 in the sibling-inode sweep: on
FAT/exFAT or some SMB volumes every entry would look like a published
alias and in-flight uploads would be deleted at any age. Such volumes
fall back to the min_age guard only.
- Build one {name: stat} map per uploads directory before the entry
loop so the sweep scans each directory once instead of rescanning it
for every staging file (O(N) instead of O(N^2) lstat calls).
Tests: 407 passed / 27 skipped across test_uploads_manager.py,
test_uploads_router.py, test_upload_copy_publication.py,
test_upload_windows_staging_aliases.py, test_channel_file_attachments.py
(Windows 11, Python 3.13 venv). The three new regression tests fail on
5353715 and pass with this change.
* style(uploads): apply Ruff formatting
* fix(uploads): report staging directory scan failures
---------
Co-authored-by: sjr666666 <sjr666666@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(subagents): leave full results out of the polled batch item list
The item list endpoint returns only previews, but list_items selected
whole rows, so every page loaded and decoded each item's full report,
evidence JSON and prompt before discarding them. The workspace polls
this list every 2 seconds while a batch runs; with the default
max_result_chars a 100-item page held about 34 MB and blocked the event
loop for about 70 ms (170 MB and 300 ms at limit=500).
Defer those columns unless the caller asks for them, with raiseload so
an accidental read fails loudly instead of lazy-loading. A 100-item page
now peaks under 1 MB; responses are unchanged.
* test(subagents): pin the item list route to compact columns
Drive list_batch_items against a real SQLite repository so a route that
starts requesting full results fails, not only a repository change.
* docs(changelog): reference #6781
* test(subagents): scope the route test's seed update to its batch
* fix(buzz): treat a scalar allowed_users entry as one member pubkey
BuzzChannel built its allowlist by iterating config.get("allowed_users") directly. A scalar string -- what an allowed_users: $BUZZ_OWNER_PUBKEY reference resolves to -- was iterated character by character, feeding each character to parse_pubkey, whose first non-hex character raised and aborted channel construction; a numeric scalar raised TypeError the same way. The sibling QQ (#6730) and Discord (#6776) connectors already treat a scalar as a single entry.
Parse the value with the shared parse_allowed_users helper via a parse_pubkey coercer: a scalar is one entry, list entries keep their per-entry validation, unusable entries (bad hex, numeric refs, mappings) are dropped with the shared warning instead of crashing, and the deny-by-default contract is unchanged -- an empty or blank value still yields an empty allowlist with start()'s existing deny-all warning.
config.example.yaml states the accepted shape next to the key, matching the mention_free_channels note added by #6776.
* docs(config): split buzz allowed_users note into aligned comment lines
The two-line note on the buzz example entry was emitted with a literal
`/n` where a newline was intended, so the second note never became its
own comment line and the `#`-column alignment shared with the
DENY-BY-DEFAULT lines below was broken. Split it to match the
mention_free_channels sibling shape (review comment on #6811).
wait_for_sandbox_ready polls a sandbox's health endpoint until ready or a
timeout-second budget elapses, but measured that budget on the wall clock
(start_time = time.time(), while time.time() - start_time < timeout). A
backward wall-clock step (NTP correction, VM resume, manual change) while
polling keeps the loop running far past timeout -- and this runs on the
sandbox acquisition path, so it pins the acquiring thread instead of giving
up -- while a forward jump ends the wait early and destroys a sandbox that
was about to become ready.
The loop also only checked the deadline at the top and then blocked on a
fixed 5s request timeout plus a fixed 1s sleep, so a single slow probe could
overshoot timeout by up to ~6s; on the 5s acquisition path (local_backend)
that roughly doubles the budget.
Rework the wait to a monotonic deadline that clamps both the per-probe
request timeout and the inter-probe sleep to the remaining budget
(min(5.0, remaining) / min(1.0, remaining)), making the sync twin a faithful
mirror of wait_for_sandbox_ready_async, which already does this on
loop.time().
Add two deterministic regressions: one advances a scripted monotonic clock
past the deadline while a fake wall clock steps backward (red on the old
wall-clock loop), and one asserts the per-probe timeout and sleep are clamped
to the remaining budget (red on the old fixed 5s/1s values).
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* fix(view-image): bound re-injected image context with per-request budgets
`viewed_images` only grows (#6825): every model call that follows a
`view_image` round re-encodes and re-sends every entry ever viewed, so a
long PDF-inspection thread pushes the request body past provider limits
(e.g. Anthropic's 32 MB) even though each single image is capped.
Admission now runs newest-first against two knobs on ViewImageMiddleware:
- `max_injected_images` (default 10)
- `max_injected_total_chars` (default 32M characters, so a single
max-size image stays injectable: 20 MiB source -> ~27 MB base64)
Over-budget entries keep their text bullet with a placeholder ("call
view_image again to re-view this image") and are never read from disk or
sandbox, bounding both the request size and the middleware's per-call
work. Existing threads with small image counts and sizes are unaffected.
Refs #6825
* fix(view-image): make re-viewing an omitted image actually rescue it
The placeholder told the model to "call view_image again to re-view"
an over-budget image, but the recovery advice could not work:
merge_viewed_images merged with `{**existing, **new}`, and updating an
existing key in Python preserves its original insertion position. The
admission pass walks `reversed(viewed_images.items())` purely by
position, so the re-viewed entry stayed at its old (oldest) slot, was
admitted last, and was omitted again — a wasted tool round.
merge_viewed_images now re-inserts updated keys at the end, so dict
order tracks recency and a re-viewed image is admitted first on the
next injection. Also fix the `_create_image_details_message` docstring,
which said entries are "admitted oldest-first" while admission runs
newest-first (content blocks are emitted oldest-first; the admission
decision is not).
Wiring the budget knobs into config.yaml stays a follow-up: both
production instantiation sites construct ViewImageMiddleware() with no
arguments today, and this PR keeps the surface minimal.
Tests: reducer unit tests for re-insertion order plus an end-to-end
regression that re-views an omitted image and asserts it is admitted
on the next injection (#6829 review).
* fix(mcp): keep resolved credentials out of discovery failure logs
A failed per-server MCP discovery logged the raw exception message and traceback. ExtensionsConfig resolves $VAR values before validation, so a discovery error can embed a resolved-credential request URL (for example an httpx error carrying a URL with userinfo). Log only the server name and the exception type instead, at both the per-server and whole-phase handlers, and pin it with a caplog regression test using a simulated credential.
* test(mcp): assert the exact retained server name when redacting logs
The redaction regression test asserted a bare "A" in the captured log, which any unrelated A would satisfy. Assert the full "server 'A'" fragment so the test actually proves the offending server name is preserved while the credential is not.
* test(mcp): assert structured discovery warning arguments
The redaction change made the per-server discovery warning a parameterized log call, so asserting the server name inside the raw format string no longer matches. Assert the structured arguments instead: the server name, the exception type, the fully formatted message, and the absence of exc_info.
* test: fix five Windows-host failures across sandbox, read_file, lark shim, and extension checks
Windows dev hosts currently fail five backend test cases that CI (ubuntu)
never exercises:
- test_local_sandbox_command_timeout: the fixture forces a bare /bin/sh
shell and the Windows dispatch spawns it directly (WinError 2), and
make(1) is not a given on Windows. Stub both runners; the contract under
test is the timeout marker assembly, not the runner.
- test_local_sandbox_provider_mounts: the Popen-capture mock delegated to
the real spawn, which cannot execute a POSIX shell path on Windows.
Capture at the runner seam instead and assert the rewritten path
shape-agnostically (the Windows runners pass the command string, not a
[shell, -c, command] list).
- test_read_file_tool_binary: text-mode write_text leaks CRLF into the
fixture on Windows; write the mount fixture as bytes (#6042 follow-up).
- test_lark_shim_atomic: skip the exact 0o755/0o644 assertions on Windows
hosts, whose stat mode approximates POSIX bits (#6257 pattern).
- test_extension_check_script: zipfile normalizes os.sep in entry names on
Windows reads, so a writestr-built fixture can never carry a verbatim
backslash entry there. Inject the hostile name into namelist unless the
archive already stores it, pinning the guard contract identically.
On a Windows 11 host these five files go from five failed cases to green
(102 passed, ruff check/format clean). Two more Windows-host failures
(skill-review NUL probe, jina shared-deadline classification) need deeper
root-causing and are left for follow-ups.
* test(skill-review): review the committed skill tree, not local residue
The video-generation review test scans the live directory, so untracked
build residue inside skills/public/video-generation fails it: once bytecode
caching drops a scripts/__pycache__ directory into the tree (importing the
skill script writes one), skillscan's package-undecodable-script finding is
correct and the test can never pass again until the residue is deleted by
hand. Reproduced on Windows: a planted scripts/__pycache__ pyc fails the
test; deleting it makes it pass.
- Materialize the tracked skill tree (git ls-files) into a temp dir and
review that, so the test reviews exactly the committed package.
- tests/skills/skill_loader.py disables bytecode caching around
exec_module, so the root skill suite stops polluting the committed tree.
Follows up on #6786 (item 6).
* docs(test): state the argv-list contract on both sandbox runners
The Windows dispatch also passes the resolved command as an argv list, so
the string fallback was unreachable on every platform.
---------
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
The LLM error-handling middleware's circuit breaker stored its recovery
deadline as time.time() + recovery_timeout_sec and fast-failed while
time.time() < _circuit_open_until. That is a duration deadline measured on
the wall clock, so a backward wall-clock step (NTP correction, VM resume,
manual change) while the circuit is open holds the fast-fail window open far
past recovery_timeout_sec -- shedding every LLM call until the clock catches
up -- while a forward jump ends it early.
Measure the recovery window with time.monotonic() instead, matching the rest
of the harness (task_tool deadlines, batch lease renewal, guardrail TTLs,
secret-file settle windows, journal latency). The clock is held on the
instance so tests can inject a deterministic one. The adjacent Retry-After
math stays on time.time(): it computes target.timestamp() - time.time() for
an absolute HTTP-date, which is genuinely wall-clock.
Add two regression tests (an injected-clock recovery test and a backward
wall-clock-step test) that go red without the fix, and update the two
existing recovery tests to drive the breaker through the same injectable
clock seam instead of patching time.time.
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* fix(loop-detection): let a warned agent save collected results before its final answer
The warn-level loop messages told the agent to "Stop calling tools and
produce your final answer now." Subagents read this literally: one whose
task names an output file stops all tool calls, including the single
write that delivers its result, and finishes as "completed" with nothing
on disk.
Only the two warn-level texts change. They now ask the agent to stop
repeating the looping call and wrap up, first saving what it has
collected through a file-writing tool if its task asks for a file. The
per-tool warning names the looping tool. Thresholds and the hard-stop
path (which strips tool calls) are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(loop-detection): bound final-save recovery after warnings
* docs(middleware): keep inherited guidance within CI budget
---------
Co-authored-by: huangzz <110116@nd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): make Feishu and DingTalk transports stoppable in-process
FeishuChannel.stop() and DingTalkChannel.stop() could not end their SDK
threads. Both SDKs run on a thread-private event loop that only returned
on a fatal error: lark-oapi's ws.Client.start() parks forever in
run_until_complete(_select()) and has no public stop, and the channel
never kept the client handle; dingtalk-stream's start_forever() is a
while-True loop that also catches CancelledError, and the disconnect()
the channel called does not exist on the SDK. So every stop joined for
5s, raised ChannelStopTimeout, and ChannelService retained the instance
while the platform connection stayed alive on the old process. That
burns the full budget on shutdown and on POST /api/channels/{name}/restart,
and it blocks the per-provider leader lease: a demoted leader could not
release the transport before a peer took over.
Both channels now publish the SDK thread's loop and client under a lock;
stop() schedules a shutdown on that loop and then joins. Feishu disarms
lark's _auto_reconnect before closing the socket and stopping the loop,
because start() otherwise swallows the interrupted run_until_complete in
its reconnect branch and dials again. DingTalk drives its own copy of the
SDK connect loop that re-checks the running flag before every dial, runs
the blocking ticket request off the loop, cancels keepalive with the
socket, and logs the endpoint rather than the ticket-bearing payload. A
shared shutdown_private_loop cancels the SDK's leftover tasks with a
bounded drain and closes the loop; a stop requested before the client
exists never dials. ChannelStopTimeout now only fires for a thread that
genuinely wedges inside an SDK call. Telegram already stopped this way.
Tests: a fake lark ws.Client with the SDK's start() shape (including the
reconnect branch that swallows a bare loop stop), a real local websockets
server that observes the DingTalk close, the stop-before-client race for
both, and a blocking-io anchor that stop() ends a live private loop
without blocking the Gateway loop. All eight were red on main.
* docs(changelog): link the stoppable-transports entry to #6798
* fix(channels): release the socket on a stalled close and bound the DingTalk ticket request
Review follow-ups on #6798 (Totoro-qaq):
Feishu: the bounded _disconnect() wait cancelled websockets' close before
its own close-timeout abort ran, and lark clears _conn in its finally, so
a peer that never answers the close frame left the socket to garbage
collection. Worse, a stop landing while _connect() still held lark's
_lock snapshotted _conn as None. _shutdown_ws now settles the in-flight
dial by taking that lock under one shared close deadline, snapshots the
connection, runs _disconnect() under the same deadline, and aborts the
transport itself (abort_connection_transport in base.py) when the
handshake did not finish, yielding once so connection_lost runs before
loop.stop(). DingTalk's websocket.close() path gets the same abort.
DingTalk: the SDK's open_connection() calls requests.post with no
timeout, and running it in an executor let a stalled ticket endpoint
block interpreter exit (executor workers are joined at shutdown). The
channel now sends the same request through httpx on the stream loop
under a 30s deadline, so stop() cancels it and no worker thread lingers.
Tests drive the real SDK clients against loopback peers: a WebSocket
peer that completes the handshake and never ACKs the close (both
channels assert the connection's socket reports fileno() == -1 right
after stop(); red with the abort disabled), a real websockets server for
the clean path, and a loopback ticket server (ok / 500 / never responds)
with a no-lingering-non-daemon-thread check and a request-shape check.
* fix(llm-error): retry streamed 200-OK responses that carry no generation chunk
The non-streaming empty-generations glitch (IndexError) is already
classified as transient and retried. The streaming path reports the same
upstream glitch as ValueError("No generations found in stream.") from
langchain_core's generate_from_stream; it fell through to "generic" and
failed after one attempt regardless of llm_call.retry_max_attempts.
Classify that ValueError as transient so it uses the configured retry
budget. Unrelated ValueErrors stay generic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(llm-error): keep comments adjacent to their constants
---------
Co-authored-by: huangzz <110116@nd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(artifacts): preserve account-scoped drafts across navigation
* docs(artifacts): keep draft lifecycle guidance within its budget
* fix(artifacts): retain drafts through unknown auth and isolate typing updates
* fix(artifacts): preserve drafts when visiting public showcases
* fix(artifacts): preserve edits after pending save
* fix(buzz): accept a single mention-free channel id
`_mention_free` iterated the configured value directly, so one unquoted
UUID became a set of its characters and a numeric or boolean scalar
raised TypeError while the channel was being constructed. Both shapes are
documented as valid for the sibling Discord connector.
Parse the key through a dedicated helper that follows the Discord
convention: a scalar is a single entry, list entries are kept verbatim,
and an unusable entry never matches so `require_mention` stays in force.
* fix(channels): share scalar-or-list channel ID parsing
* docs(channels): keep guidance within CI size budget
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(browser): publish captured screenshots and fetches atomically
_write_screenshot and _write_capture_output wrote the final path with a truncating
write and then returned its name, so a failure part-way through left a partial
capture that the caller reported as saved - and the leftovers then made the dedupe
helper pick a new name for the next capture. Publish through a temporary file plus
os.replace.
* fix(browser): preserve permissions on atomic capture publication
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): decode cmd.exe output with the ANSI code page under UTF-8 mode
The cmd.exe branch left its pipe decoder to the `encoding=None` default of
`_run_windows_command`, which follows Python text-mode semantics via
`locale.getpreferredencoding(False)`. Under Python UTF-8 mode
(`PYTHONUTF8=1`, `-X utf8`, the Windows installer's UTF-8 opt-in, and the
default once PEP 686 lands) that call reports `"utf-8"`, while cmd.exe and
the children it spawns keep writing the host ANSI code page - so every
non-ASCII byte in cmd output collapses to U+FFFD, with no fallback path.
Measured on a real zh-CN host under `PYTHONUTF8=1` (`getpreferredencoding`
-> `utf-8`, `getencoding` -> `cp936`):
- through the `encoding=None` default: `'��� �����Z
'`
- pinned to the ANSI code page: `'你好 日本語
'`
This branch pins the cmd.exe pipe decoder to `locale.getencoding()`
(3.11+, repo floor is 3.12), the same selection the MSYS and PowerShell
branches already use for their fallback (#6727, #6548).
## Scope note
The `encoding=None` default itself is deliberately unchanged: it mirrors
Python's own `text=True` semantics and is pinned by
`test_windows_capture_matches_subprocess_text_mode_encoding`. Only the
cmd.exe branch - whose output is known to be the ANSI code page - overrides
it, the same way the MSYS and PowerShell branches already pin their own
decoders.
Follow-up to willem-bd's review note on #6727 (inline comment 4247281255)
flagging this path.
## Testing
- new (POSIX-runnable, mocked): the cmd branch passes
`encoding=locale.getencoding()` to `_run_windows_command`, verified under
a simulated UTF-8-mode host (`getpreferredencoding` -> `utf-8`,
`getencoding` -> `cp936`).
- updated: `test_execute_command_uses_cmd_command_mode_on_windows` now also
pins the new `encoding` kwarg.
- new (Windows; skipped when the console output CP does not match the ANSI
page - sibling tests can change it process-wide): real `cmd.exe /c echo`
roundtrip through `execute_command` returns the text intact (passes on a
zh-CN host when run standalone).
- full `tests/test_local_sandbox_encoding.py`: 40 passed, 2 skipped under
both `PYTHONUTF8=0` and `PYTHONUTF8=1` (Windows 11, Python 3.13.12, repo
venv); the text-mode contract test stays green in all three parametrized
modes (`locale-code-page`, `PYTHONUTF8`, `-X-utf8`). Red run against
unmodified main: both new/updated cmd-branch tests fail (`KeyError:
'encoding'`).
- `ruff check` + `ruff format --check` clean on both files.
* fix(sandbox): use the cmd console output code page
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(runtime): preserve canonical delta-history order in cache
CachedHistorySaver composed a parent checkpoint's channel writes from
CheckpointTuple.pending_writes in storage order, but the saver's
authoritative get_delta_channel_history orders writes by (task_id, idx).
On the in-memory backend pending_writes follows task-arrival order, so a
fan-out super-step (several parallel tasks writing one channel in a single
checkpoint) was materialized in a different order with the cache enabled
than with the bare saver, breaking the cache's transparency invariant.
Sort a checkpoint's channel writes by task_id in _channel_writes, restoring
the canonical (task_id, idx) order while preserving each task's internal idx
sequence. The sort is a no-op on sqlite and postgres, whose pending_writes
already arrive ORDER BY task_id, idx.
Add async and sync regression tests over a real InMemorySaver seeded with a
fan-out checkpoint whose writes arrive out of task_id order; both diverge
from the inner saver's native walk without the fix and match with it.
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* docs(runtime): clarify cached saver ordering contract
---------
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* test(wechat): cover allowed_file_extensions coercion boundaries
* fix(wechat): never widen a configured allowed_file_extensions back to defaults
* fix(wechat): warn when a scalar allowed_file_extensions looks like a list
A comma- or space-separated scalar string is still one literal extension, so
`allowed_file_extensions: ".pdf, .zip"` blocks every file with nothing in the
startup log pointing at the config. Mirror the `_parse_allowed_users` guard.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
