Preserve filename delimiters and literal percent escapes before either HTTP transport builds a Contents API request. Add offline regression coverage for requests and the urllib fallback.
* fix(infoquest): contain malformed search payloads instead of crashing on them
`clean_results` indexed the envelope directly:
for content_list in raw_results:
content = content_list["content"]
results = content["results"]
if results.get("organic"):
for result in results["organic"]:
so a payload that did not match the expected shape either raised a raw Python
error that the tool then reported to the agent as the search result, or was
silently swallowed:
{"search_result": {"results": "maintenance"}} -> {"error": "... error='string indices must be integers'"}
{"search_result": {"results": [{"content": "x"}]}} -> {"error": "... error='string indices must be integers'"}
{"search_result": {"results": [1, 2]}} -> {"error": "... error='int' object is not subscriptable"}
{"search_result": {}} -> {"error": "... error='results'"}
and, worst of them, a malformed container was skipped silently, so the call
answered `[]` -- telling the agent the query legitimately matched nothing:
{"results": [{"content": {"results": {"organic": "x"}}}]} -> []
{"results": [{"content": {"results": {"organic": {"a": 1}}}}]} -> []
This is the containment rule the sibling bundled providers already follow
(brave #6550, groundroute #6603, tavily #6645, sofya #6719, serper #6779,
serply #6796, tencent-wsa #6799): a payload that is not a result page is a
format error, not an empty search.
`_envelope_entries` now validates the `content.results` envelope and
`_rows` validates one vertical's container, both shared by the web and image
verticals. A missing or null container stays an empty search; a container that
is not a list, or a non-empty one holding no object entries, returns the
provider's format error; malformed entries in an otherwise valid payload are
skipped in order so the usable rows survive. Transport failures keep their
existing error text.
Diagnostics log the container, the envelope position, or the unexpected type --
never queries or payload values.
* test(infoquest): cover the envelope guards and log skipped rows by position
Three review follow-ups on the malformed-payload containment: drop the
accidentally stacked @staticmethod on _envelope_entries, give _rows the same
indexed skip warning _envelope_entries already emits (with the container name
so a provider-side drift leaves a trace), and pin the four isinstance guards --
a non-object top-level payload and a non-object search_result across both
verticals -- which no existing case reached.
A `go test -v` run whose every case is skipped exits 0 and prints the same
`ok` package banner a passing run does, so the banner alone supplied positive
evidence for `tests_passed:` while no test executed. unittest (#6916) and Cargo
(#6891) already enforce the non-skipped bar; Go's per-case `--- SKIP` lines are
the only record content that separates the two here.
Non-verbose `go test` output is byte-identical for an all-skipped and a passing
package, so those records stay judged by the banner alone.
* fix(compaction): reject manual compaction of threads with pending execution
A thread paused at a static breakpoint (interrupt_before=["tools"]) keeps its
pending node send in the checkpoint's __pregel_tasks channel. The compaction
write goes through the state-mutation graph, which schedules no tasks, so
LangGraph drops the pending send ('Ignoring unknown node name tools in pending
sends') and a resumed run finishes success without ever executing the tool.
Fail closed: detect a non-empty __pregel_tasks in the raw checkpoint tuple
inside compact_thread_context and raise ContextCompactionPendingWork, mapped
to HTTP 409 by the compact route. The reservation (reserve_checkpoint_write)
already serializes this check against runs and other checkpoint writers, and
the write still forks from the exact head that was read.
Adds dual-mode (full/delta) regression tests: the rejected compaction leaves
checkpoint id, next and pending task ids untouched, and resuming afterwards
executes the pending tool exactly once; idle threads still compact.
* fix(compaction): reject unavailable checkpoints before summarizing
* docs: keep compaction guidance within merged size budget
* fix(runtime): defer host cancellation through the whole terminal finalization
Cancelling run_agent() during an early terminal-finalization await (e.g.
the delivery-receipt write) escaped the surrounding `except Exception`
guards, because asyncio.CancelledError derives from BaseException. The
cancellation then skipped the remaining terminal tail: terminal status
persistence, the completion hook, task-stop notification, and the stream
END marker.
Add `_await_despite_host_cancellation`, which shields each terminal step
so the started awaitable keeps running under host cancellation, defers the
interruption (uncancelling the task), and re-awaits the shield until the
step completes. Apply it to the durable finalization stages; the original
interruption is still re-raised after publish_end.
* fix(runtime): bound terminal deferral and shield abort-path cancellation
* fix(runtime): retain owned finalization through cancellation
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(search): surface DuckDuckGo failures instead of reporting an empty result
DDGS execution failures (rate limits, timeouts, blocked HTML/vqd extraction,
SDK errors, and the missing ddgs dependency) were caught in _search_text and
converted into an empty list, so web_search reported "No results found" for
every failure. The agent then read an outage as a query that matched nothing
and rewrote the query instead of trying an alternative.
_search_text now raises DDGSearchError carrying the exception type and message
for those failures, and web_search_tool answers them with a structured
"DuckDuckGo search failed: ..." error. A genuinely empty result set (the SDK's
exact DDGSException("No results found.") sentinel) still returns the existing
"No results found" response, and tool_result_meta classifies the two cases
differently (transient/config vs no_results).
Tests: tests/test_ddg_search_tools.py (three new failure tests red on main),
tests/test_agent_guidance_check.py, tests/test_tool_result_meta.py; ruff check
and ruff format --check clean.
* fix(search): keep DDGS client construction inside the error boundary
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs: add a Korean README translation
* docs(changelog): reference #6909 for the Korean README
* docs(readme-ko): restore blank lines before headings and sync with README.md
Six headings sat directly under the preceding paragraph or code fence where
README.md keeps a blank line. Also translate the README.md additions from
#6898 (Codex tool_choice), #6886 (mixed-provider citation cleanup), and #6701
(authenticated HTML preview resources).
* fix(community): reserve Browserless capture names before publishing
Name selection and publication were two separate operations:
`_dedupe_output_name` probed with `Path.exists()` and then
`publish_bytes_atomically` ran `os.replace`, so two `web_capture` calls in the
same thread outputs directory could both settle on `report.png` before either
wrote, and the second publication replaced the first capture while both reported
success for the same artifact path (#6883). `web_capture_tool` runs the writer
through `asyncio.to_thread`, and same-second default names collide the same way
because the generated stem's timestamp has second resolution.
Claim the name instead of checking for it: an `O_CREAT | O_EXCL` open either
creates an empty placeholder or reports the name held, so the probe walk
(-1, -2, ... then the microsecond timestamp) is race-free. `_write_capture_output`
releases the placeholder when publication fails, catching BaseException so an
interruption also frees the name, and a failing unlink is logged rather than
allowed to mask the original error - the same error-masking family #6871 is
addressing in the shared publisher, which this change deliberately does not
touch.
When even the timestamped names are held, the function now raises instead of
returning a name it cannot own; returning one would reintroduce the silent
overwrite. `tests/test_browser_outputs_atomic.py` pins the publisher call as
source text, so its expectation moved to the new call shape in the same commit.
Refs #6883
* fix(community): open the reserved capture name with new-file permissions
An `os.open` without an explicit mode creates the file with `0o777 & ~umask`, and
`publish_bytes_atomically` preserves an existing destination's mode, so the
reservation added in the previous commit made every capture executable (`0o755`
under the usual umask) where main produced `0o666 & ~umask` (`0o644`) through the
publisher's new-file branch. Pass `0o666` explicitly and pin the mode against a
`Path.write_bytes` reference in the same directory, which is the permission the
publisher documents for new files.
Refs #6883
* fix(browserless): warn when capture reservation cleanup fails
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(serper): report a format error for results with no usable objects
`_response_items` skipped non-object entries but returned the filtered
list even when it was empty, so a non-empty `organic`/`images` array
holding no objects fell through to "No results found" / "No images
found" -- telling the agent the query legitimately matched nothing
instead of surfacing the malformed payload.
Treat a non-empty container with no object entries as malformed, matching
the containment rule #6603 documents for this provider family (brave
#6550, groundroute #6603, tavily #6645, webz). `web_search` and
`image_search` share this helper, so both now report the provider's
format error. Missing, null, and empty containers keep reading as an
empty search, and mixed lists still keep their valid entries in order.
* fix: address review and validation findings for PR 6779
* fix(client): preserve text appended before the first state snapshot
* test(client): exercise first-snapshot reconciliation with stable IDs
---------
Co-authored-by: BaiHeng Peng <191328900+Aldenshile@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): generalize IM group targets and add task/goal run notifications
- Generalize proactive group notification capability (proactive_group_notifications), binding validation, and scheduled task group selection across Feishu, QQ, and WeCom while keeping WeChat personal-only.
- Add QQ group active notification delivery via POST /v2/groups/{group_openid}/messages (msg_type=2) and WeCom group active notification delivery via WS Client.sendMessage(chatid, body) with bot_id:group:chatid workspace isolation.
- Generalize notification outbox sources via migration 0043_runtime_im_notices (source_kind, source_key, run_id, nullable task_id/task_run_id) with backward-compatible SQLite/PostgreSQL upgrades.
- Add opt-in ordinary task & goal run IM notifications (run_completed, run_failed, human_input_requested) on the shared outbox, delivery worker, renderer, and settings UI.
* fix(notifications): address review feedback and group/runtime notification edge cases
* test(migrations): update CURRENT_HEAD to 0043_runtime_im_notices across migration chain tests
* fix(channels): preserve group binding names and legacy authorization
Carry owner-chosen group names through existing connect state, keep private bindings out of legacy group fallback, and restore completion metadata retries. Cover real binding and worker finalization paths and synchronize notification docs.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(knowledge): scope ragflow link cleanup to its native citation namespace
budget_source_artifact cleaned a delegated task summary with two regexes that
matched any #knowledge-... destination. When a task result carried both the
ragflow-native contract (#knowledge-<id>-<n>) and the unified contract from the
retrieval refactor (#knowledge-source-<id>), those regexes also flattened the
unified citation to plain text. budget_source_artifact only rebuilds the native
knowledge_sources, so the now-linkless unified evidence then failed the
prune_sources substring check and retrieval_sources was emptied. The native
citations in the same result survived, so a mixed result lost attribution for
its unified sources while keeping it for ragflow ones, even with budget headroom
to spare.
Anchor both regexes to the native namespace with a (?!source-) lookahead so they
leave #knowledge-source-<id> for prune_sources to reconcile downstream. Native
ids are 32-hex and never begin with "source-", so native link cleanup is
unchanged.
Add a regression that drives the real middleware entry point on a mixed-contract
task result and asserts the unified citation stays clickable and its
retrieval_sources record survives alongside the native one.
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* fix(knowledge): remove dangling citation fragments from task summaries
---------
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(lark): retry the Windows install lock instead of failing after ~10s
msvcrt.LK_LOCK gives up after roughly ten seconds and surfaces contention
as Permission denied, while a real install (download + setup of the
lark/sandbox CLI) can hold the lock far longer. Retry the non-blocking
acquisition on the Windows fallback branch so concurrent Gateway workers
serialize instead of failing, mirroring the extensions manager's lock
introduced in #6020. The POSIX flock branch is unchanged.
Closes#6879
* test(lark): inject msvcrt stub with raising=False for Linux runners
The regression test patches the module-level ``msvcrt`` attribute, which only
exists on Windows (the POSIX branch imports ``fcntl`` instead), so
``monkeypatch.setattr`` with the default ``raising=True`` raised AttributeError
on Linux CI (backend-unit-tests shard 4).
Pass ``raising=False`` so the stub is injected on POSIX runners: ``fcntl`` is
patched to ``None`` first, and ``_exclusive_install_lock`` then takes the
Windows fallback branch against the fake. Windows behavior is unchanged.
Scope: backend/tests/test_lark_cli_install_lock.py, 1 line.
Verification: pytest tests/test_lark_cli_install_lock.py on Windows
(Python 3.12, pytest 9.0.3) -- 3 passed incl. the real-contention case;
Linux CI rerun is the authoritative check.
* fix(lark): harden and share advisory lock retries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
`web_search_tool` skipped non-object entries but returned the filtered list
even when it was empty, so a non-empty `results`/`entries`/`articles`
container holding no objects fell through to "No results found". That tells
the agent the query legitimately matched nothing instead of surfacing the
malformed payload -- the failure mode the sibling providers already contain
(brave #6550, groundroute #6603, tavily #6645, sofya #6719): a 200 body
wrapped around an upstream error page ends a research loop as "no hits".
Extract `_response_rows`, which treats a missing or null container as an
empty search and a container that is not a list, or a non-empty list holding
no objects, as the provider's format error, keeping non-object entries
skipped in order so valid rows survive. Both call sites (all three verticals
share one helper) now return the existing structured error.
RED/GREEN: the 7 new cases fail on main with "No results found" and pass
with the fix; test_serply_tools.py 34 passed, the five provider suites
(serply/serper/tavily/sofya/tencent_wsa) 284 passed, ruff check and format
clean.
* fix(discord): accept inbound attachments and file-only messages
Empty content no longer drops messages that have attachments. Capture
CDN descriptors on inbound, download in receive_file via Bot auth, and
hand bytes through INBOUND_FILE_CONTENT_KEY like Slack/Telegram.
Closes#6856
* fix(discord): gate bot auth and share inbound file helpers
Address review on #6857: attach Bot Authorization only for Discord CDN
hosts, report Content-Encoding refusals as download failed, and hoist
filename/size helpers into app.channels.inbound_files (Telegram delegates
to the same module).
* fix(discord): sort telegram imports for ruff I001
CI lint-backend failed on unsorted imports in telegram.py after the
shared inbound_files helper was introduced.
* style: ruff-format discord inbound attachments test
* fix(discord): show the enforced attachment download limit
---------
Co-authored-by: Li-john1021 <Li-john1021@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(buzz): page a saturated history instead of skipping the oldest
buzz-relay serves at most RELAY_HISTORY_LIMIT (2000) stored events per
subscription, NEWEST-FIRST, even when the filter carries a `since`. A channel
that banked more than the cap across one disconnect therefore had its OLDEST
unread messages neither sent nor marked: the live page delivered the newest
2000, the watermark advanced past them, and the rest were lost -- the one
remaining path in this connector that could skip a message outright.
A chat/backlog page's EOSE now drives a descending `until` walk. When a page
comes back AT the cap it may have older matches held back, so the connector
re-asks with `until` set to that page's oldest `created_at` -- on a SEPARATE
`buzz-backlog-<uuid>` subscription id, never the live one -- and repeats until
a page comes back under the cap or reaches the `since` the live REQ was sent
with, bounded by MAX_BACKLOG_PAGES (50, warned when hit). Delivery is charged
to the subscription that carried it before any gate, so a page's saturation is
measured against what the relay sent, not what the allowlist/mention gates kept.
The walk is strictly additive: the live subscription, its filter, and the
watermark are never touched; a CLOSED for a backlog page only ends that walk;
re-opening the live subscription closes any superseded page rather than
orphaning it; and every failure simply leaves the pre-existing behaviour for the
next reconnect to retry.
* fix(buzz): skip the backlog walk when the channel has no read floor
Reviewer willem-bd noted the descending-until walk has no semantic stop when
`floor is None`: that is exactly a channel's first connect (no stored
watermark) and again after a MAX_CACHED_CHANNELS eviction of its cursor. On
those connects the walk would fetch up to MAX_BACKLOG_PAGES * RELAY_HISTORY_LIMIT
events per channel instead of the newest 2000 the connector used to take.
Require a floor before opening any backlog page. The backlog-loss case this walk
exists for always has a watermark (the cursor survives reconnects and restarts),
so it always has a floor; first connect and an evicted cursor keep the previous
behaviour. Covers the skip with a new test, retargets the page-bound test to a
real walk above its floor, and documents the floor requirement.
* fix(buzz): stop orphaned and stalled backlog walks
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
`_parse_results` skipped malformed page entries but returned the collected
list even when it was empty, so a non-empty `Pages` container holding no
usable page fell through to "No results found". That tells the agent the
query legitimately matched nothing instead of surfacing the malformed
payload -- the failure mode the sibling providers already contain (brave
#6550, groundroute #6603, tavily #6645, sofya #6719), and the rule the
tavily changelog entry already lists `tencent_wsa` as applying.
A missing or null `Pages` stays an empty search, a non-list container keeps
the existing structured format error, and individual malformed entries in an
otherwise valid container are still skipped so the valid rows survive. Only
the non-empty-container-with-no-usable-page case now returns the provider's
format error and logs the container, never the query or payload values.
RED/GREEN: the 5 new cases fail on main with "No results found" and pass with
the fix (stash-compared source-only); test_tencent_wsa_tools.py 33 passed,
the five provider suites 281 passed, ruff check and format clean.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): arbitrate IM transports with a per-provider leader lease
Every Gateway instance connected every enabled IM platform with the same
bot credentials, so channels could not run on more than one instance
(Telegram 409 storms, Discord answering twice, scattered per-process
state). channels.leader_lease was reserved by #6746; this implements it.
- channel_leases table (migration 0041): one row per provider. Acquire is
one dialect-native upsert decided against the database clock and bumps
a fencing epoch; renew extends only a live lease at the holder's epoch;
release sets the expiry to 0 so a peer takes over on its next tick.
- app/channels/leadership.py: the holder's local monotonic deadline
(send time of the last confirmed renewal + lease_seconds); holds() turns
false one renewal interval early, so the transport stops at least
grace_seconds before a peer may connect. DB calls are bounded.
- ChannelService: start fence in _start_channel (no DB call) behind every
start path; fixed-rate supervisor renews/acquires and runs transport
starts/stops as per-provider tasks; takeover reloads the provider's
saved config; failed starts back off; shutdown releases only leases
whose transport stopped. get_status reports leader/leader_here and the
holder's transport state on every instance.
- Notification worker leaves rows of providers another instance holds.
- Gate: the enabled=false refusal now recommends auto/true; an explicit
true with database.backend memory is refused when a channel is enabled.
- Docs, Helm, deployment guides, AGENTS.md and CHANGELOG updated.
* docs(changelog): link the channel leader lease entry to #6843
* fix(channels): see peer-held leases for providers absent from local config
Review on #6843: a provider connected from the web UI on a peer never
enters this instance's channel config, so the notification worker here
still claimed its rows (and parked them for 900s) and GET /api/channels/
reported no leader for it. Every supervision round, and startup, now also
reads the whole lease table once (bounded), and notification exclusions
and status use those observations: a provider whose live lease a peer
holds is left to it even without local config.
Running that read concurrently with renewals exposed that, through
SQLAlchemy's aiosqlite adapter, an UPDATE ... RETURNING racing a read on
another connection can hand back one of its own bind values instead of
the column (stored rows stay correct; plain sqlite3 and raw aiosqlite do
not show it). A lease must never take such a row for a confirmed
renewal, so acquire/renew/release now decide by affected-row count and
acquire reads its epoch back in the same transaction.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): control IM notifications and add WeChat/QQ delivery
* fix(scheduler): skip notification preference locks without recipients
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(agents): bound viewed image context per model request
Keep recent image payloads within a count and byte budget, and move re-viewed paths to the newest position. Add focused regressions for budget enforcement and state ordering.
* test(agents): verify cumulative image budgets and re-view recovery
* fix(agents): clarify image omission and read-attempt limits
* docs(middleware): shorten image context guidance for CI budget
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): restore saved thread files when creating E2B sandboxes
After an E2B sandbox expires between turns, a replacement starts without
the thread files already saved by the Gateway, so later reads fail.
Restore owner-scoped workspace, uploads, outputs and ACP files before a
new sandbox becomes available. Keep live sandbox reuse unchanged, share
the existing upload budget, and discard incomplete sandboxes on restore
failure. Skip links and unpublished temporary files during restoration.
Add offline regressions for release, expiry and recreation, sync and async
acquisition, owner isolation, live reuse, upload limits and failure cleanup.
* fix(sandbox): make thread restore failures actionable
* fix(sandbox): use neutral upload limit diagnostics
---------
Co-authored-by: conghua1013 <16472796+conghua1013@users.noreply.github.com>
