* fix(mcp): consume resolved notification snapshots instead of launching empty events
A tracking-degraded snapshot recorded by a failed poll can resolve before any
notification worker claims it: the next poll succeeds, or an answered
input_required prompt returns the task to working. _record_event_if_changed
left that pending snapshot in place when the event disappeared, so
claim_notification_work rebuilt it as None and still claimed it, and the
service launched an empty notification run (dispatch_event or {}).
Consume such snapshots at claim time instead: advance notified_version, return
the status to none, clear the claim, and skip the row. The same guard consumes
rows left without a dispatch event by pre-fix claims, which would otherwise
relaunch empty notifications on every retry until dead-lettering.
* fix(mcp): clear consumed snapshot state for the review follow-ups
Review feedback on the claim-time consume guard:
- clear event_fingerprint when consuming, so an identical later attention
event is recorded again instead of being suppressed by a snapshot that was
never delivered;
- clear stale notification_error/attempt/dispatch state so a status
projection cannot show a notification that no longer exists;
- cover the reclaimed expired-claim shape (claim, lease expiry, resolve,
reclaim) and the identical-event sequence with regressions.
* docs(persistence): document the notification consume rule
mcp/AGENTS.md's effective chain has only 85 bytes of headroom on main (98,219 / 98,304), so the rule moves to persistence/AGENTS.md, which owns the mcp_tasks storage contract. No behavior change.
* fix(mcp): fence the notification consume against newer events (review)
Two P2 findings on the claim-time consume:
- the consume acknowledged the latest event version from a stale read, so on
SQLite a completion recorded after the claim's SELECT was overwritten with
status none and lost forever. The consume is now a conditional UPDATE fenced
on the event version and phase that were read; a newer event makes it a
no-op and stays claimable;
- a legacy launching/dispatched row whose dispatch payload is empty but whose
event version moved on consumed that newer event. Such rows are now rebuilt
and claimed; only a generation with nothing newer is consumed.
Regressions: a completion recorded while a claim is paused after its SELECT,
and legacy launching/dispatched rows with and without a newer completion.
* fix(sandbox): let local file tools read the /mnt/user-data root
The local provider maps the aggregate /mnt/user-data root so listing it
matches AIO, but the tool layer only accepted paths below it: ls, glob
and grep on the root answered "Permission denied" (with a trailing
slash, from the resolved-path "path traversal detected" check).
validate_local_tool_path now accepts the bare root for reads, and the
user-data resolver accepts the shared parent of workspace/uploads/
outputs itself when the caller opts in. Only ls and the shared read
path opt in; writes to the root and files placed directly in it stay
rejected.
* docs(changelog): reference #6991 for the /mnt/user-data root fix
* fix(sandbox): list the /mnt/user-data root through its allowed dirs only
The first version let glob, grep and ls start at the user-data root
after validating only the root itself. The walkers then descended into
everything under the host user-data/ directory, so a file or directory
stored directly under it was listed and searched although read_file
rejects it.
ls now lists the root one allowed directory at a time (workspace,
uploads and outputs, each one level deep) through the sandbox's own
list_dir, so the host root is never walked and list_dir's existing
per-directory containment check still filters symlinks. The resolver
is back to main's behavior, so glob, grep and read_file keep rejecting
the root.
* fix(sandbox): resolve every entry of colon-joined path lists in forward path translation
* fix(sandbox): keep the original text of colon-list entries when reassembling
* fix(sandbox): keep a concurrent quarantine fence when retire() races it
`SandboxQuarantine.retire()` unlinks every generation record and then calls
`directory.rmdir()` unguarded. A peer fencing the same logical sandbox between
the `iterdir()` and the `rmdir()` — a concurrent teardown of a replacement,
which this caller's cross-worker lease does not exclude because it fences only
its own logical ID — leaves a fresh record in the directory, so `rmdir()`
raises ENOTEMPTY. That OSError escapes `_retire_absent_quarantine`'s teardown
path and aborts the release flow on a fence a peer just wrote.
Its sibling `retire_replaced_generations()` already guards the identical
`rmdir` with `try/except OSError` for exactly this reason. Apply the same guard
here: the fresh record proves the directory is live, so it is kept and teardown
does not abort.
Regression test replays the race deterministically by wrapping `Path.rmdir`;
red on main (OSError: directory not empty), green with the guard.
* docs(sandbox): scope the retire() fence guard to what it actually holds
Review feedback from @willem-bd on #6961: two non-blocking docstring accuracy
notes, both about the paragraph added by e62e0c3c. Docstring-only, no
behaviour change.
1. The mechanism was described as if a peer teardown of a replacement were a
different key. It is not. Per `_claim_ownership` / `_held_teardown_lease`
the teardown lease is keyed by the logical ID and a replacement shares that
ID, so while the `del:` marker is held a peer's claim — and an acquire-side
`take()` for the replacement — *are* excluded. What is not guaranteed is
that the exclusion is still held at `rmdir` time: it is a TTL lease, not a
lock, so the heartbeat can lose it mid-teardown (the documented "Lost the
teardown exclusion" path stops refreshing while the stop continues) and the
config schema permits TTLs below a normal stop. Reworded to that effect,
which also corrects the same sentence in e62e0c3c's message.
2. The paragraph read as if the listing → `rmdir` window were the only one.
Added an explicit scope clause: the guard is best-effort rather than full
fence preservation. A peer record landing before the listing is still
unlinked along with the rest, and the unlink loop has unguarded sibling
windows of its own — a concurrent `retire()` can remove the directory
between the existence check and the listing (`FileNotFoundError`), and on
Windows a peer's `mark()` still holding the fresh record open for its
`fsync` raises `PermissionError`.
Validation: `ruff check` and `ruff format --check` pass on the file and across
`backend/`; the new regression test plus the existing quarantine suite are 9
passed; `scripts/check_agent_guidance.py` reports 46 AGENTS.md / 0 errors.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(infoquest): contain malformed search payloads instead of crashing on them
`clean_results` indexed the envelope directly:
for content_list in raw_results:
content = content_list["content"]
results = content["results"]
if results.get("organic"):
for result in results["organic"]:
so a payload that did not match the expected shape either raised a raw Python
error that the tool then reported to the agent as the search result, or was
silently swallowed:
{"search_result": {"results": "maintenance"}} -> {"error": "... error='string indices must be integers'"}
{"search_result": {"results": [{"content": "x"}]}} -> {"error": "... error='string indices must be integers'"}
{"search_result": {"results": [1, 2]}} -> {"error": "... error='int' object is not subscriptable"}
{"search_result": {}} -> {"error": "... error='results'"}
and, worst of them, a malformed container was skipped silently, so the call
answered `[]` -- telling the agent the query legitimately matched nothing:
{"results": [{"content": {"results": {"organic": "x"}}}]} -> []
{"results": [{"content": {"results": {"organic": {"a": 1}}}}]} -> []
This is the containment rule the sibling bundled providers already follow
(brave #6550, groundroute #6603, tavily #6645, sofya #6719, serper #6779,
serply #6796, tencent-wsa #6799): a payload that is not a result page is a
format error, not an empty search.
`_envelope_entries` now validates the `content.results` envelope and
`_rows` validates one vertical's container, both shared by the web and image
verticals. A missing or null container stays an empty search; a container that
is not a list, or a non-empty one holding no object entries, returns the
provider's format error; malformed entries in an otherwise valid payload are
skipped in order so the usable rows survive. Transport failures keep their
existing error text.
Diagnostics log the container, the envelope position, or the unexpected type --
never queries or payload values.
* test(infoquest): cover the envelope guards and log skipped rows by position
Three review follow-ups on the malformed-payload containment: drop the
accidentally stacked @staticmethod on _envelope_entries, give _rows the same
indexed skip warning _envelope_entries already emits (with the container name
so a provider-side drift leaves a trace), and pin the four isinstance guards --
a non-object top-level payload and a non-object search_result across both
verticals -- which no existing case reached.
* fix(sandbox): reject shell-variable path prefixes in the host-bash guard
The local path guard scans commands for absolute paths with a regex whose
lookbehind skips a `/` that follows a word character. `~/`, `${HOME}/` and
`"$HOME"/` were rejected, but the bare `$HOME/...` form (quoted or not)
was never matched, so `cat $HOME/.ssh/id_rsa` ran on the host when
`sandbox.allow_host_bash` was enabled.
Scan `$NAME/...` separately and reject it, since the variable's value is
unknown at validation time. Variables inside URLs and non-ASCII prose keep
the existing exemptions. `$PWD/...` stays allowed because it names the same
file as the equivalent relative path, which the guard already accepts,
unless the command rebinds `PWD` itself.
* docs(changelog): reference #6930 for the host-bash variable path fix
* fix(sandbox): cover positional parameters and CDPATH in the host-bash guard
Review follow-up. `cat $1/etc/passwd` slipped both scans: the new variable
scan only matched names, and the absolute-path regex skips a `/` after a
digit. `bash -c` gets no positionals, so `$1` expands to nothing and the
shell opens `/etc/passwd`. Match `$0`-`$9` as well, but keep awk-style
arithmetic such as `$1/1024` allowed, since it was accepted before and
standard host roots do not start with a digit.
`CDPATH=$HOME cd docs` moved a relative `cd` outside the workspace, taking
both `$PWD/x` and a plain relative `x` with it. Reject any command that
mentions CDPATH in the working-directory check, so the `$PWD` exemption's
equivalence with relative paths holds.
* fix(sandbox): cover named homes and non-ASCII variable paths in the host-bash guard
- Reject `~name/...`, which expands to that user's home and slips the
absolute-path scan the same way `$HOME/...` did.
- Stop applying the non-ASCII prose exemption to the variable scan:
`$HOME/文档/...` is a host path.
- Allow awk field division `$1/$2`, which main accepts, and capture the
remainder in a lookahead so skipping `$1/` does not hide a later
`$2/etc/passwd`.
A `go test -v` run whose every case is skipped exits 0 and prints the same
`ok` package banner a passing run does, so the banner alone supplied positive
evidence for `tests_passed:` while no test executed. unittest (#6916) and Cargo
(#6891) already enforce the non-skipped bar; Go's per-case `--- SKIP` lines are
the only record content that separates the two here.
Non-verbose `go test` output is byte-identical for an all-skipped and a passing
package, so those records stay judged by the banner alone.
* fix(compaction): reject manual compaction of threads with pending execution
A thread paused at a static breakpoint (interrupt_before=["tools"]) keeps its
pending node send in the checkpoint's __pregel_tasks channel. The compaction
write goes through the state-mutation graph, which schedules no tasks, so
LangGraph drops the pending send ('Ignoring unknown node name tools in pending
sends') and a resumed run finishes success without ever executing the tool.
Fail closed: detect a non-empty __pregel_tasks in the raw checkpoint tuple
inside compact_thread_context and raise ContextCompactionPendingWork, mapped
to HTTP 409 by the compact route. The reservation (reserve_checkpoint_write)
already serializes this check against runs and other checkpoint writers, and
the write still forks from the exact head that was read.
Adds dual-mode (full/delta) regression tests: the rejected compaction leaves
checkpoint id, next and pending task ids untouched, and resuming afterwards
executes the pending tool exactly once; idle threads still compact.
* fix(compaction): reject unavailable checkpoints before summarizing
* docs: keep compaction guidance within merged size budget
Serper keeps a private _coerce_max_results in the pre-#5852 shape: a bare
int() silently turns max_results: true into 1 and 3.5 into 3, and an invalid
value falls back without a warning (#5865).
Delegate to deerflow.community.search_max_results.coerce_max_results, which
owns what invalid means for the folded providers, and keep Serper's
10-result cap at the call site the way You.com does, because clamping policy
is deliberately per-provider.
Invalid configured counts now warn under the Serper label and fall back to
the default instead of being silently coerced.
E2BSandbox.grep() applied re.escape() to the caller's pattern and then
handed the escaped text to grep -F. Fixed-string grep treats the added
backslashes as part of the text to find, so an exact literal such as
'alpha.beta' never matched. Align with the boxlite/tenki/opensandbox
seams: grep receives the RAW pattern (-F matches it literally, -E as a
regex), and only a non-literal pattern is compile-validated locally.
A shared-contract regression covers dot/space/bracket/backslash/plus
literals through every shell-backed provider.
* fix(runtime): defer host cancellation through the whole terminal finalization
Cancelling run_agent() during an early terminal-finalization await (e.g.
the delivery-receipt write) escaped the surrounding `except Exception`
guards, because asyncio.CancelledError derives from BaseException. The
cancellation then skipped the remaining terminal tail: terminal status
persistence, the completion hook, task-stop notification, and the stream
END marker.
Add `_await_despite_host_cancellation`, which shields each terminal step
so the started awaitable keeps running under host cancellation, defers the
interruption (uncancelling the task), and re-awaits the shield until the
step completes. Apply it to the durable finalization stages; the original
interruption is still re-raised after publish_end.
* fix(runtime): bound terminal deferral and shield abort-path cancellation
* fix(runtime): retain owned finalization through cancellation
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* test(runtime): cover the driver suffix in sync and store DSNs
* fix(runtime): strip the SQLAlchemy driver suffix from sync and store DSNs
With `postgres_url: postgresql+asyncpg://...` and the default empty
`postgres_schema`, the sync checkpointer, the sync store and the async
store passed the DSN to psycopg's `from_conn_string` unchanged, and
libpq cannot parse the `+asyncpg` scheme. `dsn_with_search_path` only
drops the suffix when it adds a schema, so a non-empty schema hid the
bug. The async checkpointer already calls `normalize_libpq_dsn` first.
Call `normalize_libpq_dsn` at these three call sites as well.
* refactor(persistence): drop the driver suffix inside dsn_with_search_path
dsn_with_search_path dropped a SQLAlchemy `+driver` suffix only when it
added a search_path, so every libpq caller had to wrap the DSN in
normalize_libpq_dsn to cover an empty schema, and flattening that
nesting later would bring the bug back.
Normalize at the top of dsn_with_search_path so both paths drop the
suffix and check the scheme in one place, and remove the outer call
from its five callers. The sync checkpointer and both stores are back
to their code on main.
* fix(search): surface DuckDuckGo failures instead of reporting an empty result
DDGS execution failures (rate limits, timeouts, blocked HTML/vqd extraction,
SDK errors, and the missing ddgs dependency) were caught in _search_text and
converted into an empty list, so web_search reported "No results found" for
every failure. The agent then read an outage as a query that matched nothing
and rewrote the query instead of trying an alternative.
_search_text now raises DDGSearchError carrying the exception type and message
for those failures, and web_search_tool answers them with a structured
"DuckDuckGo search failed: ..." error. A genuinely empty result set (the SDK's
exact DDGSException("No results found.") sentinel) still returns the existing
"No results found" response, and tool_result_meta classifies the two cases
differently (transient/config vs no_results).
Tests: tests/test_ddg_search_tools.py (three new failure tests red on main),
tests/test_agent_guidance_check.py, tests/test_tool_result_meta.py; ruff check
and ruff format --check clean.
* fix(search): keep DDGS client construction inside the error boundary
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* test(image-search): expect source_url in DuckDuckGo and Serper results
* fix(image-search): keep the source page URL in DuckDuckGo and Serper results
The DuckDuckGo and Serper image_search tools returned only title,
image_url and thumbnail_url and dropped the page each image comes from
(DDGS `url`, Serper `link`). Brave already returns it as `source_url`,
and the workspace UI links every image thumbnail through `source_url`,
so DuckDuckGo (the default provider) and Serper results rendered as
"Unsafe link omitted" and the agent lost the image's source page.
Return `source_url` from both providers, screened with
`safe_public_result_url` like the image and thumbnail URLs.
* fix(community): reserve Browserless capture names before publishing
Name selection and publication were two separate operations:
`_dedupe_output_name` probed with `Path.exists()` and then
`publish_bytes_atomically` ran `os.replace`, so two `web_capture` calls in the
same thread outputs directory could both settle on `report.png` before either
wrote, and the second publication replaced the first capture while both reported
success for the same artifact path (#6883). `web_capture_tool` runs the writer
through `asyncio.to_thread`, and same-second default names collide the same way
because the generated stem's timestamp has second resolution.
Claim the name instead of checking for it: an `O_CREAT | O_EXCL` open either
creates an empty placeholder or reports the name held, so the probe walk
(-1, -2, ... then the microsecond timestamp) is race-free. `_write_capture_output`
releases the placeholder when publication fails, catching BaseException so an
interruption also frees the name, and a failing unlink is logged rather than
allowed to mask the original error - the same error-masking family #6871 is
addressing in the shared publisher, which this change deliberately does not
touch.
When even the timestamped names are held, the function now raises instead of
returning a name it cannot own; returning one would reintroduce the silent
overwrite. `tests/test_browser_outputs_atomic.py` pins the publisher call as
source text, so its expectation moved to the new call shape in the same commit.
Refs #6883
* fix(community): open the reserved capture name with new-file permissions
An `os.open` without an explicit mode creates the file with `0o777 & ~umask`, and
`publish_bytes_atomically` preserves an existing destination's mode, so the
reservation added in the previous commit made every capture executable (`0o755`
under the usual umask) where main produced `0o666 & ~umask` (`0o644`) through the
publisher's new-file branch. Pass `0o666` explicitly and pin the mode against a
`Path.write_bytes` reference in the same directory, which is the permission the
publisher documents for new files.
Refs #6883
* fix(browserless): warn when capture reservation cleanup fails
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(serper): report a format error for results with no usable objects
`_response_items` skipped non-object entries but returned the filtered
list even when it was empty, so a non-empty `organic`/`images` array
holding no objects fell through to "No results found" / "No images
found" -- telling the agent the query legitimately matched nothing
instead of surfacing the malformed payload.
Treat a non-empty container with no object entries as malformed, matching
the containment rule #6603 documents for this provider family (brave
#6550, groundroute #6603, tavily #6645, webz). `web_search` and
`image_search` share this helper, so both now report the provider's
format error. Missing, null, and empty containers keep reading as an
empty search, and mixed lists still keep their valid entries in order.
* fix: address review and validation findings for PR 6779
* fix(opensandbox): report the command deadline as Exit Code: 124
`execute_command` had no timeout branch, so a command that hit its deadline
was reported as whatever the SDK happened to surface. OpenSandbox is the last
provider without the `Exit Code: 124` marker that Local/AIO/Tenki/E2B/BoxLite
emit, and which #6847 established the evidence chain parses to recover the
real shell status.
The deadline is enforced by execd, not by the client: the timeout travels to
the server in `RunCommandOpts`, the SSE read timeout is disabled, and the
process group is reaped with SIGKILL. So a timed-out command does not raise --
it comes back as a `CommandExecError` result with the signal death reported as
exit code -1, which the adapter rendered as `Exit Code: -1`.
Normalize that pairing to the 124 marker, keeping the partial output the
command produced before the kill. The guard is deliberately narrow: a command
that exits non-zero on its own never carries the error event, and a failed
launch shares the same error event with no usable code, so neither is
relabelled a timeout.
Tests: three cases pin the deadline paths (client-side TimeoutError from a
bridge, the server-side -1 kill with output, and the free-text shape that must
stay on the existing missing-exit-code path as a start failure), plus a case
pinning that ordinary exits keep their own code.
RED/GREEN: the deadline cases fail on the unmodified source and pass with it;
test_opensandbox_provider.py 114 passed (110 pre-existing unchanged); the four
sandbox provider suites 601 passed; ruff check and format clean.
* fix(opensandbox): recognize explicit deadline results
Signed-off-by: yetuge <2219677952@qq.com>
---------
Signed-off-by: yetuge <2219677952@qq.com>
* fix(sandbox): truncate quarantine path keys to stay under Windows MAX_PATH
The quarantine nests three hashed path components (scope, sandbox id,
container id). Three full 64-char SHA-256 hexdigests exceed Windows'
260-char MAX_PATH once the quarantine root lives under a long path, so
mark() fails with ENOENT creating the generation record. Truncate the
digest to 16 hex chars, matching the sandbox-naming convention used
elsewhere in this module.
* test(sandbox): pin quarantine key truncation to 16 hex chars
Guard against a regression that reverts the key digest to a full 64-char
SHA-256 hexdigest: three nested 64-char components exceed Windows' 260-char
MAX_PATH and re-break record creation, while Linux CI still passes.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(client): preserve text appended before the first state snapshot
* test(client): exercise first-snapshot reconciliation with stable IDs
---------
Co-authored-by: BaiHeng Peng <191328900+Aldenshile@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): generalize IM group targets and add task/goal run notifications
- Generalize proactive group notification capability (proactive_group_notifications), binding validation, and scheduled task group selection across Feishu, QQ, and WeCom while keeping WeChat personal-only.
- Add QQ group active notification delivery via POST /v2/groups/{group_openid}/messages (msg_type=2) and WeCom group active notification delivery via WS Client.sendMessage(chatid, body) with bot_id:group:chatid workspace isolation.
- Generalize notification outbox sources via migration 0043_runtime_im_notices (source_kind, source_key, run_id, nullable task_id/task_run_id) with backward-compatible SQLite/PostgreSQL upgrades.
- Add opt-in ordinary task & goal run IM notifications (run_completed, run_failed, human_input_requested) on the shared outbox, delivery worker, renderer, and settings UI.
* fix(notifications): address review feedback and group/runtime notification edge cases
* test(migrations): update CURRENT_HEAD to 0043_runtime_im_notices across migration chain tests
* fix(channels): preserve group binding names and legacy authorization
Carry owner-chosen group names through existing connect state, keep private bindings out of legacy group fallback, and restore completion metadata retries. Cover real binding and worker finalization paths and synchronize notification docs.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(models): route gpt-6 tool calls through the OpenAI Responses API by default
The locked langchain-openai==1.2.1 routes gpt-6 models configured with
langchain_openai:ChatOpenAI through /v1/chat/completions, where OpenAI
rejects function tools for the gpt-6 family with an HTTP 400. Upstream
only added the gpt-6 + tools -> Responses API branch in 1.7.x, so
backport that routing decision at the model factory instead of bumping
the dependency (large uv.lock churn, wider regression surface).
_apply_openai_responses_api_default() injects use_responses_api: true +
output_version: responses/v1 for BaseChatOpenAI subclasses whose model
starts with gpt-6, unless the operator configured an explicit value.
Fixes#6841
* fix(models): narrow gpt-6 Responses API default per review
Addresses review findings on #6863:
- Custom endpoints (base_url/openai_api_base) are never touched, so
OpenAI-compatible gateways/proxies (LiteLLM, vLLM, one-api) keep their
chat-completions routing instead of being silently re-pointed at /responses.
- AzureChatOpenAI is skipped too (MRO-based, so env-configured Azure and any
subclass are covered): Azure's Responses API requires api-version >=
2025-04-01-preview.
- An explicit use_responses_api still always wins; a lone explicit
output_version is preserved while use_responses_api: true is still injected,
so gpt-6 no longer silently falls back to /v1/chat/completions.
- The default injection is logged so operators can trace the /responses switch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(models): treat env-var-wired gateways as custom endpoints for gpt-6
Addresses the remaining review findings on #6863:
- A gateway wired purely via OPENAI_BASE_URL/OPENAI_API_BASE (the openai SDK
fallback when no base_url is passed) is now treated like an explicit
endpoint, so a gpt-6 model behind an env-var-wired gateway is not silently
re-pointed at <gateway>/responses. The known api_base-subclass blind spot is
documented in the comment block.
- Regression tests pin the env-var endpoint gate and the explicit-wins
precedence (an explicit use_responses_api still wins on a custom endpoint).
* test(models): isolate Responses routing from endpoint environment
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(mcp): skip tasks a cancel released while the poll claim was in flight
claim_due_tasks selected due rows with FOR UPDATE SKIP LOCKED, then set
the lease through an ORM flush that updates by id alone. FOR UPDATE is a
no-op on SQLite, so a request_cancel that committed between the SELECT
and the flush had its cleared poll lease written back. The cancellation
worker only claims rows without a live lease, so the remote cancel waited
for mcp_tasks.lease_seconds (120 s by default), and the task was polled
once more for a result apply_snapshot then discarded.
The claim now selects candidate ids and leases each one with a
conditional UPDATE that re-checks the claim predicates (pollable status,
no cancel request, due next_poll_at, no live lease), returning only the
rows it actually leased, the same atomic fence release_claim already
uses. On Postgres the row lock still orders the two writers.
* docs(changelog): reference #6859 for the MCP task claim fence
* docs(mcp): note that a short poll claim batch is not topped up
* fix(knowledge): scope ragflow link cleanup to its native citation namespace
budget_source_artifact cleaned a delegated task summary with two regexes that
matched any #knowledge-... destination. When a task result carried both the
ragflow-native contract (#knowledge-<id>-<n>) and the unified contract from the
retrieval refactor (#knowledge-source-<id>), those regexes also flattened the
unified citation to plain text. budget_source_artifact only rebuilds the native
knowledge_sources, so the now-linkless unified evidence then failed the
prune_sources substring check and retrieval_sources was emptied. The native
citations in the same result survived, so a mixed result lost attribution for
its unified sources while keeping it for ragflow ones, even with budget headroom
to spare.
Anchor both regexes to the native namespace with a (?!source-) lookahead so they
leave #knowledge-source-<id> for prune_sources to reconcile downstream. Native
ids are 32-hex and never begin with "source-", so native link cleanup is
unchanged.
Add a regression that drives the real middleware entry point on a mixed-contract
task result and asserts the unified citation stays clickable and its
retrieval_sources record survives alongside the native one.
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
* fix(knowledge): remove dangling citation fragments from task summaries
---------
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(llm-error): retry provider errors reported inside a 200 stream
An OpenAI-compatible gateway that answers 200 early, to keep a slow
request alive with SSE keepalives, reports a later provider failure (a
504, say) as an SSE error event. The openai SDK raises that as the base
APIError, with no status code and no response, so _classify_error found
no status, no transient class name and no busy text, and called it
generic: the run failed on its first attempt.
An error with no status now takes an effective status from its own
contents, and the first source that yields one decides: a numeric code
on the error, or a numeric code, status or status_code in its body or
the body's error object; an explicit status in its message ([504],
HTTP 502, status 503); or type server_error (500). Its busy check also
reads each run of _, - and whitespace as one space, so rate-limit and
RATE_LIMIT_EXECUTION_TIMEOUT match "rate limit". Retry budget, backoff
and circuit breaker then treat it as the equivalent HTTP error. Errors
that carry a status are classified exactly as before, and the quota,
auth and burst-rate checks still run first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(llm-error): note how status-less errors are classified
Add one sentence to item 9 of the middlewares guide, and an llm entry
under Fixed in both changelogs, for the retry of provider errors
reported inside a 200 stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(llm-error): pin the in-band status source order and leftmost match
Two rules of _extract_in_band_status had no test: a numeric status
field decides before the message is read, and within the message the
leftmost in-range status wins. Either rule could be inverted with every
test still green. Add one stays-generic case for each: code 400 with
"[503]" in the message, and "HTTP 400 after [503]".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(llm-error): do not read an array index as an in-band status
The bracketed alternative of the in-band status regex had no lookbehind,
so `messages[502]` or `tools[0][503]` matched and their number decided
the error's status. The bracketed form now starts with the lookbehind
`(?<![\w\]])`: a `[` directly after a word character or a `]` opens an
index, not a status. The `http` and `status` alternatives are unchanged;
`[504]: ...` in running text still counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(llm-error): let an in-band status preempt the separator busy check
The separator-normalised busy check ran for every status-less error,
even when the in-band lookup had already found a status: "rate-limit
header malformed" with `code: 400` was retried as busy, while HTTP 400
is not. It now runs only when no source yields a status, so an in-band
4xx decides like an HTTP one. The same text with no status anywhere
still counts as busy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(llm-error): let only 4xx and 5xx values decide the in-band status
The in-band lookup accepted 100-599 values from its first two sources,
so a 1xx-3xx status could decide the classification: `HTTP 200` in a
message whose server_error type means 500, or `code: 200` alongside
`status: 503`. Only 400-599 now count: `_as_http_status` skips a
smaller value as it skips a non-numeric one, so the candidate loop
moves on, and an out-of-range message match is skipped so the scan
continues. An error that carries a status stays as it was, and the
server_error source is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(llm-error): describe the narrowed in-band status rules
The guide, the CHANGELOG and the CHANGELOG_zh now describe the three
narrowed rules: only a 4xx or 5xx value decides, an index such as
`messages[502]` is not a status, and the separator-matched busy check
runs only when no source yields a status. Only the sentences that
describe the affected forms changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(llm-error): pin the remaining in-band status guards
Cover the nested error.status_code field, a code above 599 that yields
to a later status field, a code written in non-ASCII digits, and an
in-band 4xx whose plain busy text still decides, as it does for an HTTP
4xx. These pin current behaviour; no code changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(llm-error): say which in-band fields are read on the error itself
Only `code` is read on the exception; `status` and `status_code` are
read in its body and in the body's `error` object.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): link the in-stream retry entry to #6897
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(llm-error): require a separator and ASCII digits in an in-band status
The message regex read `status503` and `statuscode503` as a status
because the separator after `status` was optional, and it read
Arabic-Indic or fullwidth digits because `\d` and `int()` accept every
Unicode decimal digit. A status-less error with such text was retried
as a 503.
The `status` form now needs spaces or tabs, or a `:` or `=`, before
the number, and all three forms read only ASCII digits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(llm-error): pin the retry of an in-stream error after partial output
An `error` event that follows content chunks is retried like one that
comes first: the whole call re-runs and its result holds only the new
answer, while the tokens the first attempt already streamed to the
callbacks stay delivered, as with a mid-stream `ReadError`. The new
test pins that outcome, and a comment in `_classify_error` notes it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Handle transient errors based on status codes
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(lark): retry the Windows install lock instead of failing after ~10s
msvcrt.LK_LOCK gives up after roughly ten seconds and surfaces contention
as Permission denied, while a real install (download + setup of the
lark/sandbox CLI) can hold the lock far longer. Retry the non-blocking
acquisition on the Windows fallback branch so concurrent Gateway workers
serialize instead of failing, mirroring the extensions manager's lock
introduced in #6020. The POSIX flock branch is unchanged.
Closes#6879
* test(lark): inject msvcrt stub with raising=False for Linux runners
The regression test patches the module-level ``msvcrt`` attribute, which only
exists on Windows (the POSIX branch imports ``fcntl`` instead), so
``monkeypatch.setattr`` with the default ``raising=True`` raised AttributeError
on Linux CI (backend-unit-tests shard 4).
Pass ``raising=False`` so the stub is injected on POSIX runners: ``fcntl`` is
patched to ``None`` first, and ``_exclusive_install_lock`` then takes the
Windows fallback branch against the fake. Windows behavior is unchanged.
Scope: backend/tests/test_lark_cli_install_lock.py, 1 line.
Verification: pytest tests/test_lark_cli_install_lock.py on Windows
(Python 3.12, pytest 9.0.3) -- 3 passed incl. the real-contention case;
Linux CI rerun is the authoritative check.
* fix(lark): harden and share advisory lock retries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
`web_search_tool` skipped non-object entries but returned the filtered list
even when it was empty, so a non-empty `results`/`entries`/`articles`
container holding no objects fell through to "No results found". That tells
the agent the query legitimately matched nothing instead of surfacing the
malformed payload -- the failure mode the sibling providers already contain
(brave #6550, groundroute #6603, tavily #6645, sofya #6719): a 200 body
wrapped around an upstream error page ends a research loop as "no hits".
Extract `_response_rows`, which treats a missing or null container as an
empty search and a container that is not a list, or a non-empty list holding
no objects, as the provider's format error, keeping non-object entries
skipped in order so valid rows survive. Both call sites (all three verticals
share one helper) now return the existing structured error.
RED/GREEN: the 7 new cases fail on main with "No results found" and pass
with the fix; test_serply_tools.py 34 passed, the five provider suites
(serply/serper/tavily/sofya/tencent_wsa) 284 passed, ruff check and format
clean.
* fix(persistence): compare safe_add_column drift as the database spells it
safe_add_column's drift check compared the model's generic type and
default spellings (String -> STRING, BigInteger -> BIGINTEGER,
server_default="0" -> "DefaultClause('0', ...)", sa.false()) with the
reflected dialect spellings (VARCHAR, BIGINT, Postgres TIMESTAMP, '0').
Only JSON/JSONB ever matched, so every column a revision found already
present was reported as drift:
- any database created before 0023 logged a runs.change_seq warning on
upgrade, because 0025 re-runs 0023's safe_add_column;
- a schema whose head stamp was lost (the empty branch commits create_all
before stamping) logged 29 warnings on SQLite and 36 on Postgres, burying
the real TEXT-vs-JSON drift the check exists to surface.
Render both sides through the migration connection's dialect before
comparing: types via TypeEngine.compile (parameters stripped, the
Postgres time-zone suffix kept), defaults by unwrapping DefaultClause,
compiling SQL expressions and stripping one layer of string quoting.
Types the dialect cannot render (SQLite NullType) stay "missing info".
* docs(changelog): reference #6865 for the drift comparator fix
* fix(persistence): keep an empty-string default distinct from no default
Stripping one layer of quotes turned a reflected DEFAULT '' into "", and
`return text or None` then collapsed it to None. A desired
server_default="" (passed as a bare "") also normalized to None, so
_check_column_drift stopped warning when a column had no default but the
model wanted '' (or the reverse).
Render a plain-string DefaultClause argument as the quoted literal the DDL
emits, and return the unquoted value even when it is empty. None now means
only "no default".
