* fix(community): normalize max_results before handing it to Firecrawl and fastCRW
* test(community): pin the explicit-null max_results shape
The parametrized `omitted` case reused None as its "key absent" marker, so
`extra["max_results"] = configured` never ran for None and the key-present,
value-null YAML shape never reached _coerce_max_results. Use a distinct
sentinel for absence and add an explicit-null case per provider.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(typesafe): expose instructions/criteria overrides in the risk-gate eval
The pre-enablement evaluation could not answer the question an operator
actually asks when the gates fail: is the miss/false-block a model limit or
a rubric limit? --instructions/--criteria-true/--criteria-false pass the
provider's rubric overrides through, so threshold, max_state_chars and the
rubric can be calibrated in one loop.
First online run (28 labeled cases, jev-latest): defaults failed
risky_misses_zero (cat ~/.aws/credentials at p=0.47) and
system_false_block_within_target (two long safe calls locally refused over
max_state_chars). With threshold 0.4, max_state_chars 8000 and a
criteria.false clarifying that long-but-fully-shown workspace writes are
local and reversible, every gate passed in two independent runs
(0 missed risky, 0.0% false block; risky floor 0.48, safe ceiling 0.28).
Network p95 passed at 505-620ms in three runs and failed once at 1023ms —
re-measure in the target environment before enablement.
* feat(typesafe): record the effective risk policy in the eval report
A custom-rubric run was indistinguishable from a default-policy run in the
recorded configuration, so its verdict could not be reproduced. Publish the
provider's built-in defaults and write the resolved instructions and both
criteria as report["policy"] alongside model and threshold.
* feat(typesafe): mark the eval policy on stdout and pin flag passthrough
Address the PR #6179 review notes: read the rubric-override flags by
direct attribute access so a plumbing mistake fails loudly instead of
silently evaluating the default policy, print policy=default|custom in
the report header so a custom-rubric run is distinguishable on the
console too, and pin the flag-to-provider passthrough with an offline
provider-construction test.
* fix(channels): send Discord typing indicators through the discord.py 2.x API
_start_typing() called channel.trigger_typing(), which discord.py removed
in 2.0; the project requires discord.py>=2.7.0. The typing loop swallowed
every exception, so each tick raised AttributeError and no indicator was
ever sent, and the tests never noticed because they mocked the removed
method on a SimpleNamespace.
Await channel.typing() instead (awaiting the returned Typing sends one
indicator), and log a failed tick at DEBUG rather than discarding it. The
existing typing-lifecycle tests now mock typing(), and new tests drive
_start_typing() against a fake shaped like the 2.x Messageable, which has
no trigger_typing.
* docs(changelog): reference #6138
* fix(channels): warn on a Discord typing loop's first failed indicator
Logging every failed typing tick at DEBUG left a permanently dead
indicator invisible at the default INFO level, the same silence that hid
the trigger_typing AttributeError. A bot that lost Send Messages in a
channel (403 on every tick) or is being rate limited could not be told
apart from a working one without enabling DEBUG.
Log the first failure of each typing loop at WARNING with the traceback,
and later ticks of that loop at DEBUG so a retrying loop cannot flood the
log. The level is per loop, so the next reply's loop warns again.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
terminate() is TerminateProcess on Windows: langgraph dev never runs its
shutdown hooks, so .langgraph_ops.pckl is never flushed and state does not
survive the restart the two cross-restart tests exercise. Create the child
with CREATE_NEW_PROCESS_GROUP and stop it via CTRL_BREAK_EVENT (handled by
uvicorn as SIGBREAK) on Windows, keeping terminate() on POSIX and a hard
kill as the timeout fallback.
Fixes#6196
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
* test(blocking-io): catch socket.getaddrinfo on the event loop
Blockbuster wraps socket.socket methods but not the module-level
resolver, so a synchronous hostname lookup on the loop passes the strict
gate. The SSRF guard in deerflow.community.url_safety resolves with
socket.getaddrinfo from async web tools, and the anchors added next stay
green against that unfixed code without this rule.
Add socket.getaddrinfo to _PROJECT_BLOCKING_RULES, scoped to app.* and
deerflow.* callers. loop.getaddrinfo and asyncio.to_thread resolve in a
worker thread and are not flagged. The full make test-blocking-io suite
stays green with the rule active (187 passed).
* fix(community): resolve SSRF-guarded hostnames off the event loop
validate_public_http_url resolves hostnames with blocking
socket.getaddrinfo. The crawl4ai and Browserless web_fetch, web_capture,
browser_navigate, the Gateway browser navigate route, and the Live
stream's navigate input and seed all called it directly from async code,
so a slow DNS answer for a model- or user-chosen URL stalled the Gateway
event loop for the whole lookup.
Run the guard through asyncio.to_thread at each of those call sites, the
same way personal_network already offloads it. The screening result is
unchanged. fastcrw's web_fetch is a sync tool and personal MCP validation
already runs in a worker, so neither needed a change.
* docs(changelog): reference #6140 in the SSRF guard event-loop entry
* fix(browser): run the Playwright request guard off the shared browser loop
BrowserSession._install_request_guard screens every redirect hop, iframe
and subresource by calling the SSRF guard inside the async route
callback. The guard resolves hostnames with blocking socket.getaddrinfo,
and that callback runs on the process-wide Playwright loop, which also
pumps Live frames and input for every browser session, so one slow
lookup stalled all of them.
Run the guard through asyncio.to_thread there as well, and pin it with a
strict-gate anchor that drives the real route handler. Also note in the
url_safety docstring that the screen is check-time only: a caller that
re-resolves at connect time is not pinned to the vetted IPs.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
`_can_send_rich` checked `len(text)`, a code-point count, against the 32768
rich-message cap, while Telegram measures that cap in UTF-16 code units like
every other limit it enforces. An emoji-heavy reply that fits by code points
is still over the cap by code units, so the rich send was attempted, rejected
by the API, and the reply degraded to chunked plain text.
The check now reuses `_first_utf16_chunk`, the bounded scan the plain-text path
already uses, so eligibility costs the limit rather than the whole reply.
- measure the link-scan benchmark with perf_counter: monotonic ticks at
~15.6 ms on Windows and the linear scan finishes inside one tick, so
the ratio assertion divided by exactly 0.0;
- guard the owner-only mode assertion on Windows, where chmod cannot
express 0o600 (same class as #5932);
- pin strictly increasing updated_at values in the shelf-ordering test:
datetime.now(UTC) has ~15.6 ms granularity on Windows, so rapid inserts
can share one timestamp and the id tiebreak reorders the entries.
Fixes#6166
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
Memory router kept its own bare drain wrapper (_run_memory_mutation,
five call sites) after #6151 hoisted the superset into
app.gateway.persistent_writes.run_drained_write - this is the remaining
per-router fork called out in that review.
Each call site passes the domain errors its handler already maps
(NotImplementedError 501s, ValueError/KeyError 400/404/409 mappings,
MemoryConflictError/MemoryCorruptionError, OSError 500s), so the helper
stays silent on them exactly as the removed wrapper did; unexpected
failures gain the type-only lost-failure log line per the AGENTS.md
rule. Behavior is otherwise unchanged: same off-thread execution and
cancel-drain semantics.
The router test's drain stub moves to the shared name and pins the
clear-memory error tuple.
Signed-off-by: yetuge <2219677952@qq.com>
* docs: add extension-first evaluation gate to agent guidance
Require AI coding agents to evaluate whether a feature can ship as a
packaged extension before editing core code: route self-contained
capabilities to the extension-api contribution points (middleware,
observers, services, routers, full-stack plugins), keep bug fixes and
agent-loop/memory/compaction/auth changes in core, and prefer adding a
generic hook over hard-coding business logic when existing hooks fall
short. Mirrored in compact form in copilot-instructions.md.
* docs: fold extension gate into extensions paragraph, trim dated notes
The agent-guidance CI hard limit (98304 bytes per AGENTS.md chain) left
only ~53 bytes of headroom on the root file, so the gate moves from a new
Cross-Cutting Conventions bullet into the existing extensions paragraph,
and the dated skill-review and scheduled-task notes are compressed to
their non-discoverable rules. Net effect: root AGENTS.md is ~190 bytes
smaller than main.
* feat(community): add bounded retries for Jina web fetch
Closes#6116
Keep retries opt-in and share a total deadline across requests and backoff. Preserve cancellation and the existing Error contract. Include offline DNS and local-wheel fixture corrections needed to run the complete backend suite in the isolated verifier.
Signed-off-by: 97three <2212371308@qq.com>
* docs(community): scope Jina retry guidance to its provider
Signed-off-by: 97three <2212371308@qq.com>
* fix(community): jitter bounded Jina retry backoff
* docs(community): apply provider-scoped Jina guidance
Signed-off-by: 97three <2212371308@qq.com>
---------
Signed-off-by: 97three <2212371308@qq.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Checkpoint durable refs for host-externalized tool results and materialize verified bytes before model calls on another Gateway.
Refs: #4189
Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
* fix(nginx): canonicalize the loopback origin onto localhost
localhost and 127.0.0.1 (or [::1]) are the same loopback listener but
separate cookie jars to a browser, so an admin who sets up on one spelling
lands on /login when they later open the other, with no hint why (#6156).
Cookies cannot bridge the gap: no Domain attribute spans both hosts.
All three nginx configs now map the numeric spellings to "localhost", the
name every banner and doc already prints, and the main server answers a 301
to the same URL on the canonical host. The published port survives because
the map keys on $http_host and reattaches the captured port; Docker's
arbitrary host-port mapping is unaffected. Safe methods only: GET/HEAD
redirect, while API POSTs pass through untouched, since a redirected POST
strands clients that do not re-send bodies. Non-loopback Hosts (LAN names,
ingress domains) never match, and the canonical target itself is not a
match, so there is no redirect loop.
The new suite parses both maps out of each config and evaluates the full
method x host matrix against nginx semantics, including the suffix-spoof
and redirect-loop cases, and pins the maps identical across the Docker,
local-dev, and Helm configs so they cannot drift.
Fixes#6156
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* docs: keep the loopback note in the nginx configs
The root AGENTS.md paragraph pushed the middlewares and sandbox
instruction chains over the 96 KiB hard limit. The nginx configs
already carry the full explanation next to the maps, so drop the
duplicated paragraph instead of shrinking the budgets.
* fix(nginx): let WebSocket upgrades bypass the loopback canonical redirect
willem-bd caught this running the stack locally: the canonicalization map
keyed on method:host alone, so a GET with an Upgrade header got the same
301 as a plain GET. WebSocket clients do not follow redirects, so every
WS handshake against a 127.0.0.1/[::1] spelling died at the redirect.
Key the origin map on method:upgrade:host instead. Handshakes carry a
non-empty Upgrade value, fall off the GET::/HEAD:: patterns, and proxy
through untouched. Plain GET/HEAD behavior is unchanged.
---------
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Both the en and zh integration guides show `from deerflow.app.gateway.main import
app`, but there is no deerflow.app package and no gateway/main.py in the tree: the
FastAPI instance is created by create_app() and bound at module level in
backend/app/gateway/app.py, which is also how backend/AGENTS.md documents it.
The copy-pasteable snippet now names that module, so mounting the Gateway as a
sub-application works from the documented import instead of failing at runtime.
* test(extensions): make the extension-manager suite runnable on Windows hosts and under host uv mirrors
- use the file's existing platform-aware .venv interpreter idiom in the two
rollback tests (Scripts/python.exe on nt);
- compare the uv add source argument as a Path so the assertion is
separator-neutral;
- pin UV_DEFAULT_INDEX in an autouse fixture so host mirror configuration
cannot break the source-build tests; tests installing their own local
index keep precedence.
Fixes#6141
* test(extensions): drop the extra-index env channels from the suite environment
UV_INDEX and UV_EXTRA_INDEX_URL entries take priority over the pinned
default index and could still break the source-build tests. UV_NO_CONFIG
is intentionally not used: it would also drop the fixture's own
[tool.uv] pyproject settings the command-shape assertions rely on.
---------
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
* refactor(gateway): hoist the persistent-write drain into one shared helper
Each persistent-write surface forked its own per-router drain wrapper:
_drained_write in routers/agents.py (action label, expected_errors
pass-through, lost-failure type-only logging), _run_store_mutation in
routers/subagents.py (bare drain), and the inline await_drained call in
routers/managed_models.py. Hoist the superset into
app.gateway.persistent_writes.run_drained_write and point all three
routers at it, per the review suggestion on #6087 (its family
predecessors #6078/#6092/#6093 have since landed).
The subagents and managed-models call sites now also emit the type-only
lost-failure log line on a drained-then-failed write (previously
silent), matching the tradeoff acknowledged in the #6087 review.
artifacts.py and personal_mcp.py keep their direct await_drained forms:
the former drains a coroutine with its own rollback flow, the latter
needs no label or expected-error handling.
Router tests move their monkeypatch targets to the shared name; the
lost-failure assertion pins the helper's logger. AGENTS.md points the
next persistent-write surface at the canonical implementation.
Signed-off-by: yetuge <2219677952@qq.com>
* fix(gateway): quiet expected domain errors and scope the AGENTS.md drain claim
Review follow-ups on #6151:
- The subagents create/update call sites pass (ManagedSubagentExistsError,)
and (FileNotFoundError,) so the shared helper stays silent on routine
4xx outcomes (duplicate-name 409, stale-store 404 race), matching
create_agent's (AgentExistsError,).
- The managed-models save passes (HTTPException,), so the helper stops
double-logging failures that _save_with_failure_logging already
reports and stays silent on its routine 4xx outcomes.
- The AGENTS.md sentence now names the three converted routers instead
of implying every persistent-write surface uses the helper; the
shortened form also keeps the file under the guidance byte budget
that agent-guidance enforces.
Signed-off-by: yetuge <2219677952@qq.com>
---------
Signed-off-by: yetuge <2219677952@qq.com>
* feat(authz): extension tool authorization PR2 — middleware-declared tool path
Layer 1 now covers tools declared by middlewares, which LangChain merges
into the bound tool set after the host's explicit-list filter:
- new agents/middlewares/tool_declarations.py: record the ordinary pass's
verdict, collect declarations from the built stack, decide only the
delta seeded by the first pass (a name denied for this build stays
denied), narrow the stack build-locally on independent state-preserving
copies (direct __dict__ write, refused up front for tools data
descriptors, never mutating caller-owned instances), and verify the
exact bound stack after the final normalization copy.
- IsolatedMiddleware ships a state-preserving __copy__ so narrowing
survives full/delta normalization and copies of copies.
- TodoMiddleware degrades when write_todos is denied: no system-prompt
injection, no completion/context-loss reminders, no jump_to.
- Layer 2 forwards host-resolved tool provenance into AuthzRequest.context
and binds the infrastructure exemption to the concrete host-created
tool object instead of a name set (tool=None goes to the provider).
- Wired into all four assembly paths (lead x2, native subagent with the
decision offloaded via asyncio.to_thread, DeerFlowClient).
Tests: new test_middleware_declared_tools.py (35) and
test_layer2_tool_provenance.py (7); blocking-IO anchor pinning the
subagent offload (teeth verified red->green); placement-guarantee and
enforcement suites extended; executor test doubles converted for the
async _create_agent. Behavior change called out in CHANGELOG: with
authorization enabled and a tools policy that excludes write_todos,
plan-mode builds no longer bind it (built-in RBAC default unchanged).
* fix(authz): close residual gaps from review — narrow non-collectable declarations, dedupe candidates, warn on skipped pass
- apply_declared_tool_view now removes non-BaseTool declarations from the
bound view when authorization is enabled (they would otherwise bind via
LangChain's create_tool auto-conversion with no provider decision);
verify_declared_tool_view fails loudly if one survives normalization
- decide_declared_tools dedupes never-submitted names so the provider sees
each candidate once
- subagent executor logs a warning when the provider is set but the seeded
Layer-1 state is missing, instead of skipping the declaration pass silently
- docs: middleware-declared tools must be BaseTool instances with usable names
* fix(authz): fail loudly on non-list/tuple middleware tools declarations
_iter_declared warned and returned () for a 'tools' attribute that is not
a list/tuple, but LangChain's factory iterates the attribute as-is — so a
set/generator/dict_values declaration bound with no Layer-1 decision.
Raise DeclaredToolViewError instead, matching the module's fail-loud
standard for every other un-narrowable shape. Disabled path remains a
strict no-op. Docs updated to match.
* feat(persistence): add notification delivery outbox for scheduled task IM push
New notification_deliveries table and repository (migration 0012) backing
the outbox pattern for issue #4254: idempotent enqueue keyed on task_run_id
plus event plus provider plus target, atomic claim of due rows, and
exponential-backoff retry state moving pending to sending to sent or failed.
Execution status stays in scheduled_task_runs; delivery status lives here.
* feat(scheduler): enqueue and deliver scheduled-task notifications over IM channels
Closes the loop for issue #4254. On run completion the scheduler writes
outbox rows for the task owner's connected channel identities (success and
failure events; interrupts stay silent), and a new NotificationDeliveryWorker
polls due rows and pushes them through the channel's proactive send path.
Channel gains send_notification; WeCom implements it via the WS client's
send_message with errcode checking, the base class raises for providers
without proactive push.
Delivery is best-effort and never shadows execution status: enqueue and send
failures are logged and recorded in the outbox, which retries with backoff.
Channel liveness is re-checked at delivery time, and the worker stops before
the channel service on shutdown. Feature activates only when channel
connections are enabled; no new dependencies.
Verified end to end against a live WeCom bot: scheduled run completion
produced an outbox row that was delivered in under one second.
* feat(scheduler): include scheduled run result summary in IM notifications
The v1 notification only carried the run outcome skeleton (status, task id,
run id). Completed runs now include the agent's final answer so the IM push
is actually useful headlessly.
- The delivery worker accepts an optional resolve_run_summary(run_id,
user_id) resolver; for run_completed events it enriches the rendered text
with a "Result:" line (bounded to 1000 chars). The lookup is best-effort:
resolver failures or missing summaries degrade to the skeleton text and
never block or fail the delivery.
- The summary is read at delivery time from runs.last_ai_message (the
convenience column written on run completion), not snapshotted into the
outbox payload; retried deliveries re-read the freshest value and the
outbox row payload stays skeleton-only. The claimed outbox row is never
mutated in place.
- Gateway wiring scopes the read to the outbox row's owner_user_id so a
stale row cannot pull another user's run content.
- Failed runs keep the error-only skeleton: a partial answer on a failed
run would be misleading.
Tests: 9 new worker cases (rendering incl. truncation, sync/async
resolvers, failure/missing fallbacks, no resolver call for failed runs).
Full scheduled + outbox + channels regression green (441 passed), ruff
check + format clean.
* fix(scheduler): address review feedback on notification delivery reliability
Review follow-ups for the scheduled-task IM notification outbox:
1. Crash recovery for "sending" rows: claim now stamps updated_at and the
worker reconciles rows stuck in "sending" past a 10-minute lease back to
"pending" before every claim. Per-row try/except isolates a crashing
delivery from the rest of the claimed batch.
2. Double-send race under multiple workers: claim returns exactly the rows
its own UPDATE...RETURNING flipped (guarded by status='pending'), so a
rival worker's concurrent claim can no longer be handed out twice.
3. Restore non-ASCII comments and drop the BOM accidentally introduced in
test_scheduled_task_service.py.
4. Manual "run now" completions no longer enqueue IM notifications; the
user is already watching the UI. Missing trigger metadata fails safe
towards notifying.
5. Channel-outage failures no longer consume the retry budget: they are
parked with count_attempt=False on a flat 15-minute backoff so an
hours-long outage cannot exhaust retries and drop the notification.
Tests: repository 16, worker 21, scheduled-task service 31; targeted
regression across notification/scheduler/channel suites: 511 passed.
* fix(scheduler): address round-2 review on IM notification delivery
Park WeCom transport/outage failures without burning retries, bound worker stop, include task titles, align enqueue with delivery, and document the IM push behavior.
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(scheduler): complete IM notification docs and drop unused list API
Add backend/channels/migrations AGENTS notes, CHANGELOG and config.example hints for the outbox push path, and remove the unused list_by_task_run repository method flagged in review.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(scheduler): address round-3 review on IM notification delivery
Cap channel-down parking by row age and parked attempts, redact IM egress
content, narrow WeCom transport exceptions, add lifespan wiring tests, and
run ruff format so lint-backend passes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(scheduler): redact entire PEM blocks in IM egress text
The previous BEGIN-only pattern left key material and the END marker in notifications.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(persistence): align 0014 parked_attempts with bootstrap schema tests
Backfill existing rows with a temporary server default, then drop it so create_all matches Alembic, and move HEAD anchors to 0014.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(persistence): format notification tests and shorten alembic revision id
ruff format --check failed on test_notification_delivery_worker.py.
0015_notification_delivery_parked_attempts exceeds alembic_version.version_num
(VARCHAR 32) and truncates on Postgres CI; use 0015_parked_attempts instead.
* test: fix post-merge CI failures after migration renumber
Update migration 0015 test HEAD expectation to 0017_parked_attempts and add queue_timeout_seconds to the lifespan notification worker mock scheduler config.
* fix(persistence): chain the notification outbox migrations after 0026
Main's head moved from 0016 to 0026_mcp_task_lease_tokens, so the two
outbox revisions are renumbered and re-parented:
- 0017_notification_deliveries -> 0027_notification_deliveries
- 0018_parked_attempts -> 0028_parked_attempts
Only the revision ids, parents and the migration guide change.
* fix(scheduler): adapt the notification hook and its docs to main
- tests: the completion tests assert on the atomic complete_run record;
new cases cover a completion that was not recorded, a task deleted
meanwhile, a failed task read, and the outbox being off.
- AGENTS.md: drop the root note and shorten the backend note to a pointer;
the two notes pushed two effective guidance chains past the 96 KiB limit.
- CHANGELOG: move the entry from the released 2.1.0 section to Unreleased
and add its link references.
* test(persistence): move the chain-head pin to the outbox migrations
test_migration_0026 pinned 0026 as the head, so it failed once the outbox
revisions chained after it. It now checks chain membership like the older
migration tests, and a new test file owns the head pin, checks the parents
and the 32-character revision id limit, and runs the upgrade and downgrade.
* fix(gateway): wire scheduled-task notifications after the channel service starts
The outbox wiring sat in the scheduler block and asked for the running
channel service there. #5035 moved the channel service start to after the
scheduler, and the two changes merged without a conflict, so on every
startup the lookup returned None and the feature disabled itself with the
"write-only outbox" warning. The lifespan tests patched the lookup to a
constant and kept passing.
The wiring now runs in its own step after the channel service start. The
scheduler is built as on main, and ScheduledTaskService gains
attach_notification_outbox() so the enqueue side is enabled once the
delivery worker is running. The guard, the worker and the shutdown order
are unchanged.
Tests: the lifespan helper can model the real accessor (None until the
channel service has started); a new case fails without this fix.
* fix(scheduler): enqueue from scheduler start and keep the push when the title read fails
Follow-up to the startup-order fix, from review.
- The enqueue side is wired with the scheduler again (it only needs the
durable table), and only the delivery worker waits for the channel
service. Attaching it later left a window after the scheduler started in
which a finished run wrote no outbox row. If the channel service is
missing or the worker cannot start, the enqueue side is detached, so the
outbox still never becomes write-only.
- A failed task read after complete_run no longer drops the notification:
the title is optional and the message falls back to the task id. A task
deleted meanwhile still stays silent.
* fix(channels): park WeCom notifications while the SDK socket is closed
The WeCom SDK raises a plain RuntimeError when its socket is not open, and
send_notification only mapped ConnectionError, TimeoutError and OSError to
ChannelUnavailable. A disconnect therefore spent the counted retry budget
and settled the row as failed after about 15 minutes, although the docs
say transport outages park without consuming retries.
send_notification now asks the client's is_connected first. The new test
uses a real, never-connected SDK client: it raised RuntimeError after three
retries before, and parks now.
* docs(scheduler): describe which outcomes notify; mirror the changelog entry
- README (en/zh) and CONFIGURATION.md: only runs that finish as success or
failed notify; occurrences that end without a finished run (launch error,
queue timeout, restart recovery) do not. Parking lasts up to about a day.
- CHANGELOG: cite the issue as text like the rest of the file, and add the
entry and link reference to CHANGELOG_zh.md.
- AGENTS.md: drop the backend pointer so this branch adds nothing to the
guidance chains closest to their size limit; the scheduler-side rules
and the lifespan order now live in app/channels/AGENTS.md.
* docs(changelog): cite #6135 next to #4843 for the notification outbox entry
* fix(scheduler,channels): re-check the IM binding at delivery time; park WeCom transport errors, count its rejections
Review round 1 on #6135.
The delivery worker now takes resolve_connections (the Gateway wires
ChannelConnectionRepository.list_connections) and, before the channel
liveness check and the summary read, requires the row's owner to still
hold a connected (provider, external_account_id). A disconnected target
is finalized with mark_failed(terminal=True); a lookup failure or an
unexpected result shape parks the row. Terminal outbox rows ignore
later status writes.
WeCom: the SDK raises RuntimeError from send_message for a socket that
is not open and for a dropped connection, but also for a platform
rejection ("Reply ack error: errcode=..."). Only that prefix keeps
counting; other RuntimeErrors, ConnectionError/TimeoutError/OSError and
websockets' ConnectionClosed park the row as ChannelUnavailable.
* fix(channels): treat a WeCom ACK rejection as proof the transport is up; pin the rejection prefix through the retry branch and the real connection-row shape
---------
Co-authored-by: GaoTianyuanhr <75788039+GaoTianyuanhr@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: GaoTianyuanhr <GaoTianyuanhr@users.noreply.github.com>
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* fix(middlewares): include file line count in read-before-write block message and clarify read_file range error
* fix(middlewares): address review feedback on line count pluralization and readability
* fix(middlewares): address round-2 review feedback for PR #6020
Stamping bug (_attach_read_mark, ggnnggez item 1):
Originally planned as a standalone issue; folded here since it is the exact mechanism behind the #6019 gate bypass. _attach_read_mark now returns early when message.content is in READ_FILE_NO_CONTENT_RESULTS - {READ_FILE_EMPTY}. In-band errors (invalid range, start_line exceeds file length, etc.) are returned with status='success' by LangChain so the existing message.status == 'error' guard did not catch them.
HTTP code collision [P3] (willem-bd):
Pre-stamp TOOL_META_KEY on the blocked ToolMessage before normalize_tool_result so files with 401, 403, 404, or 500 lines are not misclassified as auth/server errors by _classify_error_text. normalize_tool_message returns early when an existing TOOL_META_KEY entry is found.
Line-count parity (ggnnggez, inline L317):
Replace splitlines() with count_file_lines() in read_file_contract.py using count('\n'), matching LocalSandbox text-mode iteration and the _truncate_read_file_output formula. splitlines() over-counts for files with \f or \u2028 separators (e.g. pdftotext output).
Read hint (ggnnggez, inline _BLOCK_MESSAGE):
For empty files the ranged-read suggestion is omitted. For non-empty files the hint gives a concrete start_line=max(1, N-29), end_line=N.
Constant rename (ggnnggez, read_file_contract.py):
READ_FILE_INVALID_RANGE is the new canonical name; READ_FILE_EMPTY_RANGE kept as a backward-compat alias. _LEGACY_READ_FILE_EMPTY_RANGE removed (unreachable after the #6019 contract update).
Tests:
- TestBlockMessageLineCount parametrized with \f and \u2028 parity cases
- TestStampingGateBugFix: 4 no-content variants, gate-closed integration probe, empty-file-still-stamps assertion
- test_gate_block_on_numeric_line_count_is_recoverable: 401/403/404/500
- test_contract_constants_membership replaced with test_contract_constants_parity_with_localsandbox
---
NOT INCLUDED: ggnnggez item 2 (file length in read_file range error strings). READ_FILE_NO_CONTENT_RESULTS uses exact string membership and is also consumed by build_skill_usage for skill-read detection; making those strings dynamic requires a dedicated contract refactor. Deferred to a follow-up PR.
Note: test_web_fetch_relative_links failures in the full suite are pre-existing on upstream/main and unrelated to this change.
* test: clarify blank ranged reads in the write gate
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(middleware): publish externalized outputs atomically
_externalize reported failure with None but had already created the file, so a
write that failed part-way (disk full, interrupted request) left a truncated
file under the final name in the outputs directory. Write to a sibling temp file
and os.replace it into place, removing the temp file when the write fails.
* style: drop the unused pytest import from the externalize test
* test(scripts): cover discord extra auto-detection
The discord mapping in scripts/detect_uv_extras.py worked but was neither
listed in the "currently maps" docstring nor covered by a test, unlike the
buzz mapping that grew up beside it.
* fix(channels): point the Discord missing-dependency hint at the discord extra
discord.py is the only IM transport that is not a core dependency — it ships in
the optional `discord` extra, which detect_uv_extras.py, backend/Dockerfile and
deploy.sh already expand. Its ImportError hint told users to run `uv add
discord.py` instead, which rewrites pyproject.toml/uv.lock in a tree every
restart syncs with `--locked`. Mirror the buzz hint wording and note the extra
in config.example.yaml next to the buzz block.
---------
Co-authored-by: huyengiang101086-spec <270684046+huyengiang101086-spec@users.noreply.github.com>
* fix(skills): drain custom-skill mutation tails across cancellation
* fix(skills): log drained skill mutation failures and drain state/install tails
Address review feedback on #6078: record cancelled-then-failed mutation
tails inside the drained unit (exception type only, per the managed-models
precedent), and extend the drained tail to the remaining write-to-cache
shapes in update_skill and the skill install path.
* test(skills): pin the lock contract under the drained cancellation timing
The cancelled update_skill caller now drains its mutation tail, so
CancelledError surfaces only after the tail settles. Start the MCP
writer without awaiting the cancelled task first, assert the section
stays owned while the drained tail is in flight, then await both and
assert the final order.
---------
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
* fix(agents): keep the todo completion reminder when a model call is retried
TodoMiddleware drained its queued completion reminder inside
wrap_model_call before calling the handler. LLMErrorHandlingMiddleware
sits outside it and retries a failed call by running that wrap again, so
the retry went out with an empty queue while the run had already spent
one of its two reminders on the lost one.
Drain the reminder once per wrap, and when the handler raises put it
back in front of the queue without counting it again, then re-raise. A
run whose reminder state was cleared while the call was in flight does
not get it back.
* docs(changelog): reference #6132 in the todo reminder retry entry
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): split Telegram messages by UTF-16 code units, not code points
* perf(telegram): clip stream previews without walking the whole reply
Every cumulative stream update measured the full answer and then split the full
answer to keep only its first chunk, so a long reply cost two complete passes
per update on the async channel path. _first_utf16_chunk stops at the first
character that does not fit, and the preview is built from that bounded prefix.
The tests now measure chunk length with the UTF-16 codec instead of a copy of
the production formula, cover the width helper and the bounded clip directly,
and fail if the preview stops being bounded by the limit.
* fix(uploads): reject filenames reserved for temporary staging
Guard thread and shelf upload names, validate SDK batches before copying, and preserve legacy shelf downloads. Add regression coverage and document the reserved staging namespace.
* fix(uploads): report reserved names before HTTP batch ingestion
Preflight reserved staging basenames before thread storage or sandbox setup. Return HTTP 400 with a rename hint; add HTTP and strict blocking-I/O regressions and update upload documentation.
* fix(uploads): reject Windows-style staging basenames on POSIX
Normalize backslashes before the HTTP reserved-name preflight. Add POSIX basename regressions for single and mixed batches and strict blocking-I/O coverage. Clarify the preflight scope and upload documentation.
Follow-up to the review of PR #6122; AI-assisted with OpenAI Codex.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): keep blocking SDK teardown off the event loop in stop()
SlackChannel.stop() called SocketModeClient.close() inline. close() joins
the SDK's message-processor thread (~0.7s every call, measured) and waits
for in-flight listeners, whose blocking Web API calls can run up to the
WebClient timeout, so every run, stream and channel on the Gateway stalled
during shutdown and POST /api/channels/{name}/restart.
Feishu and DingTalk had the same shape: they joined their SDK threads
inline with a 5s timeout, and since those threads only return on a fatal
error, the join normally froze the loop for the full 5s.
The teardown now runs through asyncio.to_thread. Slack detaches the client
first so a still-queued connect skips it, and shields and tracks the close
task so a cancelled stop() leaves it running and a retried stop() awaits
the same close instead of closing twice.
tests/blocking_io/test_channel_stop_offloop.py anchors all three stop()
paths to the strict Blockbuster gate, driving a real slack-sdk client.
* docs(changelog): reference #6134
* fix(channels): join the Discord client thread off the event loop in stop()
DiscordChannel.stop() still joined its client thread inline with a 10s
timeout. The thread usually exits right after the cross-loop client close,
but a timed-out close or a slow _run_client() drain keeps it alive for the
whole timeout, stalling the shared Gateway loop the same way the Slack,
Feishu and DingTalk teardown did. Run the join through asyncio.to_thread so
stop() matches the channels/AGENTS.md teardown rule, and add Discord to the
strict Blockbuster anchor.
extensionMap is keyed by extension and valued by language, but the Julia row
was written backwards as `julia: "jl"`. checkCodeFile tests membership with
`extension in extensionMap`, so ".jl" never matched and a Julia artifact was
not a code file at all, while ".julia" resolved to a language named "jl".
Add both `jl: "julia"` and `julia: "julia"`, the same pair form the map already
uses for haskell/hs and elixir/ex, so no extension that classified before stops
classifying.
* fix(paths): reject Windows reserved names on host-visible paths
Uploads, skill support files, and sandbox path checks now share one helper that rejects CON, PRN, AUX, NUL, COM1-COM9, and LPT1-LPT9 (including names like con.txt) and components that end in a dot or space. The check is not limited to Windows so a Linux host cannot store a name that breaks when the same tree is used on Windows.
* fix(paths): validate quoted local bash paths as whole words
The absolute-path scan stopped at whitespace, so a quoted upload name such as report. final.txt or CON notes.txt was rejected as a truncated segment. Local bash checks now use the shlex argument for traversal and Windows-incompatible segments.
* fix(paths): do not block reads of existing host names
* style(paths): format with ruff 0.15.12
* fix(frontend): strip closing hashes from web fetch titles
Signed-off-by: 97three <2212371308@qq.com>
* fix(frontend): scan web fetch heading suffixes in linear time
Signed-off-by: 97three <2212371308@qq.com>
---------
Signed-off-by: 97three <2212371308@qq.com>
* fix(middleware): validate exact byte size on sandbox tool output externalization
* test(middleware): simulate partial write in sandbox externalization size test
* Fix PR number in CHANGELOG.md
Updated the changelog to reflect the correct issue number for a truncated file write.
* Update CHANGELOG_zh.md for issue #6112
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(threads): re-anchor re-persisted history rows to their earliest seq
reconcileThreadHistoryRows collapsed a repeated (run_id, identity) to
its latest-seq copy, deleting the earliest-seq row before
buildVisibleHistoryMessages could apply its earliest-seq-wins
anchoring. A message re-persisted under a newer seq therefore jumped
to the tail of the thread, contradicting the documented contract in
this function's docstring, in buildVisibleHistoryMessages, and in the
backend get_message_seqs earliest-seq-wins rule.
Re-anchor the surviving row to the earliest seq its identity held in
the merged window after deduping, and make the retained-history
stability check value-based so re-anchored rows do not force a state
update on every pass.
Fixes#6125
* fix(threads): address history reconciliation review
---------
Co-authored-by: liwenjie200543 <175601277+liwenjie200543@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(vllm): fall back to legacy reasoning_content field
* fix(vllm): document the reasoning_content fallback and pin precedence
The previous change set left the docs describing the provider as
preserving only `reasoning`:
- the module docstring now mentions the legacy `reasoning_content`
wire field, and the vLLM Provider bullet in models/AGENTS.md is
extended to match the repo's documentation-update policy;
- _create_chat_result evaluated choice["message"] twice, once per key;
hoist it into a single local and read both keys from it;
- two regression tests pin that a payload carrying both fields keeps
`reasoning`, so a future refactor cannot silently invert the
precedence (the existing tests only covered the absent-`reasoning`
case).
Reported-by: willem-bd
* refactor(vllm): share reasoning fallback and cover follow-up payloads
---------
Co-authored-by: inchang-ing <197932532+inchang-ing@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Mirror the English README's Startup Modes section (introduced and refined in #5995) into the Chinese version: the startup mode matrix, the stop/restart commands, and the SKIP_FRONTEND_BUILD opt-in note. README_zh previously had no documented way to run services in daemon or production mode or to stop/restart them.
Co-authored-by: sjr666666 <204130665+sjr666666@users.noreply.github.com>
* fix(auth): reject a non-ASCII ID token nonce instead of raising
The nonce comparison in validate_id_token still used the local
secrets.compare_digest on str operands, which raises TypeError when either
side holds non-ASCII characters. The nonce claim is provider-controlled
text, so such a provider response turned the callback's sso_failed redirect
(OIDCValidationError) into an unhandled 500.
Route it through the shared constant_time_equals helper #6076 introduced
for the same five compare sites, and drop the now-unused local helper.
Signed-off-by: yetuge <2219677952@qq.com>
* fix(auth): reject a non-string ID token nonce claim like a mismatch
A provider returning a truthy non-string nonce claim (e.g. an int)
slipped past the missing-claim guard and crashed the comparison helper
with AttributeError, which the callback turned into a 500 instead of
the sso_failed redirect. Guard the claim type before comparing so every
malformed nonce takes the same OIDCValidationError path.
Signed-off-by: yetuge <2219677952@qq.com>
---------
Signed-off-by: yetuge <2219677952@qq.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
