* fix(goal): wait for the user when a turn ends on an unanswered question
ask_clarification and the sandbox network prompt put their question in a
ToolMessage and end the graph. The goal evaluator only reads human and AI text,
so it never saw the question, judged the goal not met, and the worker queued a
hidden continuation telling the agent to keep going while the card was still
open. The agent could then act on a guess before the user answered.
Stand the goal down with blocker needs_user_input, without calling the
evaluator, when the trailing tool results include a human input request.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(goal): cover resuming after answered clarification
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(frontend): reference conversations from the composer
Adds a "Reference a conversation" button next to the attachment button,
shown only while GET /api/features reports read_conversation enabled. It
opens a picker over the recent-conversation list (current thread excluded,
capped at max_references) and shows removable chips in the composer.
On send the thread IDs ride SendMessageOptions.conversationReferences into
run context.conversation_references, which the Gateway consumes at
admission; the LangGraph SDK drops unknown top-level body fields. A
display-only copy ({thread_id, title}) on the visible human message lets
the transcript render read-only chips linking to the source.
References are per message: not persisted with the draft and cleared on
send or thread switch; regenerating or editing a turn runs without them
unless they are attached again.
Related to #5398. Depends on #5463.
* fix(frontend): pin the run-context contract and finish the picker states
Both thread.submit paths now build their run context through one exported
buildRunContext helper, tested directly: attached references travel as a
plain string[] under context.conversation_references only when the caller
passed them, a stray key in local settings is dropped instead of forwarded,
and the regenerate/edit replay path never carries references.
The picker shows a loading row while the conversation list is still in
flight instead of claiming there are no conversations, and the transcript
chip group is labelled with the previously unused referencedConversations
translation.
* fix(frontend): route conversation-reference chips to custom-agent sources
The picker offered custom-agent conversations but kept only the thread ID
and title, so transcript chips always linked to /workspace/chats/{id} and
dropped the source's custom-agent context on navigation.
Preserve the agent identity end to end: the picker now attaches
agentNameOfThread() (context first, then metadata.agent_name, mirroring
pathOfThread) to the selection, the display-only additional_kwargs metadata
round-trips it as agent_name, and the transcript chip passes it to
pathOfThread so custom-agent sources resolve to
/workspace/agents/{agent}/chats/{id}.
Tests: agent_name metadata round-trip and malformed-entry tolerance, picker
toggle carrying the metadata agent with run context winning, and a
picker-to-transcript regression pinning the /workspace/agents/writer/chats/
source-1 href.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* feat(extensions): expose incremental run evidence reader
* fix(extensions): address run evidence review feedback
* docs(extensions): clarify run deletion reconciliation
* docs(migrations): align current head documentation
* fix(extensions): isolate run evidence event reads
* test: avoid pinning run change migration to latest head
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scheduler): serialize the SQLite launch-budget claim
claim_queued_run counts the executing occurrences and then promotes one
row to launching. Postgres serializes that pair with a transaction
advisory lock; SQLite had no counterpart. pysqlite does not begin a
transaction for a SELECT, so the budget count ran in autocommit and the
deferred transaction reserved the writer only at the promoting UPDATE.
Claimers racing on distinct rows therefore read the same stale count,
each passed its own status == 'queued' CAS, and max_concurrent_runs was
exceeded.
A manual trigger overlapping the poller reaches this concurrently within
one process, and scheduler.multi_instance over a shared database file
reaches it across processes. Two claims of the same row were already
safe, which is why the existing coverage did not catch it.
Take the writer before the count with BEGIN IMMEDIATE, the idiom
ThreadMetaRepository already uses for its read-modify-write paths and
the same reservation _lock_task makes for a parent row. The claim
targets one row but the budget is global, so this has to be the
database-wide writer rather than a row lock.
* docs(changelog): reference #5469 in the SQLite launch-budget entry
* test(scheduler): pin the launch-budget test's connection reuse
The warm-up gather is what makes the claimers actually overlap, but it
silently depended on the SQLite engine keeping pooled connections. If
that engine ever moved to a non-pooling class, every claimer would open
its own connection, the per-connection PRAGMA setup would stagger them,
and this test would pass against an unserialized claim instead of
failing -- the cold-pool case it exists to avoid.
Assert that the warm-up left connections checked in. A non-pooling class
does not implement checkedin() at all, so a missing counter reads as
zero reuse and reports the same explanation rather than an
AttributeError. Verified against NullPool: the guard fails with
"NullPool left 0 connections pooled after the warm-up".
Only pool_size connections survive the gather (the overflow is
discarded), which is why the pre-fix failure is exactly five claimants
over a cap of one rather than eight.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash
Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.
Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
admission pinning (both server-owned sets + worker hoist); latest-only
request-scoped <project> block via DynamicContextMiddleware
wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
prefix + marker + provenance, never persisted); journal audit
fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
locked check-and-set; hash-qualified immutable shelf storage with
Paths helpers; upload/list/content/delete-to-trash routes; project
delete trashes the shelf in-transaction; request-scoped bounded
<documents> index with honest count/shown + actionable overflow note;
list_project_documents/read_project_document tools registered only on
pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
parity); POST from-thread with provenance; attach-to-thread with
lock-staged copy (archived source allowed); read-only thread-files
view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
file moves), purge (continuous row lock across unlink/delete/commit,
retryable on FS errors), retention sweep (lazy + startup, 24h orphan
guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
archived banner, content-missing rows), /workspace/trash route,
sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
+ specs.
Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
artifact + content responses; unified unsandboxed-iframe PDF preview
(fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
claims + os.link commit with suffix retry; same-name re-upload now
unique-names instead of replacing); hidden staging only, no visible
placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
revalidation; drain locked filesystem work on cancellation; preserve
bytes when an insert's commit state is uncertain (including trashed
rows)
- original-integrity checks before serving text or cached conversions;
content_missing surfaced in list responses (UI reads the flag, no
409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
rejects fractional/infinite values; composer counts staged
attachments; pending attachments persist until submission or removal;
in-flight instruction/rename edits survive save refetches; shelf and
trash pagination; conversation-file and thread-files pages stay
subscribed to refetches
Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.
Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
bounded: the indexed expiry purge still runs on every trigger
(GET /api/trash/documents, POST /api/trash/purge) while the
O(all rows + all files) reconciliation is throttled to one run per
user per 15 minutes (process-local, per-user window). The startup
sweep now runs as a background task instead of blocking gateway
readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
like the render path, so a pasted, fenced <project>/<documents> snippet
survives markdown export while real injected blocks (never fenced) are
still removed. Fence regexes moved to a dependency-free leaf module to
avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
attribute (the upstream e2e contract locates it via :not([title])), and
the upstream artifact-preview spec now pins the new contract: PDFs
render unsandboxed, images keep sandbox="".
* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title
- Shutdown cancelled only the shield around the background startup sweep,
so an all-users reconciliation that outlived the 5s budget kept walking
rows and files while the document repo and DB engine were disposed
underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
which cancels the task and drains it before worker exit: the shield
keeps the wait bounded, the cancel makes it final (CancelledError lands
at the sweep's next await, and `_run_startup_trash_sweep` only catches
`Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
previous fix round, leaving the PDF frame without an accessible name
while its siblings keep theirs. Restore it (WCAG frame titles), assert
it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
instead of `iframe:not([title])`.
* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel
`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.
* fix(projects): round-4 review — make Empty trash delete what it confirms
`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.
Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.
Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
* docs(agents): trim restated rationale in the subagent acceptance checklist
The AG002 chain sat 740 bytes under the hard limit, so the next contract paragraph would re-break main. Compress the acceptance-checklist bullet by dropping only restated rationale and duplicate examples: every rule, boundary, provider name, numeric bound, and test reference is preserved verbatim, verified by a mechanical token checklist. The chain goes from 97,564 to 96,598 bytes, putting headroom at 1,706 bytes, and scripts/check_agent_guidance.py reports 0 errors. The shared sandbox lifecycle bullet was reviewed too and left alone: it is already lean.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* docs(agents): drop the dangling em-dash left by the acceptance trim
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
---------
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(channels): stream-cap and validate WeChat/WeCom inbound media downloads, fixes#5223
* fix(channels): address WeCom APPID, decompression, and log-sanitization review findings (#5223)
Round-4 review follow-ups on the inbound-media download cap:
- The COS bucket numeric suffix is the owner's Tencent Cloud APPID and bucket
names are user-chosen, so any Tencent Cloud account could register a
matching ww-aibot-img-* bucket and pass the shape gate. The built-in rule
now admits only the APPID observed in Tencent's published aibot callback
examples (1258476243), across regions; any other account (including a
future WeCom rotation) goes through channels.wecom.allowed_media_hosts.
- aiter_bytes() transparently decodes Content-Encoding, and the decoder
allocates the full decompressed body before the byte cap sees a chunk (an
~8 KB gzip wire chunk decoding to 8 MiB reproduces it). Both URL readers
now send Accept-Encoding: identity, refuse a response with a residual
Content-Encoding before reading, and iterate aiter_raw().
- httpx.HTTPStatusError formats the signed URL (path + query credentials)
into its message, so _ingest_inbound_files' reader-failure branch logs a
sanitized summary (class + status) instead of logger.exception, and the
WeChat extract paths catch httpx.HTTPError so the polling loop's
per-message logger.exception can never render a media URL.
Every change ships with a red/green regression: the reviewer's 403
mock-transport repro asserted against caplog.text (fully formatted logs),
the reviewer's different-APPID bucket host, and gzip bombs driven through
real httpx mock transports in both readers. Docs (channels AGENTS.md,
README, config.example.yaml) updated for the APPID pinning and encoding
gate.
* docs(channels): document why the inbound-media cap is 50 MB, not WeCom's 100 MB ceiling
* fix(logging): redact URLs in httpx request logs down to scheme + host, fixes#5223
httpx emits 'HTTP Request: GET <full URL>' at the Gateway's INFO level
before any response handling runs, so even successful signed-media
downloads leaked their credentials. HttpxUrlQueryRedactionFilter
(installed by configure_logging) rewrites those records in place — path
and query become /<redacted>, method/status/duration observability is
preserved — which also keeps Telegram's token-bearing Bot API paths out
of the logs. Reader-level regression tests run at production INFO level
with a real MockTransport, success paths included.
* fix(logging): blank userinfo credentials in httpx request-log redaction
* fix(logging): redact authority-only URLs and cover urllib3 redirect logs
Two follow-ups from the review plus one extrapolation of the same class:
- rest is now optional in _URL_REDACT_RE, so an authority-only URL
(scheme://user:pass@host, no path) is rewritten too — userinfo had
nowhere else to hide and previously passed through verbatim. A bare
credential-free origin still passes through unchanged.
- Renamed to UrlRedactionFilter / install_url_log_redaction and attached
to the urllib3 logger as well: urllib3 logs 'Redirecting <url> -> <url>'
at INFO with full URLs on both sides, the same leak class on a different
library logger. No gateway path today both uses requests and redirects
a signed URL, but the class stays closed instead of dormant.
- Unit tests now build records with the real httpx 0.28.1 format string
('HTTP Request: %s %s "%s %d %s"', 5 args) and httpx.URL args, per
the nit, instead of a synthetic shape httpx never emits.
* fix(logging): install URL redaction at handler level so propagated records are covered
A logging.Filter on a logger only runs for records emitted through that
exact logger — child loggers neither inherit it nor trigger it on
propagation — so the previous attachment to the bare urllib3 logger was
dead code: urllib3 emits Redirecting via urllib3.poolmanager at INFO and
urllib3.connectionpool at DEBUG. The filter is now attached to every root
handler (mirroring _install_trace_filter, which already iterates root
handlers; handler-level filters see propagated records) in addition to
the httpx logger (httpx emits via the bare name, and emission-point
coverage survives handlers added later). The wiring is pinned by tests
that emit through the real urllib3 child loggers — a mutation removing
the handler-level install turns them red. Comments, docstrings, and
AGENTS.md now state the actual emitter names and levels.
* fix(logging): redact urllib3 DEBUG request lines, whose split shape evaded the URL regex
urllib3's per-request line (connectionpool.py:545 on 2.7.0) renders as
`scheme://host:port "METHOD /path?query HTTP/x.x" status len` — the
authority ends at a space so _URL_REDACT_RE's bare-origin early return
applies, and the quoted origin-form target has no scheme, so neither half
was rewritten. UrlRedactionFilter now runs a dedicated request-line shape
first (collapsing the target to /<redacted>, keeping scheme+host+method+
version), then the absolute-URL pass. Regressions pin the exact format
string both at unit level and through the real urllib3.connectionpool
DEBUG emit path; AGENTS.md wording now names both covered DEBUG shapes.
* fix(logging): redact urllib3 retry lines and linearize scheme scanning
Closes the two open review threads on the inbound-media log hardening:
Retry/redirect targets: urllib3 logs the request target with no scheme in
five shapes the generic absolute-URL pass cannot see - `Retry: <target>`
(connectionpool.py:954 DEBUG), `Incremented Retry for (url='<target>')`
(util/retry.py:545 DEBUG, absolute on the redirect path), `Retrying (...)
after connection broken by '<err>': <target>` (connectionpool.py:869
WARNING, above the INFO root), and origin-form halves of both Redirecting
emitters (poolmanager.py:500 INFO / connectionpool.py:922 DEBUG). Each
gets a rewrite anchored to the exact urllib3 format, collapsing the
target to /<redacted>; the generic pass's rest now stops at quote
characters so a quoted URL keeps its closing punctuation (previously the
absolute-form increment line was mangled), and the request-line method
class accepts any case. The emitter enumeration in channels AGENTS.md is
closed against the installed urllib3 2.7.0 source.
Quadratic scanning: both scheme-bearing patterns start with a character
class, so re.sub retried every suffix of a long token - 64K paths cost
~1.8s and URL-free 64K error bodies ~3.1s per record, synchronously in
every root handler. The two passes are now driven from literal "://"
occurrences: _scheme_starts walks back over the scheme charset to each
run's first letter and the pattern is attempted only there, reproducing
re.sub's leftmost-non-overlapping result in linear time (256K path:
5.6ms; worst adversarial shapes <= 28ms). Long-input regressions pin the
URL-bearing and URL-free cases with mutation-verified bounds, plus
nested-scheme and digit-headed-run equivalence cases.
Validation: tests/test_logging_config.py 12/12; scheme-pass equivalence
against the old re.sub pipeline verified by two independent 30k+ case
fuzz runs; full-suite A/B against HEAD shows zero tests that pass on HEAD
and fail with this diff.
* fix(logging): boundary-aware quote stops and whole-message Redirecting anchor
Two follow-ups on the urllib3 redaction shapes:
Embedded quotes: `rest` treated ANY quote as a closing mark, so a URL
with an apostrophe in the path kept everything after it verbatim
(`https://h/path'quoted'?token=Q` rendered the credential suffix in
full) while the class docstring claimed path/query/fragment are
replaced. A quote now closes `rest` only at a boundary - followed by
whitespace, a closing parenthesis, or end of string - so urllib3's
Incremented Retry (url='...') scaffolding keeps its ') closer while an
embedded quote stays consumed. The increment line's url capture gets
the same rule narrowed to its fixed ')' closer.
Redirecting anchoring: the origin-half pass matched `(?
<=-> )/path` as a substring, and an `-> /path` arrow is not
urllib3-owned shape - the sandbox provider's actionable mount error
(`sandbox.mounts entry <host> -> /mnt/knowledge ignored: ...`) had its
container path rewritten to /<redacted>, failing
test_setup_path_mappings_logs_actionable_error_for_missing_host_path on
CI (backend-unit-tests shard 3). The pass is now anchored to the whole
`Redirecting <t> -> <t>` message, which is exactly urllib3's record;
origin slots collapse, absolute slots stay for the generic pass.
Regression tests pin the embedded-quote shapes and the sandbox error's
byte-for-byte passthrough; both mutations verified red.
Validation: tests/test_logging_config.py 14/14; the CI-failing sandbox
test green locally; every test file asserting redaction/arrow log
content passes (attachments, support bundle, run metadata, skill
secrets, ragflow, skillscan, sandbox provider); full offline backend
suite 14084 passed / 164 failed with the failure set matching this
machine's documented Windows-environment baseline (NTFS chmod/symlink,
docker/lark/langfuse absences) - no failure involves redaction output.
* fix(logging): redact redirects with spaced locations
* fix(logging): grammar-complete Redirecting anchor; neutral WeChat guard labels
Round-13 P3 (Redirecting anchor strictness): the whole-message anchor kept
the ^Redirecting prefix (the urllib3-owned literal that stops the sandbox
false positive) but required BOTH slots whitespace-free, so a Location
header with an interior space voided the pass and leaked the origin-form
request target in the first slot - redirect_location is the raw header
string and interior spaces are legal field syntax. The tail is now loose
(\S.*$) and the first slot gets the same grammar treatment (\S.*?): the
recursive urlopen frame passes the previous raw Location as its url, so
t1 can carry interior spaces too, lazy-split at the first arrow the way
the line is constructed. A space-carrying slot collapses whole when it
starts with /; the sandbox mount error keeps passing through untouched.
Round-14 nit (None conflation): _download_cdn_bytes returns None for two
reasons (in-flight cap abort, Content-Encoding refusal) but both image and
file callers labeled it "exceeds size limit (N bytes)" - contradicting the
accurate encoding line right above it, and reporting the plaintext limit
for a ciphertext-cap decision. Callers now log a neutral
"skipped by download guard" line (the manager reader callers' shape);
the accurate reason stays inside the download function. The same sweep
also logs _stage_downloaded_file's silent None (no state dir configured),
which made an attachment vanish with no log line at all.
Also anchors the emitter-enumeration closure to its urllib3 version: the
closure reopens if an upgrade changes these format strings, so the comment
now says so explicitly.
Validation: logging 15/15 and attachments 60/62 (the two pre-existing
Windows symlink-privilege failures documented in the PR body); three
mutations verified red (old wording, strict t1, silent staging None);
ruff clean. Full offline suite run before push (per round-11 lesson).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): report an exactly-full AIO glob result as complete
AioSandbox.glob's include_dirs branch returned as soon as it had
collected max_results matches, without looking at the rest of the
listing. A listing that held exactly that many matches and nothing more
was therefore reported as truncated, and the glob tool told the model
the result was incomplete — prompting a re-search or distrust of a
complete answer. The same line returned one match for max_results=0,
one past the caller's cap.
Look one match past the cap before deciding, which is what the
include_dirs=False branch in the same function already does and what
#5427 moved parse_remote_search_output to for BoxLite, Tenki, E2B and
OpenSandbox.
* review: filtered-tail cases, the glob contract docstring, and the cap wording
Addresses the three items from the review on #5449.
- Two regression cases over a tail of ignored / out-of-root / pattern-miss
entries: an exactly-full result stays complete when only filtered entries
follow, and a third eligible match after that tail still reports
truncation. Both fail against the previous return-on-the-max-th-match
behaviour.
- 'Sandbox.glob' promised the conservative flag ('``max_results`` was
reached') that this change deliberately stops producing on the AIO branch.
The contract now reads as 'may be incomplete' and records that providers
differ in how precisely they can decide it.
- The changelog no longer lumps 'parse_remote_search_output' in with the
filtered-match cap: its raw-output cap is a separate limit with its own
one-line-past accounting, and the other providers' filtered-match cap is
unchanged.
Also corrects the docstring on the existing test, which still described the
removed early return in the present tense.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): scope runs read endpoints by data identity, not authorization identity
Trusted internal callers are authorized as a synthetic internal user
(id "default", or the make_safe_user_id-normalized owner with an owner
header), while start_run stamps run rows with the raw trusted-owner value.
list_runs / get_run / list_runs_page filtered by the authorization identity,
so the store-side user filter never matched and internal callers always saw
an empty runs list (or 404) for threads they are authorized to read.
The three read endpoints now resolve their filter id through
_run_scope_user_id: internal-role callers skip the per-user filter (thread
visibility is already authorized by owner_check=True), and browser/API
sessions keep the existing per-user filter unchanged.
Regression tests cover both identities across the three endpoints (list,
keyset page, single get) with a MemoryRunStore seeded with mixed-owner rows;
without the fix the four internal-caller cases fail while the
browser-session isolation case passes.
* fix(gateway): route the message read endpoints through the same data-identity scoping
Review follow-up on #5448: list_thread_messages and list_thread_messages_page
resolved get_current_user and passed it as the data filter to the event-store
scan, hidden-run lookups, turn-duration injection and the feedback queries —
the same authorization-vs-data identity conflation fixed for the runs
endpoints, leaving the #5437 empty-read symptom in place for lossy owner
values. Both endpoints now resolve their filter id through
_run_scope_user_id as well.
Regression tests extend to the two message endpoints, asserting the resolved
filter identity at the runs-store and feedback-repo boundaries (None for
internal callers, the session user id for browser sessions).
* fix(feedback): deterministic per-run collapse for unfiltered feedback reads
Review follow-up on #5448: with _run_scope_user_id returning None for
internal callers, the feedback lookups now receive an explicit-None user id,
which skips the user_id WHERE in FeedbackRepository. On shared/NULL-owner
threads several browser users can hold feedback on the same run, and
list_by_thread_grouped / list_by_run_ids collapsed rows per run_id via a
dict comprehension over unordered results — the feedback attached to the
last AI message would be an arbitrary user's row.
Both methods now order by created_at ASC with feedback_id as the tie-break,
so the collapse deterministically keeps the most recently created feedback.
_run_scope_user_id's docstring now documents that the resolved id also
scopes feedback and event-store reads, not just run rows.
Regression test seeds multi-user feedback on one run and asserts the
collapse outcome is stable across repeated unfiltered reads.
* docs(feedback): the collapse keeps the most recently written feedback
created_at is refreshed on upsert, so the surviving row per run is the
most recently written (created or updated), not the most recently
created — align both docstrings with the ordering key's actual semantics.
* fix(agents): keep queued guard warnings when a model call is retried
LoopDetectionMiddleware, TokenBudgetMiddleware and ToolProgressMiddleware
pop their queued warning/hint before calling the model. When the call
raises, LLMErrorHandlingMiddleware (outside them) retries by running
their wrap_model_call again, and by then the queue is empty, so the
retried request goes out without the warning. Loop detection and the
token budget have already marked it as sent, so it is never queued
again, and a loop runs on to the hard stop unwarned.
Put the drained items back in front of the queue when the handler
raises, then re-raise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(agents): trim restored loop warnings from the tail and drop a dead helper
_restore_pending_warnings put the restored warnings at the front and then
trimmed the front, so if the cap ever fired it would drop exactly what it
restored. Trim the tail, as tool progress does. _augment_request had no
callers after the wrap_model_call change. Add the sync twin of the tool
progress retry test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(agents): drop tool progress's unused _augment_request
Its only remaining reference was a test name; the dedup that test checks lives
in _inject_hints.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* feat(gateway): accept conversation references in run context and report the capability
LangGraph SDK clients build a fixed run body and drop unknown top-level
fields, so they cannot send the conversation_references field from #5399.
RunCreateRequest now lifts context.conversation_references into the
top-level field before validation, so it keeps the same bounds and error
locations, and drops it from context, so it never reaches the merged run
context or the checkpointed configurable. Sending both is a 422.
GET /api/features reports conversation_references {enabled, max_references}
with the same "tool is configured" predicate as run admission, so a client
can hide an entry point on deployments without the tool.
Related to #5398.
* fix(gateway): report the field type error for a malformed top-level reference list
A malformed top-level conversation_references sent alongside a context
list now fails with the field's own type error instead of the conflict
message. The tool-configured predicate reads tool.use directly, and the
features test doubles carry that attribute like every real ToolConfig.
* fix(gateway): treat every list-like top-level reference value as a conflict
Pydantic's lax mode coerces tuples, sets, frozensets and deques into the
list[str] field, so a direct Python caller passing one of those together
with context.conversation_references now reports the conflict instead of
slipping both grants through. Unreachable over HTTP, where JSON has no
such types.
* fix(gateway): ask pydantic whether a top-level reference value is list-like
Enumerating list-like types cannot track pydantic's lax acceptance set
(generators, UserList, dict key views also coerce into list[str]). The
conflict guard now validates the top-level value with a TypeAdapter for
list[Any]: whatever pydantic would coerce reports the conflict when it is
non-empty, and whatever it rejects still surfaces the field's own type
error. Regression tests cover deque, UserList, dict keys and a generator,
plus rejected scalars.
* fix(gateway): probe top-level references with the field's own annotation
The conflict guard now validates the top-level value with the exact item
annotation the field uses, so its acceptance set is the field's rather than
a superset: an item the field rejects (an empty string, a non-string, the
ints of a range or dict view) surfaces the field's own item error instead
of a conflict. The annotation is shared through one alias so the two cannot
drift.
* fix(gateway): materialise a one-shot iterator before probing top-level references
The item-validating probe could consume a generator while collecting an
item error, after which the field re-validated the exhausted iterator,
coerced it to [] and let the request through with the key still in
context. Iterators are now read once into a list that both the probe and
the field validate, so a bad item is reported at its index and a valid
generator is kept.
* fix(gateway): materialise every once-walkable iterable before probing references
Pydantic coerces any iterable into the list field, and an object whose
__iter__ hands out a generator once is not an Iterator instance, so the
previous gate let it reach the probe and be consumed. The lift now reads
every iterable except lists, tuples and the shapes the field rejects as a
whole (str, bytes, dict) into a list first, so the probe and the field
always validate the same items.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* fix(discord): flush thread mappings on shutdown to survive restart (#2897)
The Discord channel already persists channel->thread mappings, but a mapping created right before a hard shutdown (process killed between thread creation and its background persistence write) could still be lost. Flush in-memory mappings in stop() as a best-effort safety net.
Add regression tests covering the persist/load round-trip across a simulated restart and the stop() flush path.
Refs #2897
* fix(discord): gate stop() flush on _thread_store_loaded (#5461 review)
Address review feedback:
- Only flush thread mappings from stop() once _load_active_threads() has
run, so a stop() taken before the load (start() bailed on a missing
bot_token / discord import error) cannot overwrite the store with {}.
- Fix isort order in the test module (ruff I001).
- Make the restart test a plain sync test (it never awaits).
- Add a regression test proving stop() does not clobber the store before load.
* fix(discord): gate stop() flush on _thread_store_loaded (#5461 review)
Address review feedback:
- Only flush thread mappings from stop() once _load_active_threads() has
run, so a stop() taken before the load (start() bailed on a missing
bot_token / discord import error) cannot overwrite the store with {}.
- Fix isort order in the test module (ruff I001).
- Make the restart test a plain sync test (it never awaits).
- Add a regression test proving stop() does not clobber the store before load.
---------
Co-authored-by: wcy12378 <wcy12378@users.noreply.github.com>
* fix(agents): remove provider tool-call blocks when guards strip calls
Token-budget and loop-detection hard stops, subagent-limit truncation,
and safety-finish-reason suppression removed calls from tool_calls and
the raw additional_kwargs payload, but left the provider's own
tool-call blocks in AIMessage.content. Provider adapters re-serialize
those blocks: langchain_anthropic sends a tool_use block whose id is
not in tool_calls, and the OpenAI Responses input builder sends every
function_call block. ChatAnthropic stores any tool-calling response as
a block list, so a guard firing on a Claude tool call always left a
tool_use without a tool_result. A truncated subagent call failed the
next model request of the same run; a hard stop was checkpointed under
the same message id and failed every later turn of the thread.
clone_ai_message_with_tool_calls now trims content tool-call blocks to
the calls that remain on the message: tool_use and LangChain v1
tool_call/tool_call_chunk by id, Responses function_call and
custom_tool_call by call_id (their id is the fc_ item id), Google GenAI
function_call by id, and id-less blocks by name in order. Blocks for
calls still on invalid_tool_calls stay, because
DanglingToolCallMiddleware answers those calls with placeholder
results. The token-budget and loop-detection hard stops now build their
messages through the helper instead of their own copies, and
ClarificationMiddleware drops its private filter, which matched
Responses blocks by item id.
* docs(changelog): reference #5447 in the orphaned tool-call block entry
* fix(agents): skip id-matched calls in the id-less block budget
The name budget for id-less content tool-call blocks counted every
retained call, including calls whose own id-bearing block had already
matched. In mixed-shape content, a retained call "a" with a
function_call block carrying id "a" also let a same-named id-less block
survive, leaving the unpaired block this helper exists to remove.
Collect the retained ids that id-bearing blocks matched first, and build
the name budget only from retained calls outside that set. Content with
no id-bearing blocks keeps the full budget, so the Gemini path is
unchanged.
* fix(subagents): keep the subagent system prompt through context compaction
A subagent carries its whole system prompt (role, report contract, skills,
deferred tools) as the leading SystemMessage in state, because its agent is
built with system_prompt=None. Summarization only rescued tagged reminders and
the latest user message, so the first compaction summarized the prompt away
and every later model call in that subagent ran without its instructions.
Rescue system messages as well. The lead agent's prompt is not in state, and
its in-state SystemMessages are the tagged reminders already rescued, so the
lead chain is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(compaction): clarify preservation contract and trim agent guidance
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(conversation): continue reading a cut message by offset
A referenced message longer than 4,000 characters was cut, and its
suffix could not be read back. Cut messages now carry a continuation
(message_seq, offset). read_conversation(thread_id, message_seq, offset)
returns the next part of that one message, sized to the same
tool-output budget as pages. The read scans only the requested row
under the existing visibility rules and rechecks ownership. Offsets
follow the source's current text; an offset past the end is rejected.
Related to #5398.
* docs(conversation): say continuations ignore limit
A continuation always returns one part of one message, so limit does not apply there. The tool schema now says so instead of discarding it silently.
Related to #5398.
* fix(conversation): stop instead of looping when no text fits the budget
With a read_conversation tool-output budget below the envelope size, the fitted text was empty and the continuation repeated the requested offset, so an agent would repeat the identical call forever. Page and continuation reads now return output_budget_too_small with no continuation.
Related to #5398.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* fix(sandbox): force UTF-8 console for PowerShell so CJK output is not garbled
LocalSandbox captures PowerShell output through a UTF-8 pipe reader
(errors=replace), but Windows PowerShell 5.1 writes console output in
the legacy OEM codepage (GBK on zh-CN Windows) unless told otherwise,
so every CJK character in tool output arrives as mojibake and the
decode never raises. Prepend a UTF-8 preamble
([Console]::InputEncoding/[Console]::OutputEncoding/$OutputEncoding)
to the -Command payload so both directions of the console are UTF-8
before the user command runs.
* fix(sandbox): pair PowerShell UTF-8 capture and guard console setup
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): platform-aware Lark CLI runtime validation for Windows hosts
The managed Lark CLI sandbox runtime validation and its tests assumed POSIX
semantics that Windows hosts cannot satisfy, breaking the focused AIO/Lark CLI
suites (5 failures on current main).
- _validate_lark_cli_sandbox_runtime keeps the strict executable-bit contract
on POSIX; on Windows it validates the Linux-only artifacts by content
instead (ELF/PE/Mach-O image magic for linux-*/lark-cli, shebang for the
bin/lark-cli launcher), since NTFS cannot represent the exec bit.
- The AIO runtime-mounts test now asserts the explicit Windows credential
contract: an owner-only inheritable DACL (via the existing PowerShell ACL
resolvers) instead of exact 0o700 modes, which remain asserted on POSIX.
- Accept-path runtime tests stage ELF-prefixed payloads so both platforms
exercise realistic artifact content; the extractor's mode assertion is
POSIX-only with a writability check on Windows.
Focused suites: 5 failed / 162 passed -> 167 passed, 3 skipped on Windows 11;
POSIX behavior unchanged (POSIX branches keep the original assertions).
* refactor(tests): share Windows ACL resolvers via a helper module and cover the Windows shebang gate
Review follow-ups on #5442:
- Move the PowerShell ACL resolvers (_windows_acl_env/_windows_acl_sids/
_windows_acl_protected/_windows_acl_owner_sid) from
tests/test_lark_cli_integration.py into tests/_windows_acl_helpers.py,
following the existing shared-helper convention, so the aio suite no longer
imports the full lark-cli integration module (which drags in app.gateway
routers and the FastAPI TestClient at collection time).
- Add test_managed_sandbox_runtime_rejects_launcher_without_shebang_on_windows:
a launcher without a shebang plus ELF-magic binaries, with lark_cli.os
monkeypatched via the existing Windows stub so the shebang-missing reject
branch of _runtime_artifact_is_executable is covered on every platform.
- Comment the rejects-non-executable prestaged-binary test to record that on
Windows the rejection comes from the payload's non-magic content, since
chmod() cannot clear the exec bit there.
Focused suites: 168 passed / 3 skipped on Windows 11.
* fix(agents): keep token budget signals for runs without a run_id
#5410 moved every invocation without a non-empty string run_id onto
str(id(runtime)). Two things break on that key:
- SubagentExecutor passes the parent's run_id, None when the parent run has
none (LangGraph Server, direct create_deerflow_agent callers), and reads the
stop reason back with that None. The hard stop stored it under the id string,
so a token-capped subagent reported a clean completion to the lead.
- LangGraph gives each graph node its own Runtime wrapper, so the key changed
between after_model and the next model call: the budget warning was never
delivered, and each after_model counted every AIMessage in the thread.
Key those invocations by Runtime.control, as LoopDetectionMiddleware does,
release it in after_agent, and store the stop reason under the context run_id
as given.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(agents): keep an active invocation's budget key when the anchor map is full
The fallback anchor map was FIFO, so with 1000 run_id-less invocations on a
shared instance an active one could lose its anchor mid-run and restart with a
fresh budget. Move the anchor to the end on every lookup, as loop detection
does, and note why execution_info.run_id is not consulted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(skills): close SkillScan bypasses in the skill review gate
The public skill review gate re-materialized a package snapshot into a
temp directory for SkillScan, but copied only entries the reader had
decoded as text and skipped every file under any evals/fixtures/
directory. Executable binaries and nested archives never reached the
package rules, and a fixture-shaped path hid any script from the scan.
Readers now keep binary bytes as content_base64, the analyzer writes
every non-symlink file byte for byte, and only eval fixture SKILL.md
samples stay exempt. Files are created exclusively, so a duplicate
archive member or a case-folded name fails the scan closed instead of
overwriting an earlier file.
SkillScan itself skipped any file that was not NUL-free UTF-8. One
Latin-1 byte in a comment hid a reverse shell from the review gate, and
a NUL byte skipped static analysis at install. Code files that fail
strict decoding now raise package-undecodable-script (HIGH) and are
analyzed over a lossy decode, so CRITICAL matches keep blocking.
"Code file" and "executable magic" were defined separately in the
installer and SkillScan and had drifted: SkillScan missed 32-bit
little-endian and fat Mach-O variants the installer blocks. Both rules
now live in skills/package_files.py, shared by the installer, the export
guard, and SkillScan.
* docs(changelog): link the skill review gate fix to #5431
* fix(skills): fail closed on bytes-less snapshot entries and skip text rules for executables
The review analyzer skipped any snapshot entry it could not turn into
bytes. Readers only emit such entries for oversized files, and they also
mark the snapshot truncated, but content_base64 is optional in the
contract, so a reader regression or a hand-built snapshot would silently
drop a file from SkillScan. An entry without bytes now fails the scan
closed (not_assessed: skillscan) unless the snapshot is truncated, and a
text entry without content no longer materializes as an empty file.
A real executable under scripts/ is a code file, so SkillScan decoded it
lossily and ran the text rules over its string tables. An OpenSSH binary
produced a CRITICAL secret-private-key finding from the key-format
banner it embeds. An undecodable file with executable magic still
reports package-undecodable-script, and its CRITICAL
package-executable-binary finding already blocks it, so it now skips the
text rules. Decodable files keep full text analysis.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
McpRoutingMiddleware picked the latest user message with
is_real_user_message, which rejects every hide_from_ui message. A Human
Input Card reply is hidden but is still the user's current request, so
when the routing keyword lived only in the clarification answer the
deferred MCP tool was never auto-promoted and the model had to call
tool_search by hand.
Switch _latest_user_message to is_genuine_user_message, the same
predicate summarization_middleware already uses for this reason (#5416).
Fixes#5425
* fix(goal): stand the goal down once the run has hit its token budget
Since #5410 goal continuations share the run's token budget, so a
continuation queued after the budget's hard stop only spends one more
model call before its tool calls are stripped. Pass the run's stop_reason
into the goal loop and stand the goal down with "token_capped" in that
case. The evaluator still runs first, so a goal the capped run satisfied
is cleared as before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(goal): list the token-budget stop among goal-loop preconditions
Also pin that a satisfied goal is cleared, not stood down, when the run
hit its token budget.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(sandbox): report truncated remote glob and grep results
BoxLite, Tenki, E2B, and OpenSandbox run find/grep in the sandbox, cap
the raw output with `| head`, and then filter those lines in Python:
ignored directories such as node_modules are dropped and grep's glob
scope is applied. They reported truncated only when max_results matches
survived the filter. When the capped lines were mostly filtered out, a
search with real matches past the cap came back short or empty with
truncated=False, and glob_tool/grep_tool rendered it as "No files
matched" / "No matches found". With the default max_results=200 and
1,200 files under node_modules, glob("**/*.py") reported no matches for
a workspace that has src/app.py.
remote_search_command now lets one line past its limit through, and
parse_remote_search_output(..., limit=) returns RemoteSearchOutput(text,
truncated): the first `limit` lines and whether the extra line arrived.
Exactly `limit` lines stays a complete result. Each provider passes the
cap it already computed to both calls and returns that truncated from
glob and grep when fewer than max_results results survive filtering.
The glob and grep tools now describe an empty truncated result as
incomplete instead of reporting no matches, which also covers AIO grep's
forwarded truncated flag. Sandbox.glob/grep document truncated as "the
matches may be incomplete".
* docs(changelog): reference #5427 in the remote search truncation entry
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Document that read permission expiry and source deletion do not erase
text already copied into the destination conversation, and that reads
follow the source's current visible history. Truncated results now tell
the agent to acknowledge the omission and ask for the missing material
before claiming every requirement is covered.
Pages were filled to 20,000 text characters by cutting the last message
that did not fit, and that suffix could never be paged back. They could
also exceed the default 12,000-character tool-output budget, which
externalized the page to a file. Pages are now sized by their serialized
length against the read_conversation tool-output budget; a message that
does not fit starts the next page intact, so only a message over 4,000
characters (or one whose escaped JSON alone exceeds the budget) is cut.
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* feat(scheduled-tasks): filter run history by occurrence status
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
* fix(scheduled-tasks): share occurrence status contract
---------
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): mask every host path in a colon-joined list
Host-to-virtual output masking matched a host root and then consumed the
path tail up to whitespace or shell punctuation, but not `:`. A
`:`-joined list such as $PATH or $PYTHONPATH was therefore swallowed into
the first match's tail, and scanning resumed after it, so every later
entry under the same root reached the model as a raw host path. The regex
matcher (process-stable skill roots) and the direct scanner (per-thread
roots, LocalSandbox) shared the gap. Each redundant masking pass --
separator variants, the realpath spelling, the /mnt/user-data root
mapping, LocalSandbox's own reverse resolution -- happened to recover one
entry, which hid the leak for short lists: bash output leaked from the
fourth entry, single-pass consumers from the third.
The shared tail in path_patterns.py now ends at `:` in both matchers.
`;`, the Windows list separator, already ended it. A `:` inside one path
(grep -n output, a file name) only shortens the match; the remaining text
is copied through verbatim.
Shortening the match exposed a second leak. LocalSandbox reverse
resolution realpaths the matched path and returned that realpath when no
mount contained it, so a symlink inside a mount whose target lies outside
every mount was shown as the target's host path. grep -n lines used to
hide this only because the whole line resolved as one nonexistent file;
whitespace-terminated output and LocalSandbox.glob results already leaked
it on main. Reverse resolution now falls back to the link's own spelling,
normalized so `mount/../x` does not pass, before giving up. A symlink into
another mount still reports that mount's path.
* docs(changelog): reference #5418 in the colon-joined path masking entry
* docs(changelog): split the #5418 and #5419 entries fused by the merge
Resolving the CHANGELOG conflict when main was merged in dropped the
opener of the #5419 entry, so the BoxLite grep fix continued inside this
PR's bullet in both CHANGELOG.md and CHANGELOG_zh.md. Restore it as its
own bullet; the #5419 entry is byte-identical to main again.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): scope BoxLite grep globs to the search root
BoxliteBox.grep omits grep --include for busybox portability and applies
the glob in Python, but it kept only the glob's last path segment and
matched it against each file's basename. A scoped pattern therefore lost
its directory part: grep(glob="src/*.js") returned every .js file in the
tree, including vendor/ and nested src/ subdirectories that glob() with
the same pattern excludes.
The glob now goes through path_matches against the path relative to the
search root, with the file's basename when the root is a single file --
the same scope glob() uses and the one Tenki, E2B, OpenSandbox, AIO and
LocalSandbox already enforce. Like those providers, an empty glob is now
passed to path_matches instead of being treated as no filter.
* docs(changelog): reference #5419 in the BoxLite grep glob scope entry
* test(mounts): compare host paths separator-agnostically in provisioner and mount suites
The provisioner deliberately preserves host filesystem style
(join_host_path keeps native separators and os.path.normpath re-spells
POSIX inputs on Windows), and the docker --mount args and review-CLI
PYTHONPATH inherit that spelling. Six assertions across
test_provisioner_pvc_volumes, test_three_way_skills_mount_e2e and
test_review_changed_public_skills compared those strings against
POSIX-style literals, so they fail on Windows hosts while passing on
Linux CI. Normalize the host-native side before comparing; the replace
is a no-op on POSIX, so CI expectations are unchanged.
* test(mounts): share the host-path normalization helper
Both review suggestions: define posix_path once in
backend/tests/_host_path_helpers.py (matching the existing _xxx_helpers
convention) instead of three spellings across suites, and wrap the last
raw hostPath assertion (test_hostpath_userdata_includes_thread_id) so
the whole provisioner class stays separator-agnostic.
Summarization rescues the latest user message by id so the current
request survives compaction (#4882). It picked that message with
is_real_user_message, which rejects every hide_from_ui HumanMessage,
including Human Input Card replies. When a run starts from a card
reply, the older visible request was kept and the user's answer was
summarized away.
Use is_genuine_user_message, which accepts hidden messages that carry a
valid human_input_response and still skips other hidden injections.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(agents): keep the token budget across goal continuations of a run
TokenBudgetMiddleware cleared its usage in after_agent, and before_agent
marks every existing message as seen. A Gateway run re-enters the graph
for hidden goal continuations under the same run_id, so each
continuation started from zero and could spend another full budget,
even after the user turn had already hit the hard stop.
Keep the run's usage and warning state across graph entries, as
LoopDetectionMiddleware does since #5344. Only the per-message seen map
is dropped, and before_agent rebuilds it. Invocations without a run_id
in the context still clear everything.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(agents): normalize token budget run identity and test cleanup
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): reject forged framework-injection markers in run input
`is_genuine_user_message` treats `hide_from_ui` and a human `name="summary"`
as proof the framework authored a message, and `InputSanitizationMiddleware`
skips those — escaping a real reminder's blocks would corrupt trusted context.
Neither marker was server-owned, so an external caller could set either one and
place a raw `<system-reminder>` outside the user-input boundary markers, which
the lead-agent system prompt declares trusted internal framework data. The
`hide_from_ui` variant is also filtered out of the thread UI, so the forgery
was invisible where it landed.
Both markers are now stripped from untrusted input, on the run path and on the
thread-state mutation route that writes straight into a checkpoint. Framework
injection happens inside the graph and never crosses this boundary, so nothing
the framework does is affected, and `trusted_internal` callers (IM channels,
the MCP task-notification launcher) keep writing hidden messages.
HumanInputCard replies are the one legitimate external `hide_from_ui`: the
frontend sends it alongside a `human_input_response` payload, so a message
carrying a valid one keeps the marker. That buys no bypass — the predicate
already classifies those as genuine, so they stay sanitized.
The name check reuses the predicate's own `_SUMMARY_MESSAGE_NAME` rather than a
fourth copy of the literal, and matches by `isinstance` exactly as the predicate
does: `HumanMessageChunk` is a `HumanMessage` whose `type` is not `"human"`, so
a type-based check would leave that subclass's marker settable. `name` is only
reserved on human messages — on a ToolMessage it is the tool's own name.
Three tests in test_gateway_services.py and test_message_provenance.py asserted
that a caller-supplied `hide_from_ui` survives. That assumption was the bypass;
they now pin the opposite, with a genuinely caller-owned key kept alongside to
prove the stripper is surgical.
* fix(agents): sanitize every genuine user message, not only the newest
The input guardrail scanned backwards for the first genuine user message and
returned, so only the newest turn was ever sanitized. The transformation is
request-scoped (`wrap_model_call`, never written to state), so thread state
keeps the raw text: once a newer turn arrived, the previous turn's payload was
replayed to the model verbatim, outside the boundary markers the lead-agent
prompt declares trusted framework data. The guardrail therefore held for
exactly one model call.
Reaching it needed no forged metadata and no crafted request body — type the
payload in one turn, then send anything at all in the next. A single request
carrying two user messages did it in one shot, since every message but the last
was skipped.
`_process_request` now walks the whole list and `_sanitize_message` owns the
per-message work; every existing branch (the `original_user_content` split for
upload turns, the multimodal rfind fallback, the metadata repair) is unchanged.
Framework-injected messages stay excluded by `is_genuine_user_message`.
Unexpected errors now fail open per message rather than per request. Iterating
history widened the old blast radius: one unprocessable row would have dropped
sanitization for the whole request, handing an attacker the newest turn by
crafting an older one. `GraphBubbleUp` still propagates.
Side effect worth noting: each turn's rendering is now stable across model
calls. Previously a turn was wrapped on its own call and unwrapped on the next,
changing the prompt prefix behind the newest turn and defeating prompt caching.
test_only_processes_last_user_message pinned the old scope; it now pins that
every turn is processed, and keeps driving the `wrap_model_call` entry point.
* docs: record the message-metadata trust boundary and sanitization scope
`agents/middlewares/AGENTS.md` owns the depth for InputSanitizationMiddleware
and documented only the `original_user_content` half of its trust boundary. Left
alone it would teach an agent that `hide_from_ui` is caller-owned and that the
guardrail covers one turn — and the usual failure mode is an agent "restoring"
the behaviour it believes was lost. The entry now carries both markers, the
HumanInputCard exception, the whole-history scope, and the per-message fail-open
rule.
The note lives only there. The root and `backend/AGENTS.md` layers are
orientation that points at the module guides owning the depth, and
`backend/AGENTS.md` is inherited by every backend chain — prose added there
inflates more than twenty of them, and `scripts/check_agent_guidance.py` shows
the middlewares chain has about a kilobyte of room against its hard limit.
CHANGELOG.md and CHANGELOG_zh.md record it under Security, continuing the
existing prompt-injection lineage.
* fix(gateway): mark caller-hidden messages instead of stripping the marker
Review follow-up. Stripping a caller-supplied `hide_from_ui` closed the bypass
but broke three frontend senders that use the marker purely to keep a context
message out of the transcript: the quoted conversation context
(`buildHiddenConversationQuoteMessage`), the sidecar context prompt
(`buildHiddenSidecarContextMessage`), and the agent save command. None carries a
`human_input_response`, so the HumanInputCard carve-out did not cover them, and
nothing else hides them — `_is_branch_visible_message` and the frontend's
`isHiddenFromUIMessage` both key solely on `hide_from_ui`, and no backend reads
`conversation_quote_context` or `sidecar_context`. All three would have rendered
as user-visible chat bubbles.
The marker plays two roles and only one of them is a vulnerability. The security
requirement is that a caller-supplied marker cannot skip sanitization, not that
it cannot hide a message. So the roles are separated instead of the marker being
removed: the Gateway keeps it and stamps the server-owned `UNTRUSTED_INPUT_KEY`,
and the guardrail now asks `requires_input_sanitization` — the mark, else the
genuine-user test. Hidden stays hidden; untrusted content is sanitized either
way. The reserved `summary` name is handled the same way and no longer rewritten.
Marking rather than removing is also the safer shape in general: `hide_from_ui`
is read for presentation, journal persistence, memory filtering and IM outbound
as well, and this boundary should not silently change any of them.
`is_genuine_user_message` is deliberately left alone. `ToolReceiptMiddleware`
uses it for turn-boundary detection, where a caller's hidden context message must
keep counting as not user-authored; widening it there would move the ledger's
turn window. `requires_input_sanitization` sits beside it in `message_utils` so
the two questions can be compared.
The three tests that asserted a caller-supplied `hide_from_ui` is removed now
assert it survives and carries the mark — which restores the original intent of
the two provenance cases, whose comment already read "caller-owned keys must
survive".
* docs(gateway): rewrite normalize_input's docstring around the mark
Review follow-up. The paragraph still described the pre-4a3344f6 strip model and
contradicted both the implementation and the middlewares AGENTS.md paragraph
updated in that same commit: it called `hide_from_ui` server-owned, said
carrying it skips sanitization entirely, and repeated the premise this branch
disproved — that HumanInputCard replies are the only legitimate external use.
It now describes what the code does: the markers stay caller-owned and are
preserved because three frontend senders rely on `hide_from_ui` for hiding
alone, the message is stamped with `untrusted_input` instead, and
`requires_input_sanitization` sanitizes it anyway. `untrusted_input` joins the
server-owned inventory in the preceding paragraph, which is what makes the stamp
unforgeable and unclearable.
The three surrounding docstrings now also say that these functions mark as well
as strip; `_strip_external_message_metadata` had advertised only the removal,
leaving a reader no way to find the stamp from the name.
* fix(gateway): mark state writes whose message omits additional_kwargs
Review follow-up. The state-route half of the fix missed the most natural
request shape. `_strip_external_metadata_from_message_like` returned early when
`additional_kwargs` was absent or not a dict — there was nothing to strip — and
that early return also skipped the mark. A `POST /threads/{id}/state` body of
`{"values": {"messages": [{"role": "user", "name": "summary", "content":
"<system-reminder>…</system-reminder>"}]}}` therefore reached the checkpoint
unmarked. The messages reducer's `convert_to_messages` then supplies
`additional_kwargs={}`, so at model-call time `requires_input_sanitization` fell
back to `is_genuine_user_message`, which a `summary` name fails, and the forged
tag reached the model raw and outside the boundary markers.
A missing or non-dict `additional_kwargs` is now treated as empty for both the
strip and the mark. The identity return is kept for the case where nothing
changes, so an ordinary key-omitted message does not gain an empty dict just by
passing through. The run path was never affected: `normalize_input` coerces to
BaseMessage first, which always carries the dict.
Every existing state-write test supplied an `additional_kwargs` dict, which is
why this shape slipped through; the regression now covers it at the route and
end to end through the reducer into the guardrail.
While checking the neighbouring shapes, `_skips_input_guardrail` keyed off key
presence where `is_genuine_user_message` keys off truthiness, so
`hide_from_ui: False` — already covered without a mark — would have been
stamped. It now mirrors the predicate exactly, as its docstring claimed.
* docs(middlewares): compress the sanitization note to fit the guidance chain
main's growth left the middlewares AGENTS.md chain 84 bytes under its hard
limit, and the fuller wording did not fit. The load-bearing facts stay — the
markers are marked rather than stripped, and the scan covers every turn — since
those are the two an agent editing this middleware could otherwise get wrong.
The full model lives in the normalize_input, _mark_untrusted_framework_markers
and requires_input_sanitization docstrings.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(client): emit streamed tool calls once with complete args
When a model streams a tool call as chunks (name and id first, then
argument fragments), DeerFlowClient.stream() emitted a tool_calls event
per chunk, each parsed from that fragment alone. Consumers got
`args={}` and then a call with no name or id, and never the complete
call, because the values snapshot skipped the already-streamed id.
Hold tool calls from AIMessageChunk.tool_call_chunks and emit them once
from the values snapshot, where the message is complete.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(client): pin additional_kwargs on the deferred tool_calls event
Also note why a tool-call chunk without a message id keeps the per-chunk
event.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(models): reuse the Claude Code OAuth token read from a file descriptor
ClaudeChatModel accepts a Claude Code OAuth token through
CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR, but that handoff can be drained
only once: a pipe returns EOF after the first read and a file descriptor
keeps its advanced offset. _read_secret_from_file_descriptor read it again
on every call and kept nothing. Every ClaudeChatModel instance loads
credentials in model_post_init, and create_chat_model builds fresh
instances per run, so with a descriptor-only handoff the first model
authenticated and every model after it -- including the title model of
the very first run -- had no credential. The Anthropic SDK then raised
"Could not resolve authentication method" before sending a request.
A secret read from a descriptor is now kept for the life of the process,
keyed by (env_var, fd), so a different descriptor is still read fresh.
The read happens under a lock so two threads building their first model
concurrently cannot race one of them to EOF. Empty reads and OSError are
not cached and behave as before; lookup order, config keys, and log
messages are unchanged.
* docs(changelog): reference #5411 in the Claude Code OAuth descriptor fix entry
* test(models): pin that a closed descriptor handoff keeps its token
Review follow-up on #5411: the descriptor secret cache is keyed on the
fd number, which the OS recycles. Folding os.fstat identity into the key
would break the property the cache exists for -- once the handoff fd is
closed after the first read, fstat raises EBADF and every later model
would lose the token again -- and it would still miss a regular file
rewritten in place, which keeps its st_dev/st_ino.
The handoff is fixed at process start, so keep the number as the key and
state the invariant instead: a closed handoff keeps serving its token, a
secret placed on a recycled number is not re-read, and anything handing
over a new secret in-process must clear the cache. A new test pins the
closed-handoff behavior; an fstat-fingerprinted key fails exactly that
test.
