* fix(channels): stream-cap and validate WeChat/WeCom inbound media downloads, fixes#5223
* fix(channels): address WeCom APPID, decompression, and log-sanitization review findings (#5223)
Round-4 review follow-ups on the inbound-media download cap:
- The COS bucket numeric suffix is the owner's Tencent Cloud APPID and bucket
names are user-chosen, so any Tencent Cloud account could register a
matching ww-aibot-img-* bucket and pass the shape gate. The built-in rule
now admits only the APPID observed in Tencent's published aibot callback
examples (1258476243), across regions; any other account (including a
future WeCom rotation) goes through channels.wecom.allowed_media_hosts.
- aiter_bytes() transparently decodes Content-Encoding, and the decoder
allocates the full decompressed body before the byte cap sees a chunk (an
~8 KB gzip wire chunk decoding to 8 MiB reproduces it). Both URL readers
now send Accept-Encoding: identity, refuse a response with a residual
Content-Encoding before reading, and iterate aiter_raw().
- httpx.HTTPStatusError formats the signed URL (path + query credentials)
into its message, so _ingest_inbound_files' reader-failure branch logs a
sanitized summary (class + status) instead of logger.exception, and the
WeChat extract paths catch httpx.HTTPError so the polling loop's
per-message logger.exception can never render a media URL.
Every change ships with a red/green regression: the reviewer's 403
mock-transport repro asserted against caplog.text (fully formatted logs),
the reviewer's different-APPID bucket host, and gzip bombs driven through
real httpx mock transports in both readers. Docs (channels AGENTS.md,
README, config.example.yaml) updated for the APPID pinning and encoding
gate.
* docs(channels): document why the inbound-media cap is 50 MB, not WeCom's 100 MB ceiling
* fix(logging): redact URLs in httpx request logs down to scheme + host, fixes#5223
httpx emits 'HTTP Request: GET <full URL>' at the Gateway's INFO level
before any response handling runs, so even successful signed-media
downloads leaked their credentials. HttpxUrlQueryRedactionFilter
(installed by configure_logging) rewrites those records in place — path
and query become /<redacted>, method/status/duration observability is
preserved — which also keeps Telegram's token-bearing Bot API paths out
of the logs. Reader-level regression tests run at production INFO level
with a real MockTransport, success paths included.
* fix(logging): blank userinfo credentials in httpx request-log redaction
* fix(logging): redact authority-only URLs and cover urllib3 redirect logs
Two follow-ups from the review plus one extrapolation of the same class:
- rest is now optional in _URL_REDACT_RE, so an authority-only URL
(scheme://user:pass@host, no path) is rewritten too — userinfo had
nowhere else to hide and previously passed through verbatim. A bare
credential-free origin still passes through unchanged.
- Renamed to UrlRedactionFilter / install_url_log_redaction and attached
to the urllib3 logger as well: urllib3 logs 'Redirecting <url> -> <url>'
at INFO with full URLs on both sides, the same leak class on a different
library logger. No gateway path today both uses requests and redirects
a signed URL, but the class stays closed instead of dormant.
- Unit tests now build records with the real httpx 0.28.1 format string
('HTTP Request: %s %s "%s %d %s"', 5 args) and httpx.URL args, per
the nit, instead of a synthetic shape httpx never emits.
* fix(logging): install URL redaction at handler level so propagated records are covered
A logging.Filter on a logger only runs for records emitted through that
exact logger — child loggers neither inherit it nor trigger it on
propagation — so the previous attachment to the bare urllib3 logger was
dead code: urllib3 emits Redirecting via urllib3.poolmanager at INFO and
urllib3.connectionpool at DEBUG. The filter is now attached to every root
handler (mirroring _install_trace_filter, which already iterates root
handlers; handler-level filters see propagated records) in addition to
the httpx logger (httpx emits via the bare name, and emission-point
coverage survives handlers added later). The wiring is pinned by tests
that emit through the real urllib3 child loggers — a mutation removing
the handler-level install turns them red. Comments, docstrings, and
AGENTS.md now state the actual emitter names and levels.
* fix(logging): redact urllib3 DEBUG request lines, whose split shape evaded the URL regex
urllib3's per-request line (connectionpool.py:545 on 2.7.0) renders as
`scheme://host:port "METHOD /path?query HTTP/x.x" status len` — the
authority ends at a space so _URL_REDACT_RE's bare-origin early return
applies, and the quoted origin-form target has no scheme, so neither half
was rewritten. UrlRedactionFilter now runs a dedicated request-line shape
first (collapsing the target to /<redacted>, keeping scheme+host+method+
version), then the absolute-URL pass. Regressions pin the exact format
string both at unit level and through the real urllib3.connectionpool
DEBUG emit path; AGENTS.md wording now names both covered DEBUG shapes.
* fix(logging): redact urllib3 retry lines and linearize scheme scanning
Closes the two open review threads on the inbound-media log hardening:
Retry/redirect targets: urllib3 logs the request target with no scheme in
five shapes the generic absolute-URL pass cannot see - `Retry: <target>`
(connectionpool.py:954 DEBUG), `Incremented Retry for (url='<target>')`
(util/retry.py:545 DEBUG, absolute on the redirect path), `Retrying (...)
after connection broken by '<err>': <target>` (connectionpool.py:869
WARNING, above the INFO root), and origin-form halves of both Redirecting
emitters (poolmanager.py:500 INFO / connectionpool.py:922 DEBUG). Each
gets a rewrite anchored to the exact urllib3 format, collapsing the
target to /<redacted>; the generic pass's rest now stops at quote
characters so a quoted URL keeps its closing punctuation (previously the
absolute-form increment line was mangled), and the request-line method
class accepts any case. The emitter enumeration in channels AGENTS.md is
closed against the installed urllib3 2.7.0 source.
Quadratic scanning: both scheme-bearing patterns start with a character
class, so re.sub retried every suffix of a long token - 64K paths cost
~1.8s and URL-free 64K error bodies ~3.1s per record, synchronously in
every root handler. The two passes are now driven from literal "://"
occurrences: _scheme_starts walks back over the scheme charset to each
run's first letter and the pattern is attempted only there, reproducing
re.sub's leftmost-non-overlapping result in linear time (256K path:
5.6ms; worst adversarial shapes <= 28ms). Long-input regressions pin the
URL-bearing and URL-free cases with mutation-verified bounds, plus
nested-scheme and digit-headed-run equivalence cases.
Validation: tests/test_logging_config.py 12/12; scheme-pass equivalence
against the old re.sub pipeline verified by two independent 30k+ case
fuzz runs; full-suite A/B against HEAD shows zero tests that pass on HEAD
and fail with this diff.
* fix(logging): boundary-aware quote stops and whole-message Redirecting anchor
Two follow-ups on the urllib3 redaction shapes:
Embedded quotes: `rest` treated ANY quote as a closing mark, so a URL
with an apostrophe in the path kept everything after it verbatim
(`https://h/path'quoted'?token=Q` rendered the credential suffix in
full) while the class docstring claimed path/query/fragment are
replaced. A quote now closes `rest` only at a boundary - followed by
whitespace, a closing parenthesis, or end of string - so urllib3's
Incremented Retry (url='...') scaffolding keeps its ') closer while an
embedded quote stays consumed. The increment line's url capture gets
the same rule narrowed to its fixed ')' closer.
Redirecting anchoring: the origin-half pass matched `(?
<=-> )/path` as a substring, and an `-> /path` arrow is not
urllib3-owned shape - the sandbox provider's actionable mount error
(`sandbox.mounts entry <host> -> /mnt/knowledge ignored: ...`) had its
container path rewritten to /<redacted>, failing
test_setup_path_mappings_logs_actionable_error_for_missing_host_path on
CI (backend-unit-tests shard 3). The pass is now anchored to the whole
`Redirecting <t> -> <t>` message, which is exactly urllib3's record;
origin slots collapse, absolute slots stay for the generic pass.
Regression tests pin the embedded-quote shapes and the sandbox error's
byte-for-byte passthrough; both mutations verified red.
Validation: tests/test_logging_config.py 14/14; the CI-failing sandbox
test green locally; every test file asserting redaction/arrow log
content passes (attachments, support bundle, run metadata, skill
secrets, ragflow, skillscan, sandbox provider); full offline backend
suite 14084 passed / 164 failed with the failure set matching this
machine's documented Windows-environment baseline (NTFS chmod/symlink,
docker/lark/langfuse absences) - no failure involves redaction output.
* fix(logging): redact redirects with spaced locations
* fix(logging): grammar-complete Redirecting anchor; neutral WeChat guard labels
Round-13 P3 (Redirecting anchor strictness): the whole-message anchor kept
the ^Redirecting prefix (the urllib3-owned literal that stops the sandbox
false positive) but required BOTH slots whitespace-free, so a Location
header with an interior space voided the pass and leaked the origin-form
request target in the first slot - redirect_location is the raw header
string and interior spaces are legal field syntax. The tail is now loose
(\S.*$) and the first slot gets the same grammar treatment (\S.*?): the
recursive urlopen frame passes the previous raw Location as its url, so
t1 can carry interior spaces too, lazy-split at the first arrow the way
the line is constructed. A space-carrying slot collapses whole when it
starts with /; the sandbox mount error keeps passing through untouched.
Round-14 nit (None conflation): _download_cdn_bytes returns None for two
reasons (in-flight cap abort, Content-Encoding refusal) but both image and
file callers labeled it "exceeds size limit (N bytes)" - contradicting the
accurate encoding line right above it, and reporting the plaintext limit
for a ciphertext-cap decision. Callers now log a neutral
"skipped by download guard" line (the manager reader callers' shape);
the accurate reason stays inside the download function. The same sweep
also logs _stage_downloaded_file's silent None (no state dir configured),
which made an attachment vanish with no log line at all.
Also anchors the emitter-enumeration closure to its urllib3 version: the
closure reopens if an upgrade changes these format strings, so the comment
now says so explicitly.
Validation: logging 15/15 and attachments 60/62 (the two pre-existing
Windows symlink-privilege failures documented in the PR body); three
mutations verified red (old wording, strict t1, silent staging None);
ruff clean. Full offline suite run before push (per round-11 lesson).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): cap WeCom outbound content at the 20480-byte protocol limit
Both _send_ws paths sent unbounded text. Stream replies now clip on a
character boundary with a truncation marker (one stream carries the whole
reply and cannot split mid-way), and proactive pushes split into
sequential markdown messages at newline boundaries. Measured in UTF-8
bytes, matching the documented protocol cap.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* test(channels): pin emoji boundary behavior in the WeCom content limit
Review on #5148 raised 4-byte emoji cut points. Probes show the split
path already carries a byte-split character into the next chunk and
terminates on all-emoji input; these tests pin that behavior so a later
refactor cannot regress it.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(channels): preserve the delimiter when splitting WeCom pushes
The boundary newline was stripped by lstrip, so the sequential markdown
messages lost one delimiter per split and could not rebuild the original
response. Keep it on the chunk's tail and assert the exact round trip in
the tests, including leading blank lines.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(channels): serialize WeCom proactive chunk batches per chat
Each chunk send awaits, so two manager workers pushing long texts to the
same chat could interleave markdown messages (A1, B1, A2, B2) and break
the sequential-message contract. Hold a per-chat lock across the whole
split batch; different chats still send concurrently.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(channels): keep WeCom split advancing and cap the chunk batch
Two edge cases in _split_for_byte_limit left after the delimiter fix:
- A limit narrower than one whole character made the decode window
empty, so the hard cut became 0 and the loop appended empty chunks
forever. Take the character anyway when the window decodes empty, so
the loop always advances.
- A single oversized push produced one message per 20480 bytes with no
ceiling, flooding the chat and holding the per-chat lock for the whole
drain. Cap one push at 10 messages: keep the first nine verbatim and
collapse the rest into one clipped tail carrying the truncation
marker, with a warning log when the cap trips.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(channels): reclaim completed WeCom send locks, cap the split before it does the work
Two leftovers from the last review round:
- _ws_send_locks kept one lock per chat forever. A guard-locked refcount
now reclaims an entry only when no sender is queued on it, so a waiter
can never land on a fresh lock mid-batch for the same chat. Pinned by
three reclamation tests (single push, capped batch, 20 concurrent chats).
- _split_for_byte_limit built every chunk and then joined the discarded
tail to clip it — quadratic work on pathological pushes. The batch cap
now applies inside the loop: once the kept chunks are full, the
remainder is clipped whole. A spy test pins that the clipper receives
the unsplit remainder.
tests/test_wecom_content_limit.py 25 passed, plus tests/test_wecom_ws_text.py.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* docs(channels): describe the WeCom push cap, pin the staggered-lock race
The channels guide still described proactive pushes as an uncapped split. Also add the staggered-start regression the reviewer asked for: a waiter queuing while the holder's cleanup runs must share one lock, keep batches contiguous, and leave both lock registries empty.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
---------
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(gateway): confine artifact PUT to /mnt/user-data/outputs after path resolution
The outputs-only guard on PUT /api/threads/{id}/artifacts/{path} was a
string-prefix check on the raw path. A percent-encoded `..`
(`outputs/%2e%2e/uploads/x.txt`) survives nginx's variable proxy_pass
untouched, is decoded by Starlette, passes the prefix check, and the
resolver only confines the result to `user-data/` -- so an owner could
overwrite a sibling upload or workspace file in their own thread.
Collapse dot segments before the prefix check, and re-check the resolved
host path against the resolved outputs root so a symlink planted inside
`outputs/` cannot redirect the write either. The normalized virtual path
is what the response echoes and what non-mounted sandboxes receive.
* refactor(gateway): share the outputs-confinement rule with channel attachments
Review follow-up on #5321: the "only under /mnt/user-data/outputs" rule was
implemented independently by the artifact editor and by IM-channel
attachment delivery, and the two copies had already drifted.
Move it into app/gateway/path_utils.py as normalize_outputs_virtual_path
(collapse `..` before the prefix check) and resolve_outputs_confined_path
(re-check the resolved host path against the resolved outputs root, which
also catches a symlink planted inside outputs/). PUT /artifacts and
ChannelManager._resolve_attachments both call the helper; artifact_archive
keeps its stricter ZIP-member rules layered on top.
Tests that previously stubbed resolve_thread_virtual_path for the editor now
stub resolve_outputs_confined_path, and the channel attachment tests patch
path_utils.get_paths, which the helper binds at import like the other
consumers. The confinement itself is pinned by tests/test_gateway_path_utils.py.
* fix(channels): bound Discord outbound cross-loop awaits and restart dead clients, fixes#5226
* docs: reduce inherited agent guidance size
* fix(channels): stop half-started channels before discarding them
_start_channel now tears the instance down (stop + untrack) whenever start()
raises or the channel never reaches is_running, so an outbound listener
subscribed before the transport was confirmed cannot outlive its channel.
Addresses the review on #5227.
* fix(channels): retain half-started channels until failed-start cleanup completes
Ownership in _stop_and_discard_channel now mirrors ChannelService.stop(): the instance is dropped only after its stop() completes. A cancellation arriving mid-cleanup (or a stop() that raises) leaves it tracked, so a retried readiness attempt stops it again before replacing it and service shutdown can still reach it — untracking first orphaned resources nobody could clean up. Addresses the round-3 review on #5227.
* fix(channels): defer replacement when a retained channel fails to stop
The pre-retry stop in ensure_channel_ready popped unconditionally, so a
retained instance whose second stop() raised was untracked with its
outbound listener still subscribed — the same orphan one hop later.
restart_channel (del after failed stop) and remove_channel (pop before
stop) had the same shape. All three now route through
_stop_and_discard_channel and decline the operation for that round when
the instance is retained, so _start_channel can never overwrite a
still-listening channel. Addresses the review on #5227.
* fix(channels): enforce the retention guarantee inside the readiness attempt loop
A failed attempt whose cleanup retained the instance used to let the next
attempt (attempts=2 is the production default) construct a fresh instance
and overwrite the retained one via _start_channel's unconditional
assignment — orphaning the first instance's subscribed listener one hop
earlier than the cross-round guard covers. The guard now lives at the
mechanism: _start_channel refuses to install while the name is still
tracked, and ensure_channel_ready ends the loop on retention. The shared
discard helper's log message is path-neutral. Addresses the review on
#5227.
* fix(channels): make the retained-instance guard message path-neutral
The guard can fire for any still-tracked instance, not only failed
cleanup, so the message must not assume the cause.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs: govern agent guidance size
* refactor: split agent guidance by code scope
* Clarify virtual path handling in AGENTS.md
Updated the translation section to clarify the role of `LocalSandboxProvider` and the handling of virtual paths in the tool layer.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
