deer-flow/backend/tests/test_wechat_channel.py
hataa 26800d1245
fix(channels): stream-cap and validate WeChat/WeCom inbound media downloads, fixes #5223 (#5225)
* fix(channels): stream-cap and validate WeChat/WeCom inbound media downloads, fixes #5223

* fix(channels): address WeCom APPID, decompression, and log-sanitization review findings (#5223)

Round-4 review follow-ups on the inbound-media download cap:

- The COS bucket numeric suffix is the owner's Tencent Cloud APPID and bucket
  names are user-chosen, so any Tencent Cloud account could register a
  matching ww-aibot-img-* bucket and pass the shape gate. The built-in rule
  now admits only the APPID observed in Tencent's published aibot callback
  examples (1258476243), across regions; any other account (including a
  future WeCom rotation) goes through channels.wecom.allowed_media_hosts.
- aiter_bytes() transparently decodes Content-Encoding, and the decoder
  allocates the full decompressed body before the byte cap sees a chunk (an
  ~8 KB gzip wire chunk decoding to 8 MiB reproduces it). Both URL readers
  now send Accept-Encoding: identity, refuse a response with a residual
  Content-Encoding before reading, and iterate aiter_raw().
- httpx.HTTPStatusError formats the signed URL (path + query credentials)
  into its message, so _ingest_inbound_files' reader-failure branch logs a
  sanitized summary (class + status) instead of logger.exception, and the
  WeChat extract paths catch httpx.HTTPError so the polling loop's
  per-message logger.exception can never render a media URL.

Every change ships with a red/green regression: the reviewer's 403
mock-transport repro asserted against caplog.text (fully formatted logs),
the reviewer's different-APPID bucket host, and gzip bombs driven through
real httpx mock transports in both readers. Docs (channels AGENTS.md,
README, config.example.yaml) updated for the APPID pinning and encoding
gate.

* docs(channels): document why the inbound-media cap is 50 MB, not WeCom's 100 MB ceiling

* fix(logging): redact URLs in httpx request logs down to scheme + host, fixes #5223

httpx emits 'HTTP Request: GET <full URL>' at the Gateway's INFO level
before any response handling runs, so even successful signed-media
downloads leaked their credentials. HttpxUrlQueryRedactionFilter
(installed by configure_logging) rewrites those records in place — path
and query become /<redacted>, method/status/duration observability is
preserved — which also keeps Telegram's token-bearing Bot API paths out
of the logs. Reader-level regression tests run at production INFO level
with a real MockTransport, success paths included.

* fix(logging): blank userinfo credentials in httpx request-log redaction

* fix(logging): redact authority-only URLs and cover urllib3 redirect logs

Two follow-ups from the review plus one extrapolation of the same class:

- rest is now optional in _URL_REDACT_RE, so an authority-only URL
  (scheme://user:pass@host, no path) is rewritten too — userinfo had
  nowhere else to hide and previously passed through verbatim. A bare
  credential-free origin still passes through unchanged.
- Renamed to UrlRedactionFilter / install_url_log_redaction and attached
  to the urllib3 logger as well: urllib3 logs 'Redirecting <url> -> <url>'
  at INFO with full URLs on both sides, the same leak class on a different
  library logger. No gateway path today both uses requests and redirects
  a signed URL, but the class stays closed instead of dormant.
- Unit tests now build records with the real httpx 0.28.1 format string
  ('HTTP Request: %s %s "%s %d %s"', 5 args) and httpx.URL args, per
  the nit, instead of a synthetic shape httpx never emits.

* fix(logging): install URL redaction at handler level so propagated records are covered

A logging.Filter on a logger only runs for records emitted through that
exact logger — child loggers neither inherit it nor trigger it on
propagation — so the previous attachment to the bare urllib3 logger was
dead code: urllib3 emits Redirecting via urllib3.poolmanager at INFO and
urllib3.connectionpool at DEBUG. The filter is now attached to every root
handler (mirroring _install_trace_filter, which already iterates root
handlers; handler-level filters see propagated records) in addition to
the httpx logger (httpx emits via the bare name, and emission-point
coverage survives handlers added later). The wiring is pinned by tests
that emit through the real urllib3 child loggers — a mutation removing
the handler-level install turns them red. Comments, docstrings, and
AGENTS.md now state the actual emitter names and levels.

* fix(logging): redact urllib3 DEBUG request lines, whose split shape evaded the URL regex

urllib3's per-request line (connectionpool.py:545 on 2.7.0) renders as
`scheme://host:port "METHOD /path?query HTTP/x.x" status len` — the
authority ends at a space so _URL_REDACT_RE's bare-origin early return
applies, and the quoted origin-form target has no scheme, so neither half
was rewritten. UrlRedactionFilter now runs a dedicated request-line shape
first (collapsing the target to /<redacted>, keeping scheme+host+method+
version), then the absolute-URL pass. Regressions pin the exact format
string both at unit level and through the real urllib3.connectionpool
DEBUG emit path; AGENTS.md wording now names both covered DEBUG shapes.

* fix(logging): redact urllib3 retry lines and linearize scheme scanning

Closes the two open review threads on the inbound-media log hardening:

Retry/redirect targets: urllib3 logs the request target with no scheme in
five shapes the generic absolute-URL pass cannot see - `Retry: <target>`
(connectionpool.py:954 DEBUG), `Incremented Retry for (url='<target>')`
(util/retry.py:545 DEBUG, absolute on the redirect path), `Retrying (...)
after connection broken by '<err>': <target>` (connectionpool.py:869
WARNING, above the INFO root), and origin-form halves of both Redirecting
emitters (poolmanager.py:500 INFO / connectionpool.py:922 DEBUG). Each
gets a rewrite anchored to the exact urllib3 format, collapsing the
target to /<redacted>; the generic pass's rest now stops at quote
characters so a quoted URL keeps its closing punctuation (previously the
absolute-form increment line was mangled), and the request-line method
class accepts any case. The emitter enumeration in channels AGENTS.md is
closed against the installed urllib3 2.7.0 source.

Quadratic scanning: both scheme-bearing patterns start with a character
class, so re.sub retried every suffix of a long token - 64K paths cost
~1.8s and URL-free 64K error bodies ~3.1s per record, synchronously in
every root handler. The two passes are now driven from literal "://"
occurrences: _scheme_starts walks back over the scheme charset to each
run's first letter and the pattern is attempted only there, reproducing
re.sub's leftmost-non-overlapping result in linear time (256K path:
5.6ms; worst adversarial shapes <= 28ms). Long-input regressions pin the
URL-bearing and URL-free cases with mutation-verified bounds, plus
nested-scheme and digit-headed-run equivalence cases.

Validation: tests/test_logging_config.py 12/12; scheme-pass equivalence
against the old re.sub pipeline verified by two independent 30k+ case
fuzz runs; full-suite A/B against HEAD shows zero tests that pass on HEAD
and fail with this diff.

* fix(logging): boundary-aware quote stops and whole-message Redirecting anchor

Two follow-ups on the urllib3 redaction shapes:

Embedded quotes: `rest` treated ANY quote as a closing mark, so a URL
with an apostrophe in the path kept everything after it verbatim
(`https://h/path'quoted'?token=Q` rendered the credential suffix in
full) while the class docstring claimed path/query/fragment are
replaced. A quote now closes `rest` only at a boundary - followed by
whitespace, a closing parenthesis, or end of string - so urllib3's
Incremented Retry (url='...') scaffolding keeps its ') closer while an
embedded quote stays consumed. The increment line's url capture gets
the same rule narrowed to its fixed ')' closer.

Redirecting anchoring: the origin-half pass matched `(?
<=-> )/path` as a substring, and an `-> /path` arrow is not
urllib3-owned shape - the sandbox provider's actionable mount error
(`sandbox.mounts entry <host> -> /mnt/knowledge ignored: ...`) had its
container path rewritten to /<redacted>, failing
test_setup_path_mappings_logs_actionable_error_for_missing_host_path on
CI (backend-unit-tests shard 3). The pass is now anchored to the whole
`Redirecting <t> -> <t>` message, which is exactly urllib3's record;
origin slots collapse, absolute slots stay for the generic pass.
Regression tests pin the embedded-quote shapes and the sandbox error's
byte-for-byte passthrough; both mutations verified red.

Validation: tests/test_logging_config.py 14/14; the CI-failing sandbox
test green locally; every test file asserting redaction/arrow log
content passes (attachments, support bundle, run metadata, skill
secrets, ragflow, skillscan, sandbox provider); full offline backend
suite 14084 passed / 164 failed with the failure set matching this
machine's documented Windows-environment baseline (NTFS chmod/symlink,
docker/lark/langfuse absences) - no failure involves redaction output.

* fix(logging): redact redirects with spaced locations

* fix(logging): grammar-complete Redirecting anchor; neutral WeChat guard labels

Round-13 P3 (Redirecting anchor strictness): the whole-message anchor kept
the ^Redirecting prefix (the urllib3-owned literal that stops the sandbox
false positive) but required BOTH slots whitespace-free, so a Location
header with an interior space voided the pass and leaked the origin-form
request target in the first slot - redirect_location is the raw header
string and interior spaces are legal field syntax. The tail is now loose
(\S.*$) and the first slot gets the same grammar treatment (\S.*?): the
recursive urlopen frame passes the previous raw Location as its url, so
t1 can carry interior spaces too, lazy-split at the first arrow the way
the line is constructed. A space-carrying slot collapses whole when it
starts with /; the sandbox mount error keeps passing through untouched.

Round-14 nit (None conflation): _download_cdn_bytes returns None for two
reasons (in-flight cap abort, Content-Encoding refusal) but both image and
file callers labeled it "exceeds size limit (N bytes)" - contradicting the
accurate encoding line right above it, and reporting the plaintext limit
for a ciphertext-cap decision. Callers now log a neutral
"skipped by download guard" line (the manager reader callers' shape);
the accurate reason stays inside the download function. The same sweep
also logs _stage_downloaded_file's silent None (no state dir configured),
which made an attachment vanish with no log line at all.

Also anchors the emitter-enumeration closure to its urllib3 version: the
closure reopens if an upgrade changes these format strings, so the comment
now says so explicitly.

Validation: logging 15/15 and attachments 60/62 (the two pre-existing
Windows symlink-privilege failures documented in the PR body); three
mutations verified red (old wording, strict t1, silent staging None);
ruff clean. Full offline suite run before push (per round-11 lesson).

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-16 16:33:47 +08:00

1877 lines
68 KiB
Python

"""Tests for the WeChat IM channel."""
from __future__ import annotations
import asyncio
import base64
import json
import logging
import os
from pathlib import Path
from typing import Any
from unittest import mock
from unittest.mock import AsyncMock
import pytest
from app.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage
_POSIX_MODE_BITS_REASON = "Windows chmod only toggles the read-only bit, so the 0o600 mode asserted here is never observable"
def _run(coro):
loop = asyncio.new_event_loop()
try:
return loop.run_until_complete(coro)
finally:
loop.close()
class _MockResponse:
def __init__(self, payload: dict[str, Any], content: bytes | None = None):
self._payload = payload
self.content = content or b""
self.headers = payload.get("headers", {}) if isinstance(payload, dict) else {}
def raise_for_status(self) -> None:
return None
def json(self) -> dict[str, Any]:
return self._payload
class _MockAsyncClient:
def __init__(
self,
responses: list[dict[str, Any]] | None = None,
post_calls: list[dict[str, Any]] | None = None,
get_calls: list[dict[str, Any]] | None = None,
put_calls: list[dict[str, Any]] | None = None,
get_responses: list[dict[str, Any]] | None = None,
post_responses: list[dict[str, Any]] | None = None,
put_responses: list[dict[str, Any]] | None = None,
**kwargs,
):
self._responses = list(responses or [])
self._post_responses = list(post_responses or self._responses)
self._get_responses = list(get_responses or [])
self._put_responses = list(put_responses or [])
self._post_calls = post_calls
self._get_calls = get_calls
self._put_calls = put_calls
self.kwargs = kwargs
async def post(
self,
url: str,
json: dict[str, Any] | None = None,
headers: dict[str, Any] | None = None,
**kwargs,
):
if self._post_calls is not None:
self._post_calls.append({"url": url, "json": json or {}, "headers": headers or {}, **kwargs})
payload = self._post_responses.pop(0) if self._post_responses else {"ret": 0}
return _MockResponse(payload)
async def get(self, url: str, params: dict[str, Any] | None = None, headers: dict[str, Any] | None = None, **kwargs):
if self._get_calls is not None:
self._get_calls.append({"url": url, "params": params or {}, "headers": headers or {}, **kwargs})
payload = self._get_responses.pop(0) if self._get_responses else {"ret": 0}
return _MockResponse(payload)
async def put(self, url: str, content: bytes, headers: dict[str, Any] | None = None, **kwargs):
if self._put_calls is not None:
self._put_calls.append({"url": url, "content": content, "headers": headers or {}, **kwargs})
payload = self._put_responses.pop(0) if self._put_responses else {"ret": 0}
return _MockResponse(payload)
async def aclose(self) -> None:
return None
def test_timing_config_requires_positive_finite_values():
from app.channels.wechat import WechatChannel
timing_defaults = {
"polling_timeout": WechatChannel.DEFAULT_POLLING_TIMEOUT,
"polling_retry_delay": WechatChannel.DEFAULT_RETRY_DELAY,
"qrcode_poll_interval": WechatChannel.DEFAULT_QRCODE_POLL_INTERVAL,
"qrcode_poll_timeout": WechatChannel.DEFAULT_QRCODE_POLL_TIMEOUT,
}
attributes = {
"polling_timeout": "_polling_timeout",
"polling_retry_delay": "_retry_delay",
"qrcode_poll_interval": "_qrcode_poll_interval",
"qrcode_poll_timeout": "_qrcode_poll_timeout",
}
for invalid in (0, -1, float("nan"), float("inf"), float("-inf"), 10**1000):
channel = WechatChannel(
bus=MessageBus(),
config={"bot_token": "test-token", **dict.fromkeys(timing_defaults, invalid)},
)
assert {key: getattr(channel, attributes[key]) for key in timing_defaults} == timing_defaults
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "test-token", "polling_retry_delay": "0.25"})
assert channel._retry_delay == 0.25
def test_handle_update_publishes_private_chat_message():
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
channel = WechatChannel(bus=bus, config={"bot_token": "test-token"})
await channel._handle_update(
{
"message_type": 1,
"from_user_id": "wx-user-1",
"context_token": "ctx-1",
"item_list": [{"type": 1, "text_item": {"text": "hello from wechat"}}],
}
)
assert len(published) == 1
inbound = published[0]
assert inbound.chat_id == "wx-user-1"
assert inbound.user_id == "wx-user-1"
assert inbound.text == "hello from wechat"
assert inbound.msg_type == InboundMessageType.CHAT
assert inbound.topic_id is None
assert inbound.metadata["context_token"] == "ctx-1"
assert channel._context_tokens_by_chat["wx-user-1"] == "ctx-1"
_run(go())
def test_handle_update_downloads_inbound_image(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
plaintext = b"fake-image-bytes"
aes_key = b"1234567890abcdef"
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
encrypted = channel.__class__.__dict__["_extract_image_file"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
return encrypted
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 101,
"from_user_id": "wx-user-1",
"context_token": "ctx-img-1",
"item_list": [
{
"type": 2,
"image_item": {
"aeskey": aes_key.hex(),
"media": {"full_url": "https://cdn.weixin.qq.com/image.bin"},
},
}
],
}
)
assert len(published) == 1
inbound = published[0]
assert inbound.text == ""
assert len(inbound.files) == 1
file_info = inbound.files[0]
assert file_info["source"] == "wechat"
assert file_info["message_item_type"] == 2
stored = Path(file_info["path"])
assert stored.exists()
assert stored.read_bytes() == plaintext
_run(go())
def test_handle_update_downloads_inbound_png_with_png_extension(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
plaintext = b"\x89PNG\r\n\x1a\n" + b"png-body"
aes_key = b"1234567890abcdef"
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
encrypted = channel.__class__.__dict__["_extract_image_file"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
return encrypted
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 303,
"from_user_id": "wx-user-1",
"context_token": "ctx-img-png",
"item_list": [
{
"type": 2,
"image_item": {
"aeskey": aes_key.hex(),
"media": {"full_url": "https://cdn.weixin.qq.com/image.bin"},
},
}
],
}
)
assert len(published) == 1
file_info = published[0].files[0]
assert file_info["filename"].endswith(".png")
assert file_info["mime_type"] == "image/png"
_run(go())
def test_handle_update_preserves_text_and_ref_msg_with_image(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
plaintext = b"img-2"
aes_key = b"1234567890abcdef"
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
encrypted = channel.__class__.__dict__["_extract_image_file"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
return encrypted
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 202,
"from_user_id": "wx-user-1",
"context_token": "ctx-img-2",
"item_list": [
{"type": 1, "text_item": {"text": "look at this"}},
{
"type": 2,
"ref_msg": {"title": "quoted", "message_item": {"type": 1}},
"image_item": {
"aeskey": aes_key.hex(),
"media": {"full_url": "https://cdn.weixin.qq.com/image2.bin"},
},
},
],
}
)
assert len(published) == 1
inbound = published[0]
assert inbound.text == "look at this"
assert len(inbound.files) == 1
assert inbound.metadata["ref_msg"]["title"] == "quoted"
_run(go())
def test_handle_update_skips_image_without_url_or_key(tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
await channel._handle_update(
{
"message_type": 1,
"from_user_id": "wx-user-1",
"context_token": "ctx-img-3",
"item_list": [
{
"type": 2,
"image_item": {"media": {}},
}
],
}
)
assert published == []
_run(go())
def test_handle_update_routes_slash_command_as_command():
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
channel = WechatChannel(bus=bus, config={"bot_token": "test-token"})
await channel._handle_update(
{
"message_type": 1,
"from_user_id": "wx-user-1",
"context_token": "ctx-2",
"item_list": [{"type": 1, "text_item": {"text": "/status"}}],
}
)
assert len(published) == 1
assert published[0].msg_type == InboundMessageType.COMMAND
_run(go())
def test_allowed_users_filter_blocks_non_whitelisted_sender():
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "allowed_users": ["allowed-user"]})
await channel._handle_update(
{
"message_type": 1,
"from_user_id": "blocked-user",
"context_token": "ctx-3",
"item_list": [{"type": 1, "text_item": {"text": "hello"}}],
}
)
assert published == []
_run(go())
def test_connect_code_bypasses_allowed_users_filter(tmp_path: Path):
from app.channels.wechat import WechatChannel
from deerflow.persistence.channel_connections import ChannelConnectionRepository, ChannelCredentialCipher
from deerflow.persistence.engine import close_engine, get_session_factory, init_engine
async def go():
from datetime import UTC, datetime, timedelta
await init_engine("sqlite", url=f"sqlite+aiosqlite:///{tmp_path / 'wechat.db'}", sqlite_dir=str(tmp_path))
try:
repo = ChannelConnectionRepository(
get_session_factory(),
cipher=ChannelCredentialCipher.from_key("wechat-secret"),
)
code = "wechat-bind-code"
await repo.create_oauth_state(
owner_user_id="deerflow-user-1",
provider="wechat",
state=code,
expires_at=datetime.now(UTC) + timedelta(minutes=5),
)
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
# The newcomer ("blocked-user") is not in allowed_users yet, but a valid
# /connect code must still bootstrap their first bind.
channel = WechatChannel(
bus=bus,
config={"bot_token": "test-token", "allowed_users": ["allowed-user"], "connection_repo": repo},
)
channel._send_connection_reply = AsyncMock() # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"from_user_id": "blocked-user",
"context_token": "ctx-connect",
"item_list": [{"type": 1, "text_item": {"text": f"/connect {code}"}}],
}
)
connections = await repo.list_connections("deerflow-user-1")
assert len(connections) == 1
assert connections[0]["provider"] == "wechat"
assert connections[0]["external_account_id"] == "blocked-user"
# The connect-code reply was sent and no normal inbound was published.
channel._send_connection_reply.assert_awaited_once()
assert published == []
finally:
await close_engine()
_run(go())
def test_send_uses_cached_context_token(monkeypatch):
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(responses=[{"ret": 0}], post_calls=post_calls, **kwargs)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._context_tokens_by_chat["wx-user-1"] = "ctx-send"
await channel.send(
OutboundMessage(
channel_name="wechat",
chat_id="wx-user-1",
thread_id="thread-1",
text="reply text",
)
)
assert len(post_calls) == 1
assert post_calls[0]["url"].endswith("/ilink/bot/sendmessage")
assert post_calls[0]["json"]["msg"]["to_user_id"] == "wx-user-1"
assert post_calls[0]["json"]["msg"]["context_token"] == "ctx-send"
assert post_calls[0]["headers"]["Authorization"] == "Bearer bot-token"
assert post_calls[0]["headers"]["AuthorizationType"] == "ilink_bot_token"
assert "X-WECHAT-UIN" in post_calls[0]["headers"]
assert "iLink-App-ClientVersion" in post_calls[0]["headers"]
_run(go())
def test_send_skips_when_context_token_missing(monkeypatch):
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(responses=[{"ret": 0}], post_calls=post_calls, **kwargs)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
await channel.send(
OutboundMessage(
channel_name="wechat",
chat_id="wx-user-1",
thread_id="thread-1",
text="reply text",
)
)
assert post_calls == []
_run(go())
def test_protocol_helpers_build_expected_values():
from app.channels.wechat import (
MessageItemType,
UploadMediaType,
_build_ilink_client_version,
_build_wechat_uin,
_encrypted_size_for_aes_128_ecb,
)
assert int(MessageItemType.TEXT) == 1
assert int(UploadMediaType.FILE) == 3
assert _build_ilink_client_version("1.0.11") == str((1 << 16) | 11)
encoded = _build_wechat_uin()
decoded = base64.b64decode(encoded).decode("utf-8")
assert decoded.isdigit()
assert _encrypted_size_for_aes_128_ecb(0) == 16
assert _encrypted_size_for_aes_128_ecb(1) == 16
assert _encrypted_size_for_aes_128_ecb(16) == 32
def test_aes_roundtrip_encrypts_and_decrypts():
from app.channels.wechat import _decrypt_aes_128_ecb, _encrypt_aes_128_ecb
key = b"1234567890abcdef"
plaintext = b"hello-wechat-media"
encrypted = _encrypt_aes_128_ecb(plaintext, key)
assert encrypted != plaintext
decrypted = _decrypt_aes_128_ecb(encrypted, key)
assert decrypted == plaintext
def test_build_upload_request_supports_no_need_thumb():
from app.channels.wechat import UploadMediaType, WechatChannel
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
payload = channel._build_upload_request(
filekey="file-key-1",
media_type=UploadMediaType.IMAGE,
to_user_id="wx-user-1",
plaintext=b"image-bytes",
aes_key=b"1234567890abcdef",
no_need_thumb=True,
)
assert payload["filekey"] == "file-key-1"
assert payload["media_type"] == 1
assert payload["to_user_id"] == "wx-user-1"
assert payload["rawsize"] == len(b"image-bytes")
assert payload["filesize"] >= len(b"image-bytes")
assert payload["no_need_thumb"] is True
assert payload["aeskey"] == b"1234567890abcdef".hex()
def test_send_file_uploads_and_sends_image(monkeypatch, tmp_path: Path):
from app.channels.message_bus import ResolvedAttachment
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
put_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
post_calls=post_calls,
put_calls=put_calls,
post_responses=[
{
"ret": 0,
"upload_param": "enc-query-original",
"thumb_upload_param": "enc-query-thumb",
"upload_full_url": "https://cdn.example/upload-original",
},
{"ret": 0},
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
image_path = tmp_path / "chart.png"
image_path.write_bytes(b"png-binary-data")
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._context_tokens_by_chat["wx-user-1"] = "ctx-image-send"
ok = await channel.send_file(
OutboundMessage(
channel_name="wechat",
chat_id="wx-user-1",
thread_id="thread-1",
text="reply text",
),
ResolvedAttachment(
virtual_path="/mnt/user-data/outputs/chart.png",
actual_path=image_path,
filename="chart.png",
mime_type="image/png",
size=image_path.stat().st_size,
is_image=True,
),
)
assert ok is True
assert len(post_calls) == 3
assert post_calls[0]["url"].endswith("/ilink/bot/getuploadurl")
assert post_calls[0]["json"]["media_type"] == 1
assert post_calls[0]["json"]["no_need_thumb"] is True
assert len(put_calls) == 0
assert post_calls[1]["url"] == "https://cdn.example/upload-original"
assert post_calls[2]["url"].endswith("/ilink/bot/sendmessage")
image_item = post_calls[2]["json"]["msg"]["item_list"][0]["image_item"]
assert image_item["media"]["encrypt_query_param"] == "enc-query-original"
assert image_item["media"]["encrypt_type"] == 1
assert image_item["mid_size"] > 0
assert "thumb_media" not in image_item
assert "aeskey" not in image_item
assert base64.b64decode(image_item["media"]["aes_key"]).decode("utf-8") == post_calls[0]["json"]["aeskey"]
_run(go())
def test_send_file_returns_false_without_upload_full_url(monkeypatch, tmp_path: Path):
from app.channels.message_bus import ResolvedAttachment
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
post_calls=post_calls,
post_responses=[
{"ret": 0, "upload_param": "enc-query-only"},
{"ret": 0},
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
image_path = tmp_path / "chart.png"
image_path.write_bytes(b"png-binary-data")
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._context_tokens_by_chat["wx-user-1"] = "ctx-image-send"
ok = await channel.send_file(
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
ResolvedAttachment(
virtual_path="/mnt/user-data/outputs/chart.png",
actual_path=image_path,
filename="chart.png",
mime_type="image/png",
size=image_path.stat().st_size,
is_image=True,
),
)
assert ok is True
assert len(post_calls) == 3
assert post_calls[1]["url"].startswith("https://novac2c.cdn.weixin.qq.com/c2c/upload?")
assert post_calls[2]["url"].endswith("/ilink/bot/sendmessage")
image_item = post_calls[2]["json"]["msg"]["item_list"][0]["image_item"]
assert image_item["media"]["encrypt_query_param"] == "enc-query-only"
assert image_item["media"]["encrypt_type"] == 1
_run(go())
def test_send_file_prefers_cdn_response_header_for_image(monkeypatch, tmp_path: Path):
from app.channels.message_bus import ResolvedAttachment
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
post_calls=post_calls,
post_responses=[
{"ret": 0, "upload_param": "enc-query-original", "thumb_upload_param": "enc-query-thumb"},
{"ret": 0, "headers": {"x-encrypted-param": "enc-query-downloaded"}},
{"ret": 0},
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
image_path = tmp_path / "chart.png"
image_path.write_bytes(b"png-binary-data")
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._context_tokens_by_chat["wx-user-1"] = "ctx-image-send"
ok = await channel.send_file(
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
ResolvedAttachment(
virtual_path="/mnt/user-data/outputs/chart.png",
actual_path=image_path,
filename="chart.png",
mime_type="image/png",
size=image_path.stat().st_size,
is_image=True,
),
)
assert ok is True
assert post_calls[1]["url"].startswith("https://novac2c.cdn.weixin.qq.com/c2c/upload?")
image_item = post_calls[2]["json"]["msg"]["item_list"][0]["image_item"]
assert image_item["media"]["encrypt_query_param"] == "enc-query-downloaded"
assert image_item["media"]["encrypt_type"] == 1
assert "thumb_media" not in image_item
assert "aeskey" not in image_item
_run(go())
def test_send_file_skips_non_image(monkeypatch, tmp_path: Path):
from app.channels.message_bus import ResolvedAttachment
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(post_calls=post_calls, **kwargs)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
file_path = tmp_path / "notes.txt"
file_path.write_text("hello")
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
ok = await channel.send_file(
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
ResolvedAttachment(
virtual_path="/mnt/user-data/outputs/notes.txt",
actual_path=file_path,
filename="notes.txt",
mime_type="text/plain",
size=file_path.stat().st_size,
is_image=False,
),
)
assert ok is False
assert post_calls == []
_run(go())
def test_send_file_uploads_and_sends_regular_file(monkeypatch, tmp_path: Path):
from app.channels.message_bus import ResolvedAttachment
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
put_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
post_calls=post_calls,
put_calls=put_calls,
post_responses=[
{
"ret": 0,
"upload_param": "enc-query-file",
"upload_full_url": "https://cdn.example/upload-file",
},
{"ret": 0},
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
file_path = tmp_path / "report.pdf"
file_path.write_bytes(b"%PDF-1.4 fake")
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._context_tokens_by_chat["wx-user-1"] = "ctx-file-send"
ok = await channel.send_file(
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
ResolvedAttachment(
virtual_path="/mnt/user-data/outputs/report.pdf",
actual_path=file_path,
filename="report.pdf",
mime_type="application/pdf",
size=file_path.stat().st_size,
is_image=False,
),
)
assert ok is True
assert len(post_calls) == 3
assert post_calls[0]["url"].endswith("/ilink/bot/getuploadurl")
assert post_calls[0]["json"]["media_type"] == 3
assert post_calls[0]["json"]["no_need_thumb"] is True
assert len(put_calls) == 0
assert post_calls[1]["url"] == "https://cdn.example/upload-file"
assert post_calls[2]["url"].endswith("/ilink/bot/sendmessage")
file_item = post_calls[2]["json"]["msg"]["item_list"][0]["file_item"]
assert file_item["media"]["encrypt_query_param"] == "enc-query-file"
assert file_item["file_name"] == "report.pdf"
assert file_item["media"]["encrypt_type"] == 1
assert base64.b64decode(file_item["media"]["aes_key"]).decode("utf-8") == post_calls[0]["json"]["aeskey"]
_run(go())
def test_send_regular_file_uses_cdn_upload_fallback_when_upload_full_url_missing(monkeypatch, tmp_path: Path):
from app.channels.message_bus import ResolvedAttachment
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
post_calls=post_calls,
post_responses=[
{"ret": 0, "upload_param": "enc-query-file"},
{"ret": 0, "headers": {"x-encrypted-param": "enc-query-file-final"}},
{"ret": 0},
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
file_path = tmp_path / "report.pdf"
file_path.write_bytes(b"%PDF-1.4 fake")
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._context_tokens_by_chat["wx-user-1"] = "ctx-file-send"
ok = await channel.send_file(
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
ResolvedAttachment(
virtual_path="/mnt/user-data/outputs/report.pdf",
actual_path=file_path,
filename="report.pdf",
mime_type="application/pdf",
size=file_path.stat().st_size,
is_image=False,
),
)
assert ok is True
assert post_calls[1]["url"].startswith("https://novac2c.cdn.weixin.qq.com/c2c/upload?")
assert post_calls[2]["url"].endswith("/ilink/bot/sendmessage")
file_item = post_calls[2]["json"]["msg"]["item_list"][0]["file_item"]
assert file_item["media"]["encrypt_query_param"] == "enc-query-file-final"
assert file_item["media"]["encrypt_type"] == 1
_run(go())
def test_send_image_uses_post_even_when_upload_full_url_present(monkeypatch, tmp_path: Path):
from app.channels.message_bus import ResolvedAttachment
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
put_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
post_calls=post_calls,
put_calls=put_calls,
post_responses=[
{
"ret": 0,
"upload_param": "enc-query-original",
"thumb_upload_param": "enc-query-thumb",
"upload_full_url": "https://cdn.example/upload-original",
},
{"ret": 0, "headers": {"x-encrypted-param": "enc-query-downloaded"}},
{"ret": 0},
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
image_path = tmp_path / "chart.png"
image_path.write_bytes(b"png-binary-data")
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._context_tokens_by_chat["wx-user-1"] = "ctx-image-send"
ok = await channel.send_file(
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
ResolvedAttachment(
virtual_path="/mnt/user-data/outputs/chart.png",
actual_path=image_path,
filename="chart.png",
mime_type="image/png",
size=image_path.stat().st_size,
is_image=True,
),
)
assert ok is True
assert len(put_calls) == 0
assert post_calls[1]["url"] == "https://cdn.example/upload-original"
_run(go())
def test_send_file_blocks_disallowed_regular_file(monkeypatch, tmp_path: Path):
from app.channels.message_bus import ResolvedAttachment
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(post_calls=post_calls, **kwargs)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
file_path = tmp_path / "malware.exe"
file_path.write_bytes(b"MZ")
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._context_tokens_by_chat["wx-user-1"] = "ctx-file-send"
ok = await channel.send_file(
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
ResolvedAttachment(
virtual_path="/mnt/user-data/outputs/malware.exe",
actual_path=file_path,
filename="malware.exe",
mime_type="application/octet-stream",
size=file_path.stat().st_size,
is_image=False,
),
)
assert ok is False
assert post_calls == []
_run(go())
def test_handle_update_downloads_inbound_file(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
plaintext = b"hello,file"
aes_key = b"1234567890abcdef"
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
return encrypted
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 303,
"from_user_id": "wx-user-1",
"context_token": "ctx-file-1",
"item_list": [
{
"type": 4,
"file_item": {
"file_name": "report.pdf",
"aeskey": aes_key.hex(),
"media": {"full_url": "https://cdn.weixin.qq.com/report.bin"},
},
}
],
}
)
assert len(published) == 1
inbound = published[0]
assert inbound.text == ""
assert len(inbound.files) == 1
file_info = inbound.files[0]
assert file_info["message_item_type"] == 4
stored = Path(file_info["path"])
assert stored.exists()
assert stored.read_bytes() == plaintext
_run(go())
def test_handle_update_downloads_inbound_file_with_media_aeskey_hex(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
plaintext = b"hello,file"
aes_key = b"1234567890abcdef"
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
return encrypted
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 304,
"from_user_id": "wx-user-1",
"context_token": "ctx-file-1b",
"item_list": [
{
"type": 4,
"file_item": {
"file_name": "report.pdf",
"media": {
"full_url": "https://cdn.weixin.qq.com/report.bin",
"aeskey": aes_key.hex(),
},
},
}
],
}
)
assert len(published) == 1
assert published[0].files[0]["filename"] == "report.pdf"
_run(go())
def test_handle_update_downloads_inbound_file_with_unpadded_item_aes_key(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
plaintext = b"hello,file"
aes_key = b"1234567890abcdef"
encoded_key = base64.b64encode(aes_key).decode("utf-8").rstrip("=")
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
return encrypted
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 305,
"from_user_id": "wx-user-1",
"context_token": "ctx-file-1c",
"item_list": [
{
"type": 4,
"aesKey": encoded_key,
"file_item": {
"file_name": "report.pdf",
"media": {"full_url": "https://cdn.weixin.qq.com/report.bin"},
},
}
],
}
)
assert len(published) == 1
assert published[0].files[0]["filename"] == "report.pdf"
_run(go())
def test_handle_update_downloads_inbound_file_with_media_aes_key_base64_of_hex(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
plaintext = b"hello,file"
aes_key = b"1234567890abcdef"
encoded_hex_key = base64.b64encode(aes_key.hex().encode("utf-8")).decode("utf-8")
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
return encrypted
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 306,
"from_user_id": "wx-user-1",
"context_token": "ctx-file-1d",
"item_list": [
{
"type": 4,
"file_item": {
"file_name": "report.pdf",
"media": {
"full_url": "https://cdn.weixin.qq.com/report.bin",
"aes_key": encoded_hex_key,
},
},
}
],
}
)
assert len(published) == 1
assert published[0].files[0]["filename"] == "report.pdf"
_run(go())
def test_handle_update_skips_disallowed_inbound_file(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
plaintext = b"MZ"
aes_key = b"1234567890abcdef"
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
return encrypted
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 404,
"from_user_id": "wx-user-1",
"context_token": "ctx-file-2",
"item_list": [
{
"type": 4,
"file_item": {
"file_name": "malware.exe",
"aeskey": aes_key.hex(),
"media": {"full_url": "https://cdn.weixin.qq.com/bad.bin"},
},
}
],
}
)
assert published == []
_run(go())
def test_poll_loop_updates_server_timeout(monkeypatch):
from app.channels.wechat import WechatChannel
async def go():
post_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
post_calls=post_calls,
post_responses=[
{
"ret": 0,
"msgs": [
{
"message_type": 1,
"from_user_id": "wx-user-1",
"context_token": "ctx-1",
"item_list": [{"type": 1, "text_item": {"text": "hello"}}],
}
],
"get_updates_buf": "cursor-next",
"longpolling_timeout_ms": 42000,
}
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
channel._running = True
async def _fake_handle_update(_raw):
channel._running = False
return None
channel._handle_update = _fake_handle_update # type: ignore[method-assign]
await channel._poll_loop()
assert channel._get_updates_buf == "cursor-next"
assert channel._server_longpoll_timeout_seconds == 42.0
assert post_calls[0]["url"].endswith("/ilink/bot/getupdates")
_run(go())
def test_poll_loop_one_bad_message_does_not_permanently_lose_its_siblings(monkeypatch, tmp_path: Path, caplog):
"""A single message that fails to process must not sink the rest of its batch.
Regression test for a permanent message-loss bug: the long-poll cursor was
persisted for the *whole* batch before the per-message loop ran, and the loop
had no per-message error isolation, so one bad message (e.g. an attachment
that fails to decrypt) aborted processing of every message after it in the
same batch. Because the cursor had already advanced past the whole batch,
the next poll would never re-fetch the unprocessed tail -- silent, permanent
loss of every message after the first failure in a batch.
This test sends a real 3-message batch through the *real* (unstubbed)
``_handle_update`` -- message 1 is fine, message 2 is a WeChat image item
whose "encrypted" bytes are deliberately not a multiple of the AES block
size, so ``_decrypt_aes_128_ecb`` raises a genuine ``cryptography`` library
``ValueError`` (matching how a real corrupt/undecryptable attachment would
fail), and message 3 is fine again. The fix must:
- still deliver message 1 and message 3 to the bus despite message 2's
failure (per-message isolation instead of one bad apple aborting the
for loop), and
- log message 2's failure instead of swallowing it silently, and
- only advance/persist the cursor once the whole batch has been
attempted (so a crash mid-batch re-delivers rather than silently
drops).
"""
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published: list[Any] = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
aes_key = b"1234567890abcdef"
non_block_aligned_ciphertext = b"\x01\x02\x03\x04\x05" # 5 bytes: not a multiple of 16
msg_good_1 = {
"message_type": 1,
"message_id": "msg-1-good",
"from_user_id": "wx-user-1",
"context_token": "ctx-1",
"item_list": [{"type": 1, "text_item": {"text": "message one"}}],
}
msg_bad_2 = {
"message_type": 1,
"message_id": "msg-2-bad",
"from_user_id": "wx-user-1",
"context_token": "ctx-2",
"item_list": [
{
"type": 2,
"image_item": {
"aeskey": aes_key.hex(),
"media": {"full_url": "https://cdn.weixin.qq.com/corrupt-attachment.bin"},
},
}
],
}
msg_good_3 = {
"message_type": 1,
"message_id": "msg-3-good",
"from_user_id": "wx-user-1",
"context_token": "ctx-3",
"item_list": [{"type": 1, "text_item": {"text": "message three"}}],
}
state_dir = tmp_path / "wechat-state"
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
post_responses=[
{
"ret": 0,
"msgs": [msg_good_1, msg_bad_2, msg_good_3],
"get_updates_buf": "cursor-after-batch",
}
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
channel = WechatChannel(
bus=bus,
config={"bot_token": "test-token", "state_dir": str(state_dir), "polling_retry_delay": 0.001},
)
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None) -> bytes:
return non_block_aligned_ciphertext
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
# _handle_update is intentionally left as the REAL implementation so
# message 2 hits the genuine cryptography ValueError. To keep the test
# deterministic without depending on the bug/fix under test, force the
# poll loop to stop after exactly one getupdates cycle via
# _ensure_authenticated (called at the top of every iteration, before
# any message is processed) rather than via message content.
real_ensure_authenticated = channel._ensure_authenticated
auth_calls = {"n": 0}
async def _ensure_auth_then_stop() -> bool:
auth_calls["n"] += 1
if auth_calls["n"] > 1:
channel._running = False
return False
return await real_ensure_authenticated()
channel._ensure_authenticated = _ensure_auth_then_stop # type: ignore[method-assign]
channel._running = True
with caplog.at_level(logging.INFO, logger="app.channels.wechat"):
await channel._poll_loop()
# Message 1 and message 3 must both survive message 2's failure.
assert [m.text for m in published] == ["message one", "message three"]
# Message 2's failure must be logged, not silently swallowed.
messages = [record.getMessage() for record in caplog.records]
assert any("msg-2-bad" in message for message in messages)
# The cursor must reflect that the whole batch was attempted -- both
# in memory and in the persisted state file used to resume polling.
assert channel._get_updates_buf == "cursor-after-batch"
persisted = json.loads((state_dir / "wechat-getupdates.json").read_text(encoding="utf-8"))
assert persisted["get_updates_buf"] == "cursor-after-batch"
_run(go())
def test_state_cursor_is_loaded_from_disk(tmp_path: Path):
from app.channels.wechat import WechatChannel
state_dir = tmp_path / "wechat-state"
state_dir.mkdir(parents=True, exist_ok=True)
(state_dir / "wechat-getupdates.json").write_text(
json.dumps({"get_updates_buf": "cursor-123"}, ensure_ascii=False),
encoding="utf-8",
)
channel = WechatChannel(
bus=MessageBus(),
config={"bot_token": "bot-token", "state_dir": str(state_dir)},
)
# State load moved out of __init__ (it does filesystem IO that would block
# the async path); mirror start() by loading explicitly here.
channel._load_state()
assert channel._get_updates_buf == "cursor-123"
def test_auth_state_is_loaded_from_disk(tmp_path: Path):
from app.channels.wechat import WechatChannel
state_dir = tmp_path / "wechat-state"
state_dir.mkdir(parents=True, exist_ok=True)
(state_dir / "wechat-auth.json").write_text(
json.dumps({"status": "confirmed", "bot_token": "saved-token", "ilink_bot_id": "bot-1"}, ensure_ascii=False),
encoding="utf-8",
)
channel = WechatChannel(
bus=MessageBus(),
config={"state_dir": str(state_dir), "qrcode_login_enabled": True},
)
# State load moved out of __init__ (it does filesystem IO that would block
# the async path); mirror start() by loading explicitly here.
channel._load_state()
assert channel._bot_token == "saved-token"
assert channel._ilink_bot_id == "bot-1"
def test_qrcode_login_binds_and_persists_auth_state(monkeypatch, tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
get_calls: list[dict[str, Any]] = []
def _client_factory(*args, **kwargs):
return _MockAsyncClient(
get_calls=get_calls,
get_responses=[
{"qrcode": "qr-123", "qrcode_img_content": "https://example.com/qr.png"},
{"status": "confirmed", "bot_token": "bound-token", "ilink_bot_id": "bot-99"},
],
**kwargs,
)
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
state_dir = tmp_path / "wechat-state"
channel = WechatChannel(
bus=MessageBus(),
config={
"state_dir": str(state_dir),
"qrcode_login_enabled": True,
"qrcode_poll_interval": 0.01,
"qrcode_poll_timeout": 1,
},
)
ok = await channel._ensure_authenticated()
assert ok is True
assert channel._bot_token == "bound-token"
assert channel._ilink_bot_id == "bot-99"
assert get_calls[0]["url"].endswith("/ilink/bot/get_bot_qrcode")
assert get_calls[1]["url"].endswith("/ilink/bot/get_qrcode_status")
auth_state = json.loads((state_dir / "wechat-auth.json").read_text(encoding="utf-8"))
assert auth_state["status"] == "confirmed"
assert auth_state["bot_token"] == "bound-token"
assert auth_state["ilink_bot_id"] == "bot-99"
if os.name == "nt":
pytest.skip(_POSIX_MODE_BITS_REASON)
assert ((state_dir / "wechat-auth.json").stat().st_mode & 0o777) == 0o600
_run(go())
def test_save_auth_state_tightens_preexisting_loose_file(tmp_path: Path):
"""A world-readable auth file is replaced by an owner-only one, atomically.
The bot_token must never be observable at loose permissions: the atomic
0o600-temp + ``Path.replace`` path swaps in a fresh owner-only inode rather
than truncating the existing 0o644 file in place. Seeding the destination at
0o644 first means a regression back to ``write_text`` + late ``chmod`` would
leave a detectable window (and, here, the temp-file artifact behind).
"""
from app.channels.wechat import WechatChannel
state_dir = tmp_path / "wechat-state"
state_dir.mkdir(parents=True, exist_ok=True)
auth_path = state_dir / "wechat-auth.json"
auth_path.write_text(json.dumps({"status": "pending"}), encoding="utf-8")
auth_path.chmod(0o644)
channel = WechatChannel(
bus=MessageBus(),
config={"state_dir": str(state_dir), "qrcode_login_enabled": True},
)
channel._save_auth_state(status="confirmed", bot_token="bound-token", ilink_bot_id="bot-1")
assert json.loads(auth_path.read_text(encoding="utf-8"))["bot_token"] == "bound-token"
# Atomic write leaves no temp-file residue behind.
assert list(state_dir.glob("*.tmp")) == []
# Keep the platform-independent half running on Windows; only the mode-bit
# half of the "owner-only inode" contract is unobservable there.
if os.name == "nt":
pytest.skip(_POSIX_MODE_BITS_REASON)
assert (auth_path.stat().st_mode & 0o777) == 0o600
def test_save_auth_state_chmod_failure_is_logged_not_warned(tmp_path: Path, caplog):
"""A chmod failure on a perms-less filesystem must not look like a persist failure.
With the post-replace chmod split into its own try/except, a chmod ``OSError``
is logged at debug while the JSON is genuinely on disk — operators must not see
the misleading ``failed to persist`` warning that the shared try/except produced.
"""
from app.channels.wechat import WechatChannel
state_dir = tmp_path / "wechat-state"
channel = WechatChannel(
bus=MessageBus(),
config={"state_dir": str(state_dir), "qrcode_login_enabled": True},
)
real_chmod = Path.chmod
def chmod_spy(self: Path, mode: int, *args, **kwargs):
if self.suffix == ".json":
raise OSError("chmod unsupported on this filesystem")
return real_chmod(self, mode, *args, **kwargs)
with caplog.at_level(logging.DEBUG, logger="app.channels.wechat"), mock.patch.object(Path, "chmod", chmod_spy):
channel._save_auth_state(status="confirmed", bot_token="bound-token")
auth_path = state_dir / "wechat-auth.json"
assert json.loads(auth_path.read_text(encoding="utf-8"))["bot_token"] == "bound-token"
messages = [record.getMessage() for record in caplog.records]
assert any("unable to chmod auth state" in message for message in messages)
assert not any("failed to persist auth state" in message for message in messages)
# ---------------------------------------------------------------------------
# Inbound media download cap + destination allowlist
# ---------------------------------------------------------------------------
def test_is_allowed_media_url_suffix_boundaries():
from app.channels.wechat import WechatChannel
channel = WechatChannel(MessageBus(), config={"bot_token": "test-token"})
# Platform CDN defaults (plus the configured cdn_base_url host) are allowed.
assert channel._is_allowed_media_url("https://novac2c.cdn.weixin.qq.com/c2c/x?token=1")
assert channel._is_allowed_media_url("https://cdn.weixin.qq.com/image.bin")
# Dot-boundary suffix matching: lookalike hosts never match.
assert not channel._is_allowed_media_url("https://notqq.com/image.bin")
assert not channel._is_allowed_media_url("https://qq.com.evil.io/image.bin")
assert not channel._is_allowed_media_url("https://cdn.example/image.bin")
# Loopback/private targets and non-HTTP schemes are rejected outright.
assert not channel._is_allowed_media_url("http://127.0.0.1:8001/api/user")
assert not channel._is_allowed_media_url("http://169.254.169.254/latest/meta-data")
assert not channel._is_allowed_media_url("file:///etc/passwd")
# Operator suffixes extend the allowlist; the configured cdn_base_url host
# is admitted automatically so a custom CDN endpoint keeps working. A
# leading ``*.`` (DNS habit) normalizes to the bare suffix.
custom = WechatChannel(
MessageBus(),
config={
"bot_token": "test-token",
"cdn_base_url": "https://media.internal.example/c2c",
"allowed_media_hosts": ["cdn.example", "*.wild.example"],
},
)
assert custom._is_allowed_media_url("https://media.internal.example/c2c/x")
assert custom._is_allowed_media_url("https://a.cdn.example/image.bin")
assert custom._is_allowed_media_url("https://cdn.weixin.qq.com/image.bin")
assert custom._is_allowed_media_url("https://b.wild.example/image.bin")
def test_handle_update_skips_media_from_disallowed_host(tmp_path: Path):
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
channel = WechatChannel(bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
async def _forbidden_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None) -> bytes:
raise AssertionError("download must not be attempted for a disallowed media host")
channel._download_cdn_bytes = _forbidden_download # type: ignore[method-assign]
await channel._handle_update(
{
"message_type": 1,
"message_id": 201,
"from_user_id": "wx-user-1",
"context_token": "ctx-evil-1",
"item_list": [
{
"type": 2,
"image_item": {
"aeskey": b"1234567890abcdef".hex(),
"media": {"full_url": "https://evil.example/image.bin"},
},
}
],
}
)
# The image is dropped and, with no text either, nothing is published.
assert published == []
downloads_dir = tmp_path / "downloads"
assert not downloads_dir.exists() or not list(downloads_dir.iterdir())
_run(go())
def test_handle_update_http_download_failure_is_sanitized_not_raised(tmp_path: Path, caplog):
"""An HTTP failure during the media download drops that attachment with a
sanitized log instead of escaping to the polling loop's logger.exception.
httpx.HTTPStatusError formats the signed URL (path + query credentials)
into its message, so letting it propagate would render the URL in the
per-message traceback. Reproduced with a real 403 mock transport.
"""
import httpx
from app.channels.wechat import WechatChannel
async def go():
bus = MessageBus()
published = []
async def capture(msg):
published.append(msg)
bus.publish_inbound = capture # type: ignore[method-assign]
channel = WechatChannel(bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
channel._client = httpx.AsyncClient( # type: ignore[assignment]
transport=httpx.MockTransport(lambda _request: httpx.Response(403))
)
try:
await channel._handle_update(
{
"message_type": 1,
"message_id": 202,
"from_user_id": "wx-user-1",
"context_token": "ctx-403-1",
"item_list": [
{
"type": 2,
"image_item": {
"aeskey": b"1234567890abcdef".hex(),
"media": {"full_url": "https://cdn.weixin.qq.com/private/BearerSecret?token=QuerySecret"},
},
}
],
}
)
finally:
await channel._client.aclose()
# The image is dropped and, with no text either, nothing is published.
assert published == []
with caplog.at_level(logging.WARNING, logger="app.channels.wechat"):
_run(go())
assert "BearerSecret" not in caplog.text
assert "QuerySecret" not in caplog.text
assert "/private/" not in caplog.text
# The operator still sees what failed and for which host.
assert "HTTPStatusError (403)" in caplog.text
assert "cdn.weixin.qq.com" in caplog.text
class _FakeStreamResponse:
def __init__(self, chunks: list[bytes]):
self._chunks = chunks
self.headers: dict[str, str] = {}
def raise_for_status(self) -> None:
return None
# Deliberately no aiter_bytes: the reader must consume undecoded bytes, so
# an accidental switch back to the decoding iterator fails loudly here.
async def aiter_raw(self):
for chunk in self._chunks:
yield chunk
class _FakeStreamContext:
def __init__(self, response: _FakeStreamResponse):
self._response = response
async def __aenter__(self) -> _FakeStreamResponse:
return self._response
async def __aexit__(self, *exc_info) -> bool:
return False
class _FakeStreamingClient:
def __init__(self, chunks: list[bytes]):
self._chunks = chunks
def stream(self, _method: str, _url: str, timeout: float | None = None, **_kwargs) -> _FakeStreamContext:
return _FakeStreamContext(_FakeStreamResponse(self._chunks))
def test_download_cdn_bytes_aborts_when_stream_exceeds_cap():
from app.channels.wechat import WechatChannel
async def go():
channel = WechatChannel(MessageBus(), config={"bot_token": "test-token"})
channel._client = _FakeStreamingClient([b"abc", b"def"]) # type: ignore[assignment]
# 6 bytes total vs a 5-byte cap: aborted mid-stream before full read.
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=5) is None
# At/under the cap and with the cap disabled the chunks are joined as before.
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=6) == b"abcdef"
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=None) == b"abcdef"
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=0) == b"abcdef"
_run(go())
def test_download_cdn_bytes_rejects_compressed_response_before_decode(caplog):
"""aiter_bytes() would transparently decode Content-Encoding, allocating the
whole decompressed body before the cap sees a byte — an ~8 KB gzip wire
chunk decoding to 8 MiB (reproduced here) bypasses max_bytes entirely. The
download must request identity and refuse a residual encoding before
reading, even with the cap disabled."""
import gzip
import httpx
from app.channels.wechat import WechatChannel
class _AsyncChunks(httpx.AsyncByteStream):
def __init__(self, chunks: list[bytes]):
self._chunks = chunks
async def __aiter__(self):
for chunk in self._chunks:
yield chunk
seen_requests: list[httpx.Request] = []
def handler(request: httpx.Request) -> httpx.Response:
seen_requests.append(request)
return httpx.Response(
200,
headers={"Content-Encoding": "gzip"},
stream=_AsyncChunks([gzip.compress(b"\x00" * (8 * 1024 * 1024))]),
)
async def go():
channel = WechatChannel(MessageBus(), config={"bot_token": "test-token"})
channel._client = httpx.AsyncClient(transport=httpx.MockTransport(handler)) # type: ignore[assignment]
try:
# Cap disabled: the old aiter_bytes() code path would happily
# return the 8 MiB decompressed payload here.
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=None) is None
# With a cap in place the rejection still happens for the encoding,
# before any decode/size accounting.
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=1024 * 1024) is None
finally:
await channel._client.aclose()
with caplog.at_level(logging.WARNING, logger="app.channels.wechat"):
_run(go())
assert "Content-Encoding" in caplog.text
assert "exceeds" not in caplog.text # rejected for the encoding, not the size
assert seen_requests[0].headers.get("accept-encoding") == "identity"
def test_download_cdn_bytes_streams_identity_response():
"""An unencoded response still round-trips through a real httpx transport."""
import httpx
from app.channels.wechat import WechatChannel
class _AsyncChunks(httpx.AsyncByteStream):
def __init__(self, chunks: list[bytes]):
self._chunks = chunks
async def __aiter__(self):
for chunk in self._chunks:
yield chunk
def handler(_request: httpx.Request) -> httpx.Response:
return httpx.Response(200, stream=_AsyncChunks([b"abcdef"]))
async def go():
channel = WechatChannel(MessageBus(), config={"bot_token": "test-token"})
channel._client = httpx.AsyncClient(transport=httpx.MockTransport(handler)) # type: ignore[assignment]
try:
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x") == b"abcdef"
finally:
await channel._client.aclose()
_run(go())
def test_boundary_sized_inbound_image_survives_pkcs7_padding(tmp_path: Path):
"""A valid attachment whose plaintext is exactly the configured limit must load.
The limit bounds plaintext, but the stream measures ciphertext — AES-128-ECB
with PKCS#7 pads 32 plaintext bytes to 48 — so the stream cap must be the
padded size of exactly-limit plaintext, not the plaintext limit itself.
Exercises the real streaming download (no _download_cdn_bytes stub).
"""
from app.channels.wechat import WechatChannel, _encrypt_aes_128_ecb
async def go():
channel = WechatChannel(
MessageBus(),
config={"bot_token": "test-token", "state_dir": str(tmp_path), "max_inbound_image_bytes": 32},
)
aes_key = b"1234567890abcdef"
plaintext = b"\x89PNG\r\n\x1a\n" + b"x" * 24 # exactly 32 bytes, valid PNG magic
encrypted = _encrypt_aes_128_ecb(plaintext, aes_key) # 48 bytes > 32-byte limit
assert len(encrypted) == 48
channel._client = _FakeStreamingClient([encrypted]) # type: ignore[assignment]
assert channel._stream_cap_for(32) == 48
assert channel._stream_cap_for(0) is None
files = await channel._extract_inbound_files(
{
"message_id": 301,
"item_list": [
{
"type": 2,
"image_item": {
"aeskey": aes_key.hex(),
"media": {"full_url": "https://cdn.weixin.qq.com/image.bin"},
},
}
],
}
)
assert len(files) == 1
assert files[0]["type"] == "image"
assert files[0]["size"] == 32
assert Path(files[0]["path"]).read_bytes() == plaintext
_run(go())