mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-19 19:16:17 +00:00
* fix(channels): stream-cap and validate WeChat/WeCom inbound media downloads, fixes #5223 * fix(channels): address WeCom APPID, decompression, and log-sanitization review findings (#5223) Round-4 review follow-ups on the inbound-media download cap: - The COS bucket numeric suffix is the owner's Tencent Cloud APPID and bucket names are user-chosen, so any Tencent Cloud account could register a matching ww-aibot-img-* bucket and pass the shape gate. The built-in rule now admits only the APPID observed in Tencent's published aibot callback examples (1258476243), across regions; any other account (including a future WeCom rotation) goes through channels.wecom.allowed_media_hosts. - aiter_bytes() transparently decodes Content-Encoding, and the decoder allocates the full decompressed body before the byte cap sees a chunk (an ~8 KB gzip wire chunk decoding to 8 MiB reproduces it). Both URL readers now send Accept-Encoding: identity, refuse a response with a residual Content-Encoding before reading, and iterate aiter_raw(). - httpx.HTTPStatusError formats the signed URL (path + query credentials) into its message, so _ingest_inbound_files' reader-failure branch logs a sanitized summary (class + status) instead of logger.exception, and the WeChat extract paths catch httpx.HTTPError so the polling loop's per-message logger.exception can never render a media URL. Every change ships with a red/green regression: the reviewer's 403 mock-transport repro asserted against caplog.text (fully formatted logs), the reviewer's different-APPID bucket host, and gzip bombs driven through real httpx mock transports in both readers. Docs (channels AGENTS.md, README, config.example.yaml) updated for the APPID pinning and encoding gate. * docs(channels): document why the inbound-media cap is 50 MB, not WeCom's 100 MB ceiling * fix(logging): redact URLs in httpx request logs down to scheme + host, fixes #5223 httpx emits 'HTTP Request: GET <full URL>' at the Gateway's INFO level before any response handling runs, so even successful signed-media downloads leaked their credentials. HttpxUrlQueryRedactionFilter (installed by configure_logging) rewrites those records in place — path and query become /<redacted>, method/status/duration observability is preserved — which also keeps Telegram's token-bearing Bot API paths out of the logs. Reader-level regression tests run at production INFO level with a real MockTransport, success paths included. * fix(logging): blank userinfo credentials in httpx request-log redaction * fix(logging): redact authority-only URLs and cover urllib3 redirect logs Two follow-ups from the review plus one extrapolation of the same class: - rest is now optional in _URL_REDACT_RE, so an authority-only URL (scheme://user:pass@host, no path) is rewritten too — userinfo had nowhere else to hide and previously passed through verbatim. A bare credential-free origin still passes through unchanged. - Renamed to UrlRedactionFilter / install_url_log_redaction and attached to the urllib3 logger as well: urllib3 logs 'Redirecting <url> -> <url>' at INFO with full URLs on both sides, the same leak class on a different library logger. No gateway path today both uses requests and redirects a signed URL, but the class stays closed instead of dormant. - Unit tests now build records with the real httpx 0.28.1 format string ('HTTP Request: %s %s "%s %d %s"', 5 args) and httpx.URL args, per the nit, instead of a synthetic shape httpx never emits. * fix(logging): install URL redaction at handler level so propagated records are covered A logging.Filter on a logger only runs for records emitted through that exact logger — child loggers neither inherit it nor trigger it on propagation — so the previous attachment to the bare urllib3 logger was dead code: urllib3 emits Redirecting via urllib3.poolmanager at INFO and urllib3.connectionpool at DEBUG. The filter is now attached to every root handler (mirroring _install_trace_filter, which already iterates root handlers; handler-level filters see propagated records) in addition to the httpx logger (httpx emits via the bare name, and emission-point coverage survives handlers added later). The wiring is pinned by tests that emit through the real urllib3 child loggers — a mutation removing the handler-level install turns them red. Comments, docstrings, and AGENTS.md now state the actual emitter names and levels. * fix(logging): redact urllib3 DEBUG request lines, whose split shape evaded the URL regex urllib3's per-request line (connectionpool.py:545 on 2.7.0) renders as `scheme://host:port "METHOD /path?query HTTP/x.x" status len` — the authority ends at a space so _URL_REDACT_RE's bare-origin early return applies, and the quoted origin-form target has no scheme, so neither half was rewritten. UrlRedactionFilter now runs a dedicated request-line shape first (collapsing the target to /<redacted>, keeping scheme+host+method+ version), then the absolute-URL pass. Regressions pin the exact format string both at unit level and through the real urllib3.connectionpool DEBUG emit path; AGENTS.md wording now names both covered DEBUG shapes. * fix(logging): redact urllib3 retry lines and linearize scheme scanning Closes the two open review threads on the inbound-media log hardening: Retry/redirect targets: urllib3 logs the request target with no scheme in five shapes the generic absolute-URL pass cannot see - `Retry: <target>` (connectionpool.py:954 DEBUG), `Incremented Retry for (url='<target>')` (util/retry.py:545 DEBUG, absolute on the redirect path), `Retrying (...) after connection broken by '<err>': <target>` (connectionpool.py:869 WARNING, above the INFO root), and origin-form halves of both Redirecting emitters (poolmanager.py:500 INFO / connectionpool.py:922 DEBUG). Each gets a rewrite anchored to the exact urllib3 format, collapsing the target to /<redacted>; the generic pass's rest now stops at quote characters so a quoted URL keeps its closing punctuation (previously the absolute-form increment line was mangled), and the request-line method class accepts any case. The emitter enumeration in channels AGENTS.md is closed against the installed urllib3 2.7.0 source. Quadratic scanning: both scheme-bearing patterns start with a character class, so re.sub retried every suffix of a long token - 64K paths cost ~1.8s and URL-free 64K error bodies ~3.1s per record, synchronously in every root handler. The two passes are now driven from literal "://" occurrences: _scheme_starts walks back over the scheme charset to each run's first letter and the pattern is attempted only there, reproducing re.sub's leftmost-non-overlapping result in linear time (256K path: 5.6ms; worst adversarial shapes <= 28ms). Long-input regressions pin the URL-bearing and URL-free cases with mutation-verified bounds, plus nested-scheme and digit-headed-run equivalence cases. Validation: tests/test_logging_config.py 12/12; scheme-pass equivalence against the old re.sub pipeline verified by two independent 30k+ case fuzz runs; full-suite A/B against HEAD shows zero tests that pass on HEAD and fail with this diff. * fix(logging): boundary-aware quote stops and whole-message Redirecting anchor Two follow-ups on the urllib3 redaction shapes: Embedded quotes: `rest` treated ANY quote as a closing mark, so a URL with an apostrophe in the path kept everything after it verbatim (`https://h/path'quoted'?token=Q` rendered the credential suffix in full) while the class docstring claimed path/query/fragment are replaced. A quote now closes `rest` only at a boundary - followed by whitespace, a closing parenthesis, or end of string - so urllib3's Incremented Retry (url='...') scaffolding keeps its ') closer while an embedded quote stays consumed. The increment line's url capture gets the same rule narrowed to its fixed ')' closer. Redirecting anchoring: the origin-half pass matched `(? <=-> )/path` as a substring, and an `-> /path` arrow is not urllib3-owned shape - the sandbox provider's actionable mount error (`sandbox.mounts entry <host> -> /mnt/knowledge ignored: ...`) had its container path rewritten to /<redacted>, failing test_setup_path_mappings_logs_actionable_error_for_missing_host_path on CI (backend-unit-tests shard 3). The pass is now anchored to the whole `Redirecting <t> -> <t>` message, which is exactly urllib3's record; origin slots collapse, absolute slots stay for the generic pass. Regression tests pin the embedded-quote shapes and the sandbox error's byte-for-byte passthrough; both mutations verified red. Validation: tests/test_logging_config.py 14/14; the CI-failing sandbox test green locally; every test file asserting redaction/arrow log content passes (attachments, support bundle, run metadata, skill secrets, ragflow, skillscan, sandbox provider); full offline backend suite 14084 passed / 164 failed with the failure set matching this machine's documented Windows-environment baseline (NTFS chmod/symlink, docker/lark/langfuse absences) - no failure involves redaction output. * fix(logging): redact redirects with spaced locations * fix(logging): grammar-complete Redirecting anchor; neutral WeChat guard labels Round-13 P3 (Redirecting anchor strictness): the whole-message anchor kept the ^Redirecting prefix (the urllib3-owned literal that stops the sandbox false positive) but required BOTH slots whitespace-free, so a Location header with an interior space voided the pass and leaked the origin-form request target in the first slot - redirect_location is the raw header string and interior spaces are legal field syntax. The tail is now loose (\S.*$) and the first slot gets the same grammar treatment (\S.*?): the recursive urlopen frame passes the previous raw Location as its url, so t1 can carry interior spaces too, lazy-split at the first arrow the way the line is constructed. A space-carrying slot collapses whole when it starts with /; the sandbox mount error keeps passing through untouched. Round-14 nit (None conflation): _download_cdn_bytes returns None for two reasons (in-flight cap abort, Content-Encoding refusal) but both image and file callers labeled it "exceeds size limit (N bytes)" - contradicting the accurate encoding line right above it, and reporting the plaintext limit for a ciphertext-cap decision. Callers now log a neutral "skipped by download guard" line (the manager reader callers' shape); the accurate reason stays inside the download function. The same sweep also logs _stage_downloaded_file's silent None (no state dir configured), which made an attachment vanish with no log line at all. Also anchors the emitter-enumeration closure to its urllib3 version: the closure reopens if an upgrade changes these format strings, so the comment now says so explicitly. Validation: logging 15/15 and attachments 60/62 (the two pre-existing Windows symlink-privilege failures documented in the PR body); three mutations verified red (old wording, strict t1, silent staging None); ruff clean. Full offline suite run before push (per round-11 lesson). --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
1877 lines
68 KiB
Python
1877 lines
68 KiB
Python
"""Tests for the WeChat IM channel."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import base64
|
|
import json
|
|
import logging
|
|
import os
|
|
from pathlib import Path
|
|
from typing import Any
|
|
from unittest import mock
|
|
from unittest.mock import AsyncMock
|
|
|
|
import pytest
|
|
|
|
from app.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage
|
|
|
|
_POSIX_MODE_BITS_REASON = "Windows chmod only toggles the read-only bit, so the 0o600 mode asserted here is never observable"
|
|
|
|
|
|
def _run(coro):
|
|
loop = asyncio.new_event_loop()
|
|
try:
|
|
return loop.run_until_complete(coro)
|
|
finally:
|
|
loop.close()
|
|
|
|
|
|
class _MockResponse:
|
|
def __init__(self, payload: dict[str, Any], content: bytes | None = None):
|
|
self._payload = payload
|
|
self.content = content or b""
|
|
self.headers = payload.get("headers", {}) if isinstance(payload, dict) else {}
|
|
|
|
def raise_for_status(self) -> None:
|
|
return None
|
|
|
|
def json(self) -> dict[str, Any]:
|
|
return self._payload
|
|
|
|
|
|
class _MockAsyncClient:
|
|
def __init__(
|
|
self,
|
|
responses: list[dict[str, Any]] | None = None,
|
|
post_calls: list[dict[str, Any]] | None = None,
|
|
get_calls: list[dict[str, Any]] | None = None,
|
|
put_calls: list[dict[str, Any]] | None = None,
|
|
get_responses: list[dict[str, Any]] | None = None,
|
|
post_responses: list[dict[str, Any]] | None = None,
|
|
put_responses: list[dict[str, Any]] | None = None,
|
|
**kwargs,
|
|
):
|
|
self._responses = list(responses or [])
|
|
self._post_responses = list(post_responses or self._responses)
|
|
self._get_responses = list(get_responses or [])
|
|
self._put_responses = list(put_responses or [])
|
|
self._post_calls = post_calls
|
|
self._get_calls = get_calls
|
|
self._put_calls = put_calls
|
|
self.kwargs = kwargs
|
|
|
|
async def post(
|
|
self,
|
|
url: str,
|
|
json: dict[str, Any] | None = None,
|
|
headers: dict[str, Any] | None = None,
|
|
**kwargs,
|
|
):
|
|
if self._post_calls is not None:
|
|
self._post_calls.append({"url": url, "json": json or {}, "headers": headers or {}, **kwargs})
|
|
payload = self._post_responses.pop(0) if self._post_responses else {"ret": 0}
|
|
return _MockResponse(payload)
|
|
|
|
async def get(self, url: str, params: dict[str, Any] | None = None, headers: dict[str, Any] | None = None, **kwargs):
|
|
if self._get_calls is not None:
|
|
self._get_calls.append({"url": url, "params": params or {}, "headers": headers or {}, **kwargs})
|
|
payload = self._get_responses.pop(0) if self._get_responses else {"ret": 0}
|
|
return _MockResponse(payload)
|
|
|
|
async def put(self, url: str, content: bytes, headers: dict[str, Any] | None = None, **kwargs):
|
|
if self._put_calls is not None:
|
|
self._put_calls.append({"url": url, "content": content, "headers": headers or {}, **kwargs})
|
|
payload = self._put_responses.pop(0) if self._put_responses else {"ret": 0}
|
|
return _MockResponse(payload)
|
|
|
|
async def aclose(self) -> None:
|
|
return None
|
|
|
|
|
|
def test_timing_config_requires_positive_finite_values():
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
timing_defaults = {
|
|
"polling_timeout": WechatChannel.DEFAULT_POLLING_TIMEOUT,
|
|
"polling_retry_delay": WechatChannel.DEFAULT_RETRY_DELAY,
|
|
"qrcode_poll_interval": WechatChannel.DEFAULT_QRCODE_POLL_INTERVAL,
|
|
"qrcode_poll_timeout": WechatChannel.DEFAULT_QRCODE_POLL_TIMEOUT,
|
|
}
|
|
attributes = {
|
|
"polling_timeout": "_polling_timeout",
|
|
"polling_retry_delay": "_retry_delay",
|
|
"qrcode_poll_interval": "_qrcode_poll_interval",
|
|
"qrcode_poll_timeout": "_qrcode_poll_timeout",
|
|
}
|
|
|
|
for invalid in (0, -1, float("nan"), float("inf"), float("-inf"), 10**1000):
|
|
channel = WechatChannel(
|
|
bus=MessageBus(),
|
|
config={"bot_token": "test-token", **dict.fromkeys(timing_defaults, invalid)},
|
|
)
|
|
assert {key: getattr(channel, attributes[key]) for key in timing_defaults} == timing_defaults
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "test-token", "polling_retry_delay": "0.25"})
|
|
assert channel._retry_delay == 0.25
|
|
|
|
|
|
def test_handle_update_publishes_private_chat_message():
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token"})
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-1",
|
|
"item_list": [{"type": 1, "text_item": {"text": "hello from wechat"}}],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
inbound = published[0]
|
|
assert inbound.chat_id == "wx-user-1"
|
|
assert inbound.user_id == "wx-user-1"
|
|
assert inbound.text == "hello from wechat"
|
|
assert inbound.msg_type == InboundMessageType.CHAT
|
|
assert inbound.topic_id is None
|
|
assert inbound.metadata["context_token"] == "ctx-1"
|
|
assert channel._context_tokens_by_chat["wx-user-1"] == "ctx-1"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_downloads_inbound_image(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
plaintext = b"fake-image-bytes"
|
|
aes_key = b"1234567890abcdef"
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
encrypted = channel.__class__.__dict__["_extract_image_file"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
|
|
return encrypted
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 101,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-img-1",
|
|
"item_list": [
|
|
{
|
|
"type": 2,
|
|
"image_item": {
|
|
"aeskey": aes_key.hex(),
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/image.bin"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
inbound = published[0]
|
|
assert inbound.text == ""
|
|
assert len(inbound.files) == 1
|
|
file_info = inbound.files[0]
|
|
assert file_info["source"] == "wechat"
|
|
assert file_info["message_item_type"] == 2
|
|
stored = Path(file_info["path"])
|
|
assert stored.exists()
|
|
assert stored.read_bytes() == plaintext
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_downloads_inbound_png_with_png_extension(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
plaintext = b"\x89PNG\r\n\x1a\n" + b"png-body"
|
|
aes_key = b"1234567890abcdef"
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
encrypted = channel.__class__.__dict__["_extract_image_file"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
|
|
return encrypted
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 303,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-img-png",
|
|
"item_list": [
|
|
{
|
|
"type": 2,
|
|
"image_item": {
|
|
"aeskey": aes_key.hex(),
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/image.bin"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
file_info = published[0].files[0]
|
|
assert file_info["filename"].endswith(".png")
|
|
assert file_info["mime_type"] == "image/png"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_preserves_text_and_ref_msg_with_image(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
plaintext = b"img-2"
|
|
aes_key = b"1234567890abcdef"
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
encrypted = channel.__class__.__dict__["_extract_image_file"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
|
|
return encrypted
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 202,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-img-2",
|
|
"item_list": [
|
|
{"type": 1, "text_item": {"text": "look at this"}},
|
|
{
|
|
"type": 2,
|
|
"ref_msg": {"title": "quoted", "message_item": {"type": 1}},
|
|
"image_item": {
|
|
"aeskey": aes_key.hex(),
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/image2.bin"},
|
|
},
|
|
},
|
|
],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
inbound = published[0]
|
|
assert inbound.text == "look at this"
|
|
assert len(inbound.files) == 1
|
|
assert inbound.metadata["ref_msg"]["title"] == "quoted"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_skips_image_without_url_or_key(tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-img-3",
|
|
"item_list": [
|
|
{
|
|
"type": 2,
|
|
"image_item": {"media": {}},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert published == []
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_routes_slash_command_as_command():
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token"})
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-2",
|
|
"item_list": [{"type": 1, "text_item": {"text": "/status"}}],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
assert published[0].msg_type == InboundMessageType.COMMAND
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_allowed_users_filter_blocks_non_whitelisted_sender():
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "allowed_users": ["allowed-user"]})
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"from_user_id": "blocked-user",
|
|
"context_token": "ctx-3",
|
|
"item_list": [{"type": 1, "text_item": {"text": "hello"}}],
|
|
}
|
|
)
|
|
|
|
assert published == []
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_connect_code_bypasses_allowed_users_filter(tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
from deerflow.persistence.channel_connections import ChannelConnectionRepository, ChannelCredentialCipher
|
|
from deerflow.persistence.engine import close_engine, get_session_factory, init_engine
|
|
|
|
async def go():
|
|
from datetime import UTC, datetime, timedelta
|
|
|
|
await init_engine("sqlite", url=f"sqlite+aiosqlite:///{tmp_path / 'wechat.db'}", sqlite_dir=str(tmp_path))
|
|
try:
|
|
repo = ChannelConnectionRepository(
|
|
get_session_factory(),
|
|
cipher=ChannelCredentialCipher.from_key("wechat-secret"),
|
|
)
|
|
code = "wechat-bind-code"
|
|
await repo.create_oauth_state(
|
|
owner_user_id="deerflow-user-1",
|
|
provider="wechat",
|
|
state=code,
|
|
expires_at=datetime.now(UTC) + timedelta(minutes=5),
|
|
)
|
|
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
# The newcomer ("blocked-user") is not in allowed_users yet, but a valid
|
|
# /connect code must still bootstrap their first bind.
|
|
channel = WechatChannel(
|
|
bus=bus,
|
|
config={"bot_token": "test-token", "allowed_users": ["allowed-user"], "connection_repo": repo},
|
|
)
|
|
channel._send_connection_reply = AsyncMock() # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"from_user_id": "blocked-user",
|
|
"context_token": "ctx-connect",
|
|
"item_list": [{"type": 1, "text_item": {"text": f"/connect {code}"}}],
|
|
}
|
|
)
|
|
|
|
connections = await repo.list_connections("deerflow-user-1")
|
|
assert len(connections) == 1
|
|
assert connections[0]["provider"] == "wechat"
|
|
assert connections[0]["external_account_id"] == "blocked-user"
|
|
# The connect-code reply was sent and no normal inbound was published.
|
|
channel._send_connection_reply.assert_awaited_once()
|
|
assert published == []
|
|
finally:
|
|
await close_engine()
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_uses_cached_context_token(monkeypatch):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(responses=[{"ret": 0}], post_calls=post_calls, **kwargs)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._context_tokens_by_chat["wx-user-1"] = "ctx-send"
|
|
|
|
await channel.send(
|
|
OutboundMessage(
|
|
channel_name="wechat",
|
|
chat_id="wx-user-1",
|
|
thread_id="thread-1",
|
|
text="reply text",
|
|
)
|
|
)
|
|
|
|
assert len(post_calls) == 1
|
|
assert post_calls[0]["url"].endswith("/ilink/bot/sendmessage")
|
|
assert post_calls[0]["json"]["msg"]["to_user_id"] == "wx-user-1"
|
|
assert post_calls[0]["json"]["msg"]["context_token"] == "ctx-send"
|
|
assert post_calls[0]["headers"]["Authorization"] == "Bearer bot-token"
|
|
assert post_calls[0]["headers"]["AuthorizationType"] == "ilink_bot_token"
|
|
assert "X-WECHAT-UIN" in post_calls[0]["headers"]
|
|
assert "iLink-App-ClientVersion" in post_calls[0]["headers"]
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_skips_when_context_token_missing(monkeypatch):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(responses=[{"ret": 0}], post_calls=post_calls, **kwargs)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
await channel.send(
|
|
OutboundMessage(
|
|
channel_name="wechat",
|
|
chat_id="wx-user-1",
|
|
thread_id="thread-1",
|
|
text="reply text",
|
|
)
|
|
)
|
|
|
|
assert post_calls == []
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_protocol_helpers_build_expected_values():
|
|
from app.channels.wechat import (
|
|
MessageItemType,
|
|
UploadMediaType,
|
|
_build_ilink_client_version,
|
|
_build_wechat_uin,
|
|
_encrypted_size_for_aes_128_ecb,
|
|
)
|
|
|
|
assert int(MessageItemType.TEXT) == 1
|
|
assert int(UploadMediaType.FILE) == 3
|
|
assert _build_ilink_client_version("1.0.11") == str((1 << 16) | 11)
|
|
|
|
encoded = _build_wechat_uin()
|
|
decoded = base64.b64decode(encoded).decode("utf-8")
|
|
assert decoded.isdigit()
|
|
|
|
assert _encrypted_size_for_aes_128_ecb(0) == 16
|
|
assert _encrypted_size_for_aes_128_ecb(1) == 16
|
|
assert _encrypted_size_for_aes_128_ecb(16) == 32
|
|
|
|
|
|
def test_aes_roundtrip_encrypts_and_decrypts():
|
|
from app.channels.wechat import _decrypt_aes_128_ecb, _encrypt_aes_128_ecb
|
|
|
|
key = b"1234567890abcdef"
|
|
plaintext = b"hello-wechat-media"
|
|
|
|
encrypted = _encrypt_aes_128_ecb(plaintext, key)
|
|
assert encrypted != plaintext
|
|
|
|
decrypted = _decrypt_aes_128_ecb(encrypted, key)
|
|
assert decrypted == plaintext
|
|
|
|
|
|
def test_build_upload_request_supports_no_need_thumb():
|
|
from app.channels.wechat import UploadMediaType, WechatChannel
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
payload = channel._build_upload_request(
|
|
filekey="file-key-1",
|
|
media_type=UploadMediaType.IMAGE,
|
|
to_user_id="wx-user-1",
|
|
plaintext=b"image-bytes",
|
|
aes_key=b"1234567890abcdef",
|
|
no_need_thumb=True,
|
|
)
|
|
|
|
assert payload["filekey"] == "file-key-1"
|
|
assert payload["media_type"] == 1
|
|
assert payload["to_user_id"] == "wx-user-1"
|
|
assert payload["rawsize"] == len(b"image-bytes")
|
|
assert payload["filesize"] >= len(b"image-bytes")
|
|
assert payload["no_need_thumb"] is True
|
|
assert payload["aeskey"] == b"1234567890abcdef".hex()
|
|
|
|
|
|
def test_send_file_uploads_and_sends_image(monkeypatch, tmp_path: Path):
|
|
from app.channels.message_bus import ResolvedAttachment
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
put_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
post_calls=post_calls,
|
|
put_calls=put_calls,
|
|
post_responses=[
|
|
{
|
|
"ret": 0,
|
|
"upload_param": "enc-query-original",
|
|
"thumb_upload_param": "enc-query-thumb",
|
|
"upload_full_url": "https://cdn.example/upload-original",
|
|
},
|
|
{"ret": 0},
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
image_path = tmp_path / "chart.png"
|
|
image_path.write_bytes(b"png-binary-data")
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._context_tokens_by_chat["wx-user-1"] = "ctx-image-send"
|
|
|
|
ok = await channel.send_file(
|
|
OutboundMessage(
|
|
channel_name="wechat",
|
|
chat_id="wx-user-1",
|
|
thread_id="thread-1",
|
|
text="reply text",
|
|
),
|
|
ResolvedAttachment(
|
|
virtual_path="/mnt/user-data/outputs/chart.png",
|
|
actual_path=image_path,
|
|
filename="chart.png",
|
|
mime_type="image/png",
|
|
size=image_path.stat().st_size,
|
|
is_image=True,
|
|
),
|
|
)
|
|
|
|
assert ok is True
|
|
assert len(post_calls) == 3
|
|
assert post_calls[0]["url"].endswith("/ilink/bot/getuploadurl")
|
|
assert post_calls[0]["json"]["media_type"] == 1
|
|
assert post_calls[0]["json"]["no_need_thumb"] is True
|
|
assert len(put_calls) == 0
|
|
assert post_calls[1]["url"] == "https://cdn.example/upload-original"
|
|
assert post_calls[2]["url"].endswith("/ilink/bot/sendmessage")
|
|
image_item = post_calls[2]["json"]["msg"]["item_list"][0]["image_item"]
|
|
assert image_item["media"]["encrypt_query_param"] == "enc-query-original"
|
|
assert image_item["media"]["encrypt_type"] == 1
|
|
assert image_item["mid_size"] > 0
|
|
assert "thumb_media" not in image_item
|
|
assert "aeskey" not in image_item
|
|
assert base64.b64decode(image_item["media"]["aes_key"]).decode("utf-8") == post_calls[0]["json"]["aeskey"]
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_file_returns_false_without_upload_full_url(monkeypatch, tmp_path: Path):
|
|
from app.channels.message_bus import ResolvedAttachment
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
post_calls=post_calls,
|
|
post_responses=[
|
|
{"ret": 0, "upload_param": "enc-query-only"},
|
|
{"ret": 0},
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
image_path = tmp_path / "chart.png"
|
|
image_path.write_bytes(b"png-binary-data")
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._context_tokens_by_chat["wx-user-1"] = "ctx-image-send"
|
|
|
|
ok = await channel.send_file(
|
|
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
|
|
ResolvedAttachment(
|
|
virtual_path="/mnt/user-data/outputs/chart.png",
|
|
actual_path=image_path,
|
|
filename="chart.png",
|
|
mime_type="image/png",
|
|
size=image_path.stat().st_size,
|
|
is_image=True,
|
|
),
|
|
)
|
|
|
|
assert ok is True
|
|
assert len(post_calls) == 3
|
|
assert post_calls[1]["url"].startswith("https://novac2c.cdn.weixin.qq.com/c2c/upload?")
|
|
assert post_calls[2]["url"].endswith("/ilink/bot/sendmessage")
|
|
image_item = post_calls[2]["json"]["msg"]["item_list"][0]["image_item"]
|
|
assert image_item["media"]["encrypt_query_param"] == "enc-query-only"
|
|
assert image_item["media"]["encrypt_type"] == 1
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_file_prefers_cdn_response_header_for_image(monkeypatch, tmp_path: Path):
|
|
from app.channels.message_bus import ResolvedAttachment
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
post_calls=post_calls,
|
|
post_responses=[
|
|
{"ret": 0, "upload_param": "enc-query-original", "thumb_upload_param": "enc-query-thumb"},
|
|
{"ret": 0, "headers": {"x-encrypted-param": "enc-query-downloaded"}},
|
|
{"ret": 0},
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
image_path = tmp_path / "chart.png"
|
|
image_path.write_bytes(b"png-binary-data")
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._context_tokens_by_chat["wx-user-1"] = "ctx-image-send"
|
|
|
|
ok = await channel.send_file(
|
|
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
|
|
ResolvedAttachment(
|
|
virtual_path="/mnt/user-data/outputs/chart.png",
|
|
actual_path=image_path,
|
|
filename="chart.png",
|
|
mime_type="image/png",
|
|
size=image_path.stat().st_size,
|
|
is_image=True,
|
|
),
|
|
)
|
|
|
|
assert ok is True
|
|
assert post_calls[1]["url"].startswith("https://novac2c.cdn.weixin.qq.com/c2c/upload?")
|
|
image_item = post_calls[2]["json"]["msg"]["item_list"][0]["image_item"]
|
|
assert image_item["media"]["encrypt_query_param"] == "enc-query-downloaded"
|
|
assert image_item["media"]["encrypt_type"] == 1
|
|
assert "thumb_media" not in image_item
|
|
assert "aeskey" not in image_item
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_file_skips_non_image(monkeypatch, tmp_path: Path):
|
|
from app.channels.message_bus import ResolvedAttachment
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(post_calls=post_calls, **kwargs)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
file_path = tmp_path / "notes.txt"
|
|
file_path.write_text("hello")
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
ok = await channel.send_file(
|
|
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
|
|
ResolvedAttachment(
|
|
virtual_path="/mnt/user-data/outputs/notes.txt",
|
|
actual_path=file_path,
|
|
filename="notes.txt",
|
|
mime_type="text/plain",
|
|
size=file_path.stat().st_size,
|
|
is_image=False,
|
|
),
|
|
)
|
|
|
|
assert ok is False
|
|
assert post_calls == []
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_file_uploads_and_sends_regular_file(monkeypatch, tmp_path: Path):
|
|
from app.channels.message_bus import ResolvedAttachment
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
put_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
post_calls=post_calls,
|
|
put_calls=put_calls,
|
|
post_responses=[
|
|
{
|
|
"ret": 0,
|
|
"upload_param": "enc-query-file",
|
|
"upload_full_url": "https://cdn.example/upload-file",
|
|
},
|
|
{"ret": 0},
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
file_path = tmp_path / "report.pdf"
|
|
file_path.write_bytes(b"%PDF-1.4 fake")
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._context_tokens_by_chat["wx-user-1"] = "ctx-file-send"
|
|
|
|
ok = await channel.send_file(
|
|
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
|
|
ResolvedAttachment(
|
|
virtual_path="/mnt/user-data/outputs/report.pdf",
|
|
actual_path=file_path,
|
|
filename="report.pdf",
|
|
mime_type="application/pdf",
|
|
size=file_path.stat().st_size,
|
|
is_image=False,
|
|
),
|
|
)
|
|
|
|
assert ok is True
|
|
assert len(post_calls) == 3
|
|
assert post_calls[0]["url"].endswith("/ilink/bot/getuploadurl")
|
|
assert post_calls[0]["json"]["media_type"] == 3
|
|
assert post_calls[0]["json"]["no_need_thumb"] is True
|
|
assert len(put_calls) == 0
|
|
assert post_calls[1]["url"] == "https://cdn.example/upload-file"
|
|
assert post_calls[2]["url"].endswith("/ilink/bot/sendmessage")
|
|
file_item = post_calls[2]["json"]["msg"]["item_list"][0]["file_item"]
|
|
assert file_item["media"]["encrypt_query_param"] == "enc-query-file"
|
|
assert file_item["file_name"] == "report.pdf"
|
|
assert file_item["media"]["encrypt_type"] == 1
|
|
assert base64.b64decode(file_item["media"]["aes_key"]).decode("utf-8") == post_calls[0]["json"]["aeskey"]
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_regular_file_uses_cdn_upload_fallback_when_upload_full_url_missing(monkeypatch, tmp_path: Path):
|
|
from app.channels.message_bus import ResolvedAttachment
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
post_calls=post_calls,
|
|
post_responses=[
|
|
{"ret": 0, "upload_param": "enc-query-file"},
|
|
{"ret": 0, "headers": {"x-encrypted-param": "enc-query-file-final"}},
|
|
{"ret": 0},
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
file_path = tmp_path / "report.pdf"
|
|
file_path.write_bytes(b"%PDF-1.4 fake")
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._context_tokens_by_chat["wx-user-1"] = "ctx-file-send"
|
|
|
|
ok = await channel.send_file(
|
|
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
|
|
ResolvedAttachment(
|
|
virtual_path="/mnt/user-data/outputs/report.pdf",
|
|
actual_path=file_path,
|
|
filename="report.pdf",
|
|
mime_type="application/pdf",
|
|
size=file_path.stat().st_size,
|
|
is_image=False,
|
|
),
|
|
)
|
|
|
|
assert ok is True
|
|
assert post_calls[1]["url"].startswith("https://novac2c.cdn.weixin.qq.com/c2c/upload?")
|
|
assert post_calls[2]["url"].endswith("/ilink/bot/sendmessage")
|
|
file_item = post_calls[2]["json"]["msg"]["item_list"][0]["file_item"]
|
|
assert file_item["media"]["encrypt_query_param"] == "enc-query-file-final"
|
|
assert file_item["media"]["encrypt_type"] == 1
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_image_uses_post_even_when_upload_full_url_present(monkeypatch, tmp_path: Path):
|
|
from app.channels.message_bus import ResolvedAttachment
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
put_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
post_calls=post_calls,
|
|
put_calls=put_calls,
|
|
post_responses=[
|
|
{
|
|
"ret": 0,
|
|
"upload_param": "enc-query-original",
|
|
"thumb_upload_param": "enc-query-thumb",
|
|
"upload_full_url": "https://cdn.example/upload-original",
|
|
},
|
|
{"ret": 0, "headers": {"x-encrypted-param": "enc-query-downloaded"}},
|
|
{"ret": 0},
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
image_path = tmp_path / "chart.png"
|
|
image_path.write_bytes(b"png-binary-data")
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._context_tokens_by_chat["wx-user-1"] = "ctx-image-send"
|
|
|
|
ok = await channel.send_file(
|
|
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
|
|
ResolvedAttachment(
|
|
virtual_path="/mnt/user-data/outputs/chart.png",
|
|
actual_path=image_path,
|
|
filename="chart.png",
|
|
mime_type="image/png",
|
|
size=image_path.stat().st_size,
|
|
is_image=True,
|
|
),
|
|
)
|
|
|
|
assert ok is True
|
|
assert len(put_calls) == 0
|
|
assert post_calls[1]["url"] == "https://cdn.example/upload-original"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_send_file_blocks_disallowed_regular_file(monkeypatch, tmp_path: Path):
|
|
from app.channels.message_bus import ResolvedAttachment
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(post_calls=post_calls, **kwargs)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
file_path = tmp_path / "malware.exe"
|
|
file_path.write_bytes(b"MZ")
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._context_tokens_by_chat["wx-user-1"] = "ctx-file-send"
|
|
|
|
ok = await channel.send_file(
|
|
OutboundMessage(channel_name="wechat", chat_id="wx-user-1", thread_id="thread-1", text="reply text"),
|
|
ResolvedAttachment(
|
|
virtual_path="/mnt/user-data/outputs/malware.exe",
|
|
actual_path=file_path,
|
|
filename="malware.exe",
|
|
mime_type="application/octet-stream",
|
|
size=file_path.stat().st_size,
|
|
is_image=False,
|
|
),
|
|
)
|
|
|
|
assert ok is False
|
|
assert post_calls == []
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_downloads_inbound_file(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
plaintext = b"hello,file"
|
|
aes_key = b"1234567890abcdef"
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
|
|
return encrypted
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 303,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-file-1",
|
|
"item_list": [
|
|
{
|
|
"type": 4,
|
|
"file_item": {
|
|
"file_name": "report.pdf",
|
|
"aeskey": aes_key.hex(),
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/report.bin"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
inbound = published[0]
|
|
assert inbound.text == ""
|
|
assert len(inbound.files) == 1
|
|
file_info = inbound.files[0]
|
|
assert file_info["message_item_type"] == 4
|
|
stored = Path(file_info["path"])
|
|
assert stored.exists()
|
|
assert stored.read_bytes() == plaintext
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_downloads_inbound_file_with_media_aeskey_hex(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
plaintext = b"hello,file"
|
|
aes_key = b"1234567890abcdef"
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
|
|
return encrypted
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 304,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-file-1b",
|
|
"item_list": [
|
|
{
|
|
"type": 4,
|
|
"file_item": {
|
|
"file_name": "report.pdf",
|
|
"media": {
|
|
"full_url": "https://cdn.weixin.qq.com/report.bin",
|
|
"aeskey": aes_key.hex(),
|
|
},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
assert published[0].files[0]["filename"] == "report.pdf"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_downloads_inbound_file_with_unpadded_item_aes_key(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
plaintext = b"hello,file"
|
|
aes_key = b"1234567890abcdef"
|
|
encoded_key = base64.b64encode(aes_key).decode("utf-8").rstrip("=")
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
|
|
return encrypted
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 305,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-file-1c",
|
|
"item_list": [
|
|
{
|
|
"type": 4,
|
|
"aesKey": encoded_key,
|
|
"file_item": {
|
|
"file_name": "report.pdf",
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/report.bin"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
assert published[0].files[0]["filename"] == "report.pdf"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_downloads_inbound_file_with_media_aes_key_base64_of_hex(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
plaintext = b"hello,file"
|
|
aes_key = b"1234567890abcdef"
|
|
encoded_hex_key = base64.b64encode(aes_key.hex().encode("utf-8")).decode("utf-8")
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
|
|
return encrypted
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 306,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-file-1d",
|
|
"item_list": [
|
|
{
|
|
"type": 4,
|
|
"file_item": {
|
|
"file_name": "report.pdf",
|
|
"media": {
|
|
"full_url": "https://cdn.weixin.qq.com/report.bin",
|
|
"aes_key": encoded_hex_key,
|
|
},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert len(published) == 1
|
|
assert published[0].files[0]["filename"] == "report.pdf"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_skips_disallowed_inbound_file(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
plaintext = b"MZ"
|
|
aes_key = b"1234567890abcdef"
|
|
|
|
channel = WechatChannel(bus=bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
encrypted = channel.__class__.__dict__["_extract_file_item"].__globals__["_encrypt_aes_128_ecb"](plaintext, aes_key)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None):
|
|
return encrypted
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 404,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-file-2",
|
|
"item_list": [
|
|
{
|
|
"type": 4,
|
|
"file_item": {
|
|
"file_name": "malware.exe",
|
|
"aeskey": aes_key.hex(),
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/bad.bin"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert published == []
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_poll_loop_updates_server_timeout(monkeypatch):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
post_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
post_calls=post_calls,
|
|
post_responses=[
|
|
{
|
|
"ret": 0,
|
|
"msgs": [
|
|
{
|
|
"message_type": 1,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-1",
|
|
"item_list": [{"type": 1, "text_item": {"text": "hello"}}],
|
|
}
|
|
],
|
|
"get_updates_buf": "cursor-next",
|
|
"longpolling_timeout_ms": 42000,
|
|
}
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
channel = WechatChannel(bus=MessageBus(), config={"bot_token": "bot-token"})
|
|
channel._running = True
|
|
|
|
async def _fake_handle_update(_raw):
|
|
channel._running = False
|
|
return None
|
|
|
|
channel._handle_update = _fake_handle_update # type: ignore[method-assign]
|
|
|
|
await channel._poll_loop()
|
|
|
|
assert channel._get_updates_buf == "cursor-next"
|
|
assert channel._server_longpoll_timeout_seconds == 42.0
|
|
assert post_calls[0]["url"].endswith("/ilink/bot/getupdates")
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_poll_loop_one_bad_message_does_not_permanently_lose_its_siblings(monkeypatch, tmp_path: Path, caplog):
|
|
"""A single message that fails to process must not sink the rest of its batch.
|
|
|
|
Regression test for a permanent message-loss bug: the long-poll cursor was
|
|
persisted for the *whole* batch before the per-message loop ran, and the loop
|
|
had no per-message error isolation, so one bad message (e.g. an attachment
|
|
that fails to decrypt) aborted processing of every message after it in the
|
|
same batch. Because the cursor had already advanced past the whole batch,
|
|
the next poll would never re-fetch the unprocessed tail -- silent, permanent
|
|
loss of every message after the first failure in a batch.
|
|
|
|
This test sends a real 3-message batch through the *real* (unstubbed)
|
|
``_handle_update`` -- message 1 is fine, message 2 is a WeChat image item
|
|
whose "encrypted" bytes are deliberately not a multiple of the AES block
|
|
size, so ``_decrypt_aes_128_ecb`` raises a genuine ``cryptography`` library
|
|
``ValueError`` (matching how a real corrupt/undecryptable attachment would
|
|
fail), and message 3 is fine again. The fix must:
|
|
- still deliver message 1 and message 3 to the bus despite message 2's
|
|
failure (per-message isolation instead of one bad apple aborting the
|
|
for loop), and
|
|
- log message 2's failure instead of swallowing it silently, and
|
|
- only advance/persist the cursor once the whole batch has been
|
|
attempted (so a crash mid-batch re-delivers rather than silently
|
|
drops).
|
|
"""
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published: list[Any] = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
aes_key = b"1234567890abcdef"
|
|
non_block_aligned_ciphertext = b"\x01\x02\x03\x04\x05" # 5 bytes: not a multiple of 16
|
|
|
|
msg_good_1 = {
|
|
"message_type": 1,
|
|
"message_id": "msg-1-good",
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-1",
|
|
"item_list": [{"type": 1, "text_item": {"text": "message one"}}],
|
|
}
|
|
msg_bad_2 = {
|
|
"message_type": 1,
|
|
"message_id": "msg-2-bad",
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-2",
|
|
"item_list": [
|
|
{
|
|
"type": 2,
|
|
"image_item": {
|
|
"aeskey": aes_key.hex(),
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/corrupt-attachment.bin"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
msg_good_3 = {
|
|
"message_type": 1,
|
|
"message_id": "msg-3-good",
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-3",
|
|
"item_list": [{"type": 1, "text_item": {"text": "message three"}}],
|
|
}
|
|
|
|
state_dir = tmp_path / "wechat-state"
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
post_responses=[
|
|
{
|
|
"ret": 0,
|
|
"msgs": [msg_good_1, msg_bad_2, msg_good_3],
|
|
"get_updates_buf": "cursor-after-batch",
|
|
}
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
channel = WechatChannel(
|
|
bus=bus,
|
|
config={"bot_token": "test-token", "state_dir": str(state_dir), "polling_retry_delay": 0.001},
|
|
)
|
|
|
|
async def _fake_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None) -> bytes:
|
|
return non_block_aligned_ciphertext
|
|
|
|
channel._download_cdn_bytes = _fake_download # type: ignore[method-assign]
|
|
|
|
# _handle_update is intentionally left as the REAL implementation so
|
|
# message 2 hits the genuine cryptography ValueError. To keep the test
|
|
# deterministic without depending on the bug/fix under test, force the
|
|
# poll loop to stop after exactly one getupdates cycle via
|
|
# _ensure_authenticated (called at the top of every iteration, before
|
|
# any message is processed) rather than via message content.
|
|
real_ensure_authenticated = channel._ensure_authenticated
|
|
auth_calls = {"n": 0}
|
|
|
|
async def _ensure_auth_then_stop() -> bool:
|
|
auth_calls["n"] += 1
|
|
if auth_calls["n"] > 1:
|
|
channel._running = False
|
|
return False
|
|
return await real_ensure_authenticated()
|
|
|
|
channel._ensure_authenticated = _ensure_auth_then_stop # type: ignore[method-assign]
|
|
channel._running = True
|
|
|
|
with caplog.at_level(logging.INFO, logger="app.channels.wechat"):
|
|
await channel._poll_loop()
|
|
|
|
# Message 1 and message 3 must both survive message 2's failure.
|
|
assert [m.text for m in published] == ["message one", "message three"]
|
|
|
|
# Message 2's failure must be logged, not silently swallowed.
|
|
messages = [record.getMessage() for record in caplog.records]
|
|
assert any("msg-2-bad" in message for message in messages)
|
|
|
|
# The cursor must reflect that the whole batch was attempted -- both
|
|
# in memory and in the persisted state file used to resume polling.
|
|
assert channel._get_updates_buf == "cursor-after-batch"
|
|
persisted = json.loads((state_dir / "wechat-getupdates.json").read_text(encoding="utf-8"))
|
|
assert persisted["get_updates_buf"] == "cursor-after-batch"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_state_cursor_is_loaded_from_disk(tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
state_dir = tmp_path / "wechat-state"
|
|
state_dir.mkdir(parents=True, exist_ok=True)
|
|
(state_dir / "wechat-getupdates.json").write_text(
|
|
json.dumps({"get_updates_buf": "cursor-123"}, ensure_ascii=False),
|
|
encoding="utf-8",
|
|
)
|
|
|
|
channel = WechatChannel(
|
|
bus=MessageBus(),
|
|
config={"bot_token": "bot-token", "state_dir": str(state_dir)},
|
|
)
|
|
# State load moved out of __init__ (it does filesystem IO that would block
|
|
# the async path); mirror start() by loading explicitly here.
|
|
channel._load_state()
|
|
|
|
assert channel._get_updates_buf == "cursor-123"
|
|
|
|
|
|
def test_auth_state_is_loaded_from_disk(tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
state_dir = tmp_path / "wechat-state"
|
|
state_dir.mkdir(parents=True, exist_ok=True)
|
|
(state_dir / "wechat-auth.json").write_text(
|
|
json.dumps({"status": "confirmed", "bot_token": "saved-token", "ilink_bot_id": "bot-1"}, ensure_ascii=False),
|
|
encoding="utf-8",
|
|
)
|
|
|
|
channel = WechatChannel(
|
|
bus=MessageBus(),
|
|
config={"state_dir": str(state_dir), "qrcode_login_enabled": True},
|
|
)
|
|
# State load moved out of __init__ (it does filesystem IO that would block
|
|
# the async path); mirror start() by loading explicitly here.
|
|
channel._load_state()
|
|
|
|
assert channel._bot_token == "saved-token"
|
|
assert channel._ilink_bot_id == "bot-1"
|
|
|
|
|
|
def test_qrcode_login_binds_and_persists_auth_state(monkeypatch, tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
get_calls: list[dict[str, Any]] = []
|
|
|
|
def _client_factory(*args, **kwargs):
|
|
return _MockAsyncClient(
|
|
get_calls=get_calls,
|
|
get_responses=[
|
|
{"qrcode": "qr-123", "qrcode_img_content": "https://example.com/qr.png"},
|
|
{"status": "confirmed", "bot_token": "bound-token", "ilink_bot_id": "bot-99"},
|
|
],
|
|
**kwargs,
|
|
)
|
|
|
|
monkeypatch.setattr("app.channels.wechat.httpx.AsyncClient", _client_factory)
|
|
|
|
state_dir = tmp_path / "wechat-state"
|
|
channel = WechatChannel(
|
|
bus=MessageBus(),
|
|
config={
|
|
"state_dir": str(state_dir),
|
|
"qrcode_login_enabled": True,
|
|
"qrcode_poll_interval": 0.01,
|
|
"qrcode_poll_timeout": 1,
|
|
},
|
|
)
|
|
|
|
ok = await channel._ensure_authenticated()
|
|
|
|
assert ok is True
|
|
assert channel._bot_token == "bound-token"
|
|
assert channel._ilink_bot_id == "bot-99"
|
|
assert get_calls[0]["url"].endswith("/ilink/bot/get_bot_qrcode")
|
|
assert get_calls[1]["url"].endswith("/ilink/bot/get_qrcode_status")
|
|
|
|
auth_state = json.loads((state_dir / "wechat-auth.json").read_text(encoding="utf-8"))
|
|
assert auth_state["status"] == "confirmed"
|
|
assert auth_state["bot_token"] == "bound-token"
|
|
assert auth_state["ilink_bot_id"] == "bot-99"
|
|
if os.name == "nt":
|
|
pytest.skip(_POSIX_MODE_BITS_REASON)
|
|
assert ((state_dir / "wechat-auth.json").stat().st_mode & 0o777) == 0o600
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_save_auth_state_tightens_preexisting_loose_file(tmp_path: Path):
|
|
"""A world-readable auth file is replaced by an owner-only one, atomically.
|
|
|
|
The bot_token must never be observable at loose permissions: the atomic
|
|
0o600-temp + ``Path.replace`` path swaps in a fresh owner-only inode rather
|
|
than truncating the existing 0o644 file in place. Seeding the destination at
|
|
0o644 first means a regression back to ``write_text`` + late ``chmod`` would
|
|
leave a detectable window (and, here, the temp-file artifact behind).
|
|
"""
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
state_dir = tmp_path / "wechat-state"
|
|
state_dir.mkdir(parents=True, exist_ok=True)
|
|
auth_path = state_dir / "wechat-auth.json"
|
|
auth_path.write_text(json.dumps({"status": "pending"}), encoding="utf-8")
|
|
auth_path.chmod(0o644)
|
|
|
|
channel = WechatChannel(
|
|
bus=MessageBus(),
|
|
config={"state_dir": str(state_dir), "qrcode_login_enabled": True},
|
|
)
|
|
channel._save_auth_state(status="confirmed", bot_token="bound-token", ilink_bot_id="bot-1")
|
|
|
|
assert json.loads(auth_path.read_text(encoding="utf-8"))["bot_token"] == "bound-token"
|
|
# Atomic write leaves no temp-file residue behind.
|
|
assert list(state_dir.glob("*.tmp")) == []
|
|
# Keep the platform-independent half running on Windows; only the mode-bit
|
|
# half of the "owner-only inode" contract is unobservable there.
|
|
if os.name == "nt":
|
|
pytest.skip(_POSIX_MODE_BITS_REASON)
|
|
assert (auth_path.stat().st_mode & 0o777) == 0o600
|
|
|
|
|
|
def test_save_auth_state_chmod_failure_is_logged_not_warned(tmp_path: Path, caplog):
|
|
"""A chmod failure on a perms-less filesystem must not look like a persist failure.
|
|
|
|
With the post-replace chmod split into its own try/except, a chmod ``OSError``
|
|
is logged at debug while the JSON is genuinely on disk — operators must not see
|
|
the misleading ``failed to persist`` warning that the shared try/except produced.
|
|
"""
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
state_dir = tmp_path / "wechat-state"
|
|
channel = WechatChannel(
|
|
bus=MessageBus(),
|
|
config={"state_dir": str(state_dir), "qrcode_login_enabled": True},
|
|
)
|
|
|
|
real_chmod = Path.chmod
|
|
|
|
def chmod_spy(self: Path, mode: int, *args, **kwargs):
|
|
if self.suffix == ".json":
|
|
raise OSError("chmod unsupported on this filesystem")
|
|
return real_chmod(self, mode, *args, **kwargs)
|
|
|
|
with caplog.at_level(logging.DEBUG, logger="app.channels.wechat"), mock.patch.object(Path, "chmod", chmod_spy):
|
|
channel._save_auth_state(status="confirmed", bot_token="bound-token")
|
|
|
|
auth_path = state_dir / "wechat-auth.json"
|
|
assert json.loads(auth_path.read_text(encoding="utf-8"))["bot_token"] == "bound-token"
|
|
messages = [record.getMessage() for record in caplog.records]
|
|
assert any("unable to chmod auth state" in message for message in messages)
|
|
assert not any("failed to persist auth state" in message for message in messages)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Inbound media download cap + destination allowlist
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_is_allowed_media_url_suffix_boundaries():
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
channel = WechatChannel(MessageBus(), config={"bot_token": "test-token"})
|
|
|
|
# Platform CDN defaults (plus the configured cdn_base_url host) are allowed.
|
|
assert channel._is_allowed_media_url("https://novac2c.cdn.weixin.qq.com/c2c/x?token=1")
|
|
assert channel._is_allowed_media_url("https://cdn.weixin.qq.com/image.bin")
|
|
# Dot-boundary suffix matching: lookalike hosts never match.
|
|
assert not channel._is_allowed_media_url("https://notqq.com/image.bin")
|
|
assert not channel._is_allowed_media_url("https://qq.com.evil.io/image.bin")
|
|
assert not channel._is_allowed_media_url("https://cdn.example/image.bin")
|
|
# Loopback/private targets and non-HTTP schemes are rejected outright.
|
|
assert not channel._is_allowed_media_url("http://127.0.0.1:8001/api/user")
|
|
assert not channel._is_allowed_media_url("http://169.254.169.254/latest/meta-data")
|
|
assert not channel._is_allowed_media_url("file:///etc/passwd")
|
|
|
|
# Operator suffixes extend the allowlist; the configured cdn_base_url host
|
|
# is admitted automatically so a custom CDN endpoint keeps working. A
|
|
# leading ``*.`` (DNS habit) normalizes to the bare suffix.
|
|
custom = WechatChannel(
|
|
MessageBus(),
|
|
config={
|
|
"bot_token": "test-token",
|
|
"cdn_base_url": "https://media.internal.example/c2c",
|
|
"allowed_media_hosts": ["cdn.example", "*.wild.example"],
|
|
},
|
|
)
|
|
assert custom._is_allowed_media_url("https://media.internal.example/c2c/x")
|
|
assert custom._is_allowed_media_url("https://a.cdn.example/image.bin")
|
|
assert custom._is_allowed_media_url("https://cdn.weixin.qq.com/image.bin")
|
|
assert custom._is_allowed_media_url("https://b.wild.example/image.bin")
|
|
|
|
|
|
def test_handle_update_skips_media_from_disallowed_host(tmp_path: Path):
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
channel = WechatChannel(bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
|
|
async def _forbidden_download(_url: str, *, timeout: float | None = None, max_bytes: int | None = None) -> bytes:
|
|
raise AssertionError("download must not be attempted for a disallowed media host")
|
|
|
|
channel._download_cdn_bytes = _forbidden_download # type: ignore[method-assign]
|
|
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 201,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-evil-1",
|
|
"item_list": [
|
|
{
|
|
"type": 2,
|
|
"image_item": {
|
|
"aeskey": b"1234567890abcdef".hex(),
|
|
"media": {"full_url": "https://evil.example/image.bin"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
# The image is dropped and, with no text either, nothing is published.
|
|
assert published == []
|
|
downloads_dir = tmp_path / "downloads"
|
|
assert not downloads_dir.exists() or not list(downloads_dir.iterdir())
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_handle_update_http_download_failure_is_sanitized_not_raised(tmp_path: Path, caplog):
|
|
"""An HTTP failure during the media download drops that attachment with a
|
|
sanitized log instead of escaping to the polling loop's logger.exception.
|
|
|
|
httpx.HTTPStatusError formats the signed URL (path + query credentials)
|
|
into its message, so letting it propagate would render the URL in the
|
|
per-message traceback. Reproduced with a real 403 mock transport.
|
|
"""
|
|
import httpx
|
|
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
bus = MessageBus()
|
|
published = []
|
|
|
|
async def capture(msg):
|
|
published.append(msg)
|
|
|
|
bus.publish_inbound = capture # type: ignore[method-assign]
|
|
|
|
channel = WechatChannel(bus, config={"bot_token": "test-token", "state_dir": str(tmp_path)})
|
|
channel._client = httpx.AsyncClient( # type: ignore[assignment]
|
|
transport=httpx.MockTransport(lambda _request: httpx.Response(403))
|
|
)
|
|
try:
|
|
await channel._handle_update(
|
|
{
|
|
"message_type": 1,
|
|
"message_id": 202,
|
|
"from_user_id": "wx-user-1",
|
|
"context_token": "ctx-403-1",
|
|
"item_list": [
|
|
{
|
|
"type": 2,
|
|
"image_item": {
|
|
"aeskey": b"1234567890abcdef".hex(),
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/private/BearerSecret?token=QuerySecret"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
finally:
|
|
await channel._client.aclose()
|
|
|
|
# The image is dropped and, with no text either, nothing is published.
|
|
assert published == []
|
|
|
|
with caplog.at_level(logging.WARNING, logger="app.channels.wechat"):
|
|
_run(go())
|
|
|
|
assert "BearerSecret" not in caplog.text
|
|
assert "QuerySecret" not in caplog.text
|
|
assert "/private/" not in caplog.text
|
|
# The operator still sees what failed and for which host.
|
|
assert "HTTPStatusError (403)" in caplog.text
|
|
assert "cdn.weixin.qq.com" in caplog.text
|
|
|
|
|
|
class _FakeStreamResponse:
|
|
def __init__(self, chunks: list[bytes]):
|
|
self._chunks = chunks
|
|
self.headers: dict[str, str] = {}
|
|
|
|
def raise_for_status(self) -> None:
|
|
return None
|
|
|
|
# Deliberately no aiter_bytes: the reader must consume undecoded bytes, so
|
|
# an accidental switch back to the decoding iterator fails loudly here.
|
|
async def aiter_raw(self):
|
|
for chunk in self._chunks:
|
|
yield chunk
|
|
|
|
|
|
class _FakeStreamContext:
|
|
def __init__(self, response: _FakeStreamResponse):
|
|
self._response = response
|
|
|
|
async def __aenter__(self) -> _FakeStreamResponse:
|
|
return self._response
|
|
|
|
async def __aexit__(self, *exc_info) -> bool:
|
|
return False
|
|
|
|
|
|
class _FakeStreamingClient:
|
|
def __init__(self, chunks: list[bytes]):
|
|
self._chunks = chunks
|
|
|
|
def stream(self, _method: str, _url: str, timeout: float | None = None, **_kwargs) -> _FakeStreamContext:
|
|
return _FakeStreamContext(_FakeStreamResponse(self._chunks))
|
|
|
|
|
|
def test_download_cdn_bytes_aborts_when_stream_exceeds_cap():
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
async def go():
|
|
channel = WechatChannel(MessageBus(), config={"bot_token": "test-token"})
|
|
channel._client = _FakeStreamingClient([b"abc", b"def"]) # type: ignore[assignment]
|
|
|
|
# 6 bytes total vs a 5-byte cap: aborted mid-stream before full read.
|
|
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=5) is None
|
|
# At/under the cap and with the cap disabled the chunks are joined as before.
|
|
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=6) == b"abcdef"
|
|
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=None) == b"abcdef"
|
|
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=0) == b"abcdef"
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_download_cdn_bytes_rejects_compressed_response_before_decode(caplog):
|
|
"""aiter_bytes() would transparently decode Content-Encoding, allocating the
|
|
whole decompressed body before the cap sees a byte — an ~8 KB gzip wire
|
|
chunk decoding to 8 MiB (reproduced here) bypasses max_bytes entirely. The
|
|
download must request identity and refuse a residual encoding before
|
|
reading, even with the cap disabled."""
|
|
import gzip
|
|
|
|
import httpx
|
|
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
class _AsyncChunks(httpx.AsyncByteStream):
|
|
def __init__(self, chunks: list[bytes]):
|
|
self._chunks = chunks
|
|
|
|
async def __aiter__(self):
|
|
for chunk in self._chunks:
|
|
yield chunk
|
|
|
|
seen_requests: list[httpx.Request] = []
|
|
|
|
def handler(request: httpx.Request) -> httpx.Response:
|
|
seen_requests.append(request)
|
|
return httpx.Response(
|
|
200,
|
|
headers={"Content-Encoding": "gzip"},
|
|
stream=_AsyncChunks([gzip.compress(b"\x00" * (8 * 1024 * 1024))]),
|
|
)
|
|
|
|
async def go():
|
|
channel = WechatChannel(MessageBus(), config={"bot_token": "test-token"})
|
|
channel._client = httpx.AsyncClient(transport=httpx.MockTransport(handler)) # type: ignore[assignment]
|
|
try:
|
|
# Cap disabled: the old aiter_bytes() code path would happily
|
|
# return the 8 MiB decompressed payload here.
|
|
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=None) is None
|
|
# With a cap in place the rejection still happens for the encoding,
|
|
# before any decode/size accounting.
|
|
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x", max_bytes=1024 * 1024) is None
|
|
finally:
|
|
await channel._client.aclose()
|
|
|
|
with caplog.at_level(logging.WARNING, logger="app.channels.wechat"):
|
|
_run(go())
|
|
|
|
assert "Content-Encoding" in caplog.text
|
|
assert "exceeds" not in caplog.text # rejected for the encoding, not the size
|
|
assert seen_requests[0].headers.get("accept-encoding") == "identity"
|
|
|
|
|
|
def test_download_cdn_bytes_streams_identity_response():
|
|
"""An unencoded response still round-trips through a real httpx transport."""
|
|
import httpx
|
|
|
|
from app.channels.wechat import WechatChannel
|
|
|
|
class _AsyncChunks(httpx.AsyncByteStream):
|
|
def __init__(self, chunks: list[bytes]):
|
|
self._chunks = chunks
|
|
|
|
async def __aiter__(self):
|
|
for chunk in self._chunks:
|
|
yield chunk
|
|
|
|
def handler(_request: httpx.Request) -> httpx.Response:
|
|
return httpx.Response(200, stream=_AsyncChunks([b"abcdef"]))
|
|
|
|
async def go():
|
|
channel = WechatChannel(MessageBus(), config={"bot_token": "test-token"})
|
|
channel._client = httpx.AsyncClient(transport=httpx.MockTransport(handler)) # type: ignore[assignment]
|
|
try:
|
|
assert await channel._download_cdn_bytes("https://cdn.weixin.qq.com/x") == b"abcdef"
|
|
finally:
|
|
await channel._client.aclose()
|
|
|
|
_run(go())
|
|
|
|
|
|
def test_boundary_sized_inbound_image_survives_pkcs7_padding(tmp_path: Path):
|
|
"""A valid attachment whose plaintext is exactly the configured limit must load.
|
|
|
|
The limit bounds plaintext, but the stream measures ciphertext — AES-128-ECB
|
|
with PKCS#7 pads 32 plaintext bytes to 48 — so the stream cap must be the
|
|
padded size of exactly-limit plaintext, not the plaintext limit itself.
|
|
Exercises the real streaming download (no _download_cdn_bytes stub).
|
|
"""
|
|
from app.channels.wechat import WechatChannel, _encrypt_aes_128_ecb
|
|
|
|
async def go():
|
|
channel = WechatChannel(
|
|
MessageBus(),
|
|
config={"bot_token": "test-token", "state_dir": str(tmp_path), "max_inbound_image_bytes": 32},
|
|
)
|
|
aes_key = b"1234567890abcdef"
|
|
plaintext = b"\x89PNG\r\n\x1a\n" + b"x" * 24 # exactly 32 bytes, valid PNG magic
|
|
encrypted = _encrypt_aes_128_ecb(plaintext, aes_key) # 48 bytes > 32-byte limit
|
|
assert len(encrypted) == 48
|
|
channel._client = _FakeStreamingClient([encrypted]) # type: ignore[assignment]
|
|
|
|
assert channel._stream_cap_for(32) == 48
|
|
assert channel._stream_cap_for(0) is None
|
|
|
|
files = await channel._extract_inbound_files(
|
|
{
|
|
"message_id": 301,
|
|
"item_list": [
|
|
{
|
|
"type": 2,
|
|
"image_item": {
|
|
"aeskey": aes_key.hex(),
|
|
"media": {"full_url": "https://cdn.weixin.qq.com/image.bin"},
|
|
},
|
|
}
|
|
],
|
|
}
|
|
)
|
|
|
|
assert len(files) == 1
|
|
assert files[0]["type"] == "image"
|
|
assert files[0]["size"] == 32
|
|
assert Path(files[0]["path"]).read_bytes() == plaintext
|
|
|
|
_run(go())
|