deer-flow/backend/app/channels/AGENTS.md

IM Channels System (app/channels/)

Store JSON decode errors follow corrupt-file recovery; SQL import keeps the original bytes.

IM channels use langgraph-sdk HTTP; Gateway owns thread lifecycle. WeChat outbound: off-loop cap+1; overflow None before encryption/upload; <=0 unlimited. Cancel drains cursors; expiry drains cursor/auth reset under _auth_lock. Inbound: ingest/drop drains off-loop cleanup of owned dirs, never metadata paths.

Telegram retries share bootstrap/Bot cleanup. Dead loops warn once; clear matching futures under _shutdown_resources_lock (closure unconfirmed). Finalize asyncgens; keep primary errors; propagate lone Bot errors.

Auth: SDK/raw-httpx /goal use process-local auth + _csrf_headers(); internal auth needs CSRF.

DingTalk retains tasks and logs failures.

Shared attachment feedback uses public attachment_delivery_text, preserves routing/cancellation, and never retries or pushes proactively. Built-in channels choose EN/ZH from the first non-empty reply line; an explicit locale overrides it. WeCom keeps frame cleanup and refuses feedback after its reply context expires.

Components:

• message_bus.py - Inbound queue → dispatcher; outbound uses a dispatch-start listener snapshot. Unsubscribe neither cancels nor drains captured callbacks.
• store.py - ChannelStore protocol (async get_thread_id/set_thread_id/claim_thread_id/remove/list_entries) for channel_name:chat_id[:topic_id] → thread_id. SqlChannelStore (channel_thread_bindings, harness persistence/channel_thread_bindings/; upsert keeps created_at, topic-less remove deletes by key prefix) for sqlite/postgres, so replicas share bindings; JsonChannelStore ({base_dir}/channels/store.json, loaded once per process, file I/O via to_thread under _lock) only for database.backend: memory. resolve_channel_store picks; ChannelService.start() runs SqlChannelStore.import_legacy_json once (file + empty table → INSERT … ON CONFLICT DO NOTHING, rename to store.json.migrated, a peer's rename tolerated; populated table → file kept). Feishu's sync lark callback (on the SDK's own loop) bridges _resolve_routing_bounded to the Gateway loop via _run_store_call (tracked future; refuses only a wait from the Gateway loop itself). Store calls run under FEISHU_STORE_LOOKUP_TIMEOUT_SECONDS (3s; the SDK loop also services ws keepalives); timeout or store error = unresolved (StoreLookupUnresolved), never a miss: the message joins a per-chat FIFO drained on the Gateway loop with backoff up to FEISHU_STORE_RESOLVE_TOTAL_BUDGET_SECONDS (30s), then is dropped with a warning, never routed to a new conversation; each deferred message holds a bus intake reservation (the inbound_queue_maxsize bound) and its 30s budget runs from arrival. discord.py reads channels_dir. Tests: test_channel_thread_binding_store.py (also under blocking_io/).
• manager.py - Core dispatcher: creates threads via client.threads.create(), routes commands including /goal (setting a goal persists it through Gateway and then routes the objective as a chat turn) and /agent (list is owner-scoped; use creates a fresh thread and persists the Custom Agent selection under both the channel restart key and Web's canonical routing metadata so existing checkpoint lineage never changes agent across IM/Web), keeps Slack/Discord on client.runs.wait(), uses client.runs.stream(["messages-tuple", "values"]) for Feishu/Telegram incremental outbound updates, serializes same-thread Feishu turns in-manager when the channel's ChannelRunPolicy.serialize_thread_runs=True so rapid follow-ups queue instead of tripping the runtime busy reply, and switches to client.runs.create() (fire-and-forget, returns once the run is pending) for channels whose ChannelRunPolicy.fire_and_forget=True so long autonomous runs do not hit the SDK default 300s httpx.ReadTimeout; runs.wait/runs.stream pass on_disconnect: continue (a run outlives its requester; reply lost, thread kept), so once a request is sent, cancellation or a transport error keeps the inbound dedupe key (_keep_dedupe_key_after); only Gateway rejections, pre-request failures and error frames release it. Keyed messages send Idempotency-Key (from the dedupe key) and a uuid5 human message id, so a late redelivery gets the admitted run: a status-only /wait or a stream gap resolves this message's turn via threads.get_state; a 409 on its own run (_ADMITTED_RUN_CONFLICT_DETAILS) replies ALREADY_ADMITTED_MESSAGE and keeps the key. First bindings use claim_thread_id (insert-if-absent): a loser adopts the winner and deletes its minted thread (never a preferred id); /new rebinds. Discord keys (discord, guild|dm, channel_id, message_id) and claims via MessageBus.claim_inbound before _create_thread; dedupe_claimed skips the worker's record; uncommitted (incl. cancelled) or shutdown-discarded claims are released Streaming error frames use the same failure path as transport exceptions. Publish the final outbound before releasing the inbound dedupe key, so provider redelivery can retry without overtaking the terminal reply. Log the frame's error name and message at WARNING with the thread ID; keep the synthetic error generic so payload details do not change reply selection. Stream publishing is an allowlist (_accumulate_stream_text): only assistant types ("ai", "AIMessageChunk", OpenAI-style "assistant") become text, matched by prefix, never substring (chain, domain contain "ai"). The old "tool" denylist leaked hidden model context: DynamicContextMiddleware (recalled memory, the rewritten user turn) and DurableContextMiddleware inject hidden HumanMessages that fan out on messages-tuple. _stream_payload_type reads the model_dump() shape and LangChain's to_json() constructor shape (top-level type "constructor", class name at the tail of id). Bare str payloads are rejected: untyped, and serialize_messages_tuple always emits [message_dict, metadata].
• base.py - Abstract Channel base class (start/stop/send lifecycle). Provider callbacks that submit coroutines from SDK threads must use _submit_threadsafe_coroutine() (or _submit_threadsafe_coroutine_future() to await the outcome through asyncio.shield): it creates and retains the real asyncio.Task on the owner loop instead of treating run_coroutine_threadsafe()'s proxy Future as a completion signal. Submission is closed atomically with shutdown, and stop() must call _close_and_drain_threadsafe_futures() before tearing down SDK resources. Blocking SDK teardown in stop() (a client close(), a thread join()) must run off the event loop through asyncio.to_thread: the shared Gateway loop also serves POST /api/channels/{name}/restart; Slack cleanup awaits connect then closes off-loop; a 5s deadline/cancellation retains it for retry. SDK close errors propagate. Anchored by tests/blocking_io/test_channel_stop_offloop.py. Proactive push: WeCom, Feishu, WeChat and QQ C2C share the outbox; capabilities.py gates enqueue/UI. Unsupported rows fail terminally. Await ACK; ChannelUnavailable parks, business rejection counts. WeCom parks socket RuntimeError except Reply ack error. Feishu/QQ/WeCom group hooks use owner connection IDs; Feishu p2p resolves owner. group_name: OAuth state only, no suffix/raw-ID labels. WeCom legacy-group fallback excludes private bindings. QQ: owner/current-App C2C only, never member OpenIDs/passive context. WeChat: encrypted owner/bot/private context across restart; ignore public tokens. store_connection_context locks/checks the active identity before merging encrypted extra; browser responses strip legacy tokens. Preferences gate enqueue/send/retry, cancel pending/sending and fence late writes; reenable never replays. In-flight sends may finish. Binding lookup errors park; revoked targets fail before liveness. _on_finalization commits one notice with the outcome (stop > pause > finish > run); recovery dedupes, idle uses idle:, trials/interrupts stay silent. Web/background events default off: confirm durable terminal state; recheck Human Input before send/retry. Text: localized, no IDs/links. Delivery starts after channels/stops first; failure detaches enqueue. IM guide.
• wecom.py - WeCom WebSocket channel with proactive send_notification; _ws_transport_up maps disconnects to ChannelUnavailable (no retry used). capabilities.py - SDK-free CHANNEL_CAPABILITIES.
• service.py - Drain manager work before transports; retain timed-out instances for retry. Feishu/DingTalk stop() stop the SDK thread's private loop via _request_*_shutdown (Feishu first disarms lark's _auto_reconnect; DingTalk runs its own SDK connect-loop copy), then join; a live join or Slack cleanup raises ChannelStopTimeout: retain, defer replacement/restart, log WARNING without traceback; wedged instances report not running. Reload returns snapshots; restart owns cached fallback/publication under the channel lock. _stopping fences starts/publication (including after pre-start I/O) until shutdown succeeds. Lock manager startup through stale cleanup; capture the shutdown generation before locking to fence queued starts. stop() stays unlocked to fence in-flight startup. Leader lease (leadership.py): only the channel_leases holder starts a non-GitHub transport (_start_channel checks holds(); callers acquire first); _supervise renews at a fixed rate, runs starts/stops per provider, stops a transport before its lease lapses, re-applies runtime config whose version changed; reloads read a disconnect as its tombstone.
• slack.py / feishu.py / telegram.py / discord.py / dingtalk.py - Platform-specific implementations (feishu.py tracks the running card message_id in memory and patches the same card in place; telegram.py accepts inbound text/photos/documents, preserves media captions, hands token-free attachment bytes to the shared upload pipeline, edits the "Working on it..." stream target in place via editMessageText, and can optionally send final Markdown replies as Rich Messages through channels.telegram.rich_messages — the Rich path only fires when rich_messages is on and the text contains a rich construct (fenced code, emphasis, a table, a task list, <details>, block math, or a link), so structured command/error replies stay plain text and their newlines and <placeholder> tokens survive; discord.py runs _on_message on discord.py's private loop, so connection-repository work (identity lookup with the reservation commit, /connect binding) goes to the Gateway loop through _submit_threadsafe_coroutine — the SQLAlchemy engine is bound there — and bind replies return through _run_on_discord_loop; typing registers before that hand-off settles, and a failed hand-off skips the ack and stops the indicator it started unless another message still relies on it; discord.py registers typing loops before inbound handling yields and _start_typing() refuses work once _running is false; each tick sends the indicator with discord.py 2.x's await channel.typing(); because stop() runs on the main loop while typing tasks belong to _discord_loop, normal cross-thread cancellation, awaiting, and map cleanup are scheduled there with a bounded wait, while _run_client() drains tasks in its finally block before an exception/disconnect can make that loop unusable; an already-stopped foreign loop must never have its tasks awaited from the main loop; to that end every outbound cross-loop call (send / send_file / _get_channel_or_thread) goes through _run_on_discord_loop, which bounds the await (DISCORD_OUTBOUND_TIMEOUT_SECONDS 30s; DISCORD_UPLOAD_TIMEOUT_SECONDS 120s for file uploads: slow uplink plus 429 retry-after) and fails fast with a RuntimeError when the client loop is missing or not running — a dead client becomes a logged send failure instead of a permanently hung ChannelManager worker — and is_running reports client-thread aliveness (like feishu.py) so ensure_channel_ready can restart the channel after _run_client() exits on a fatal error; dingtalk.py: card_template_id enables in-place AI Card streams; receive_file downloads picture/richText images and file documents via downloadCode to thread uploads, like Feishu. Invalid expireIn or <=300s uses 7200s before caching)
• buzz.py - Buzz (Nostr relay) implementation: one NIP-42-authenticated websocket, pubkey-allowlist + mention/DM/thread-follow gating (thread-follow reads via the manager's connection_identity.lookup_thread_id, so bound authors' repo-only threads count), streaming replies via in-place kind-40003 edits; requires the buzz dependency extra. Its durable seen-event replay guard uses async aseen() / arecord() / aflush() boundaries: initial JSON reads and coalesced atomic writes run off the Gateway event loop, and stop() awaits the final flush before returning. Subscription model (operator-facing version in IM_CHANNEL_CONNECTIONS.md): the relay fans kind-9 chat events out only to channel-scoped subscriptions, proved on a live relay: REQ {"kinds":[9]} is accepted and answered with EOSE but never receives an event, REQ {"kinds":[9],"#h":[uuid]} works, and a multi-value #h matches nothing, so it is strictly one REQ per channel. Every connection therefore rebuilds three kinds of subscription after NIP-42 auth: buzz-discovery ({"kinds":[39000]}, a historical query returning exactly the channels this identity belongs to, one stored event each then EOSE — adding #p returns zero, do not "narrow" it), buzz-membership ({"kinds":[44100,44101],"#p":[us],"since":<connect time − MEMBERSHIP_LOOKBACK_SECONDS>}, the relay-signed member-added/member-removed notifications whose p tag names the affected member and h tag the channel), and one buzz-chat-<uuid> per discovered channel. Chat subscriptions open as each kind-39000 arrives; the discovery EOSE is the completeness barrier that retries any that failed and warns when discovery found nothing. A kind-44100 for our pubkey subscribes to the new channel live (then re-issues discovery so its name/type reach the DM-detection cache, but only when that channel's metadata is actually missing — an unconditional refresh is how a burst of 44100s multiplied into one discovery pass each); a kind-44101 issues buzz_nostr.close_frame for exactly that channel's subscription and drops its metadata. The membership subscription is scoped to LIVE events and this is load-bearing: buzz-relay stores 44100/44101 and serves history newest-first (default limit 2000), so an unscoped filter replayed the whole membership history on every connect — every stored add read as live, re-running discovery once each (M+1 discovery passes × N stored kind-39000 events per connect, observed live as two channel discovery complete lines and one channel logged <unnamed> because the 44100 path subscribed before its metadata arrived), re-subscribing channels we have since been removed from, and letting a stored 44101 transiently unsubscribe a channel we are still in. since is anchored at the moment the socket opened (_session_started_at) minus MEMBERSHIP_LOOKBACK_SECONDS (60) of slack, which covers both relay clock skew and a membership change published during the connect/auth handshake; the slack can only cost an idempotent replay of the last minute. A relay CLOSED frame is recovered — every subscription on the socket fails silently when dropped, so _handle_closed re-issues it, bounded by MAX_RESUBSCRIBE_ATTEMPTS (3) per subscription id per connection and per auth epoch (_resubscribe_attempts is reset on session start, on re-auth, and by stop(), so pre-auth rejections — which the auth branch already recovers wholesale — never spend the authenticated session's budget). The first retry is immediate (a one-off hiccup is the common case); later ones back off 1s then 2s, awaited inline in the read loop, not in a background task that could outlive its socket. auth-required: before this socket has completed its NIP-42 handshake is the expected bootstrap sequence, not a refusal, and _handle_closed short-circuits it ahead of every recovery path: the connector opens its control REQs immediately in case the relay serves unauthenticated reads, a closed relay answers auth-required: plus an AUTH challenge, and the auth branch re-opens everything. That case is logged at DEBUG and consumes neither the permanent-refusal branch nor the retry budget (a chat subscription is still dropped from _chat_subscriptions, since it genuinely is not subscribed and discovery is what re-opens it). Treating it as permanent produced a false operator-facing warning that discovery/membership tracking was DOWN while discovery then completed normally. The boundary is the per-socket _auth_completed flag, set once the signed AUTH event has been sent and cleared on session entry, session exit, and stop(); the same reason after that stays loud, and says the subscription is down until the relay's next AUTH challenge or the next reconnect rather than borrowing the non-auth wording. Then _is_transient_close decides whether to retry at all: NIP-01/NIP-42 auth-required:/restricted:/blocked:/mute:/invalid:/pow: prefixes and buzz-relay's own removal/revocation prose are permanent (do not fight the relay over a channel that is no longer ours), rate-limited:/error:/no-reason-at-all and anything unrecognized are transient — the default resolves toward "keep listening" because going silently deaf is the failure this exists to remove, and the attempt budget bounds a wrong guess. A chat CLOSED is only ever recovered for a channel already in _chat_subscriptions: a CLOSED is relay-supplied, so acting on an unknown one would let a relay induce a subscription just by naming a channel. Every subscription that goes unlistened is logged at WARNING, never INFO. Subscription ids are deterministic per channel so one can be replaced or closed without disturbing the others, and _chat_subscriptions is per-socket state cleared on session end, on re-auth (a pre-auth REQ may have been rejected), and by stop(). Three bounds on remote-fed state: MAX_CACHED_CHANNELS (512) caps the kind-39000 metadata cache, the watermark map, and the resubscribe-attempt map; MAX_CHANNEL_SUBSCRIPTIONS (256, well under buzz-relay's own 1024-per-connection ceiling) caps live chat subscriptions — at the cap new channels are refused and named in a warning rather than evicting a working subscription. Backlog paging: freeze the read floor and track _backlog_until; stop/warn on stalled timestamps, never skip a tied second. Removal/live close clears paging and closes backlog; ignore stale frames. Reset cursors per socket. Bounds: subscription model. Trust model (operator-facing version in IM_CHANNEL_CONNECTIONS.md): every inbound EVENT is authenticated at the single handle_relay_frame choke point — the NIP-01 id is recomputed from the delivered payload and the BIP-340 Schnorr signature verified against the claimed pubkey (buzz_nostr.verify_event, pure and total: malformed input returns False, never raises) — so ev["pubkey"], the authorization principal for both the allowlist and the /connect bind, cannot be forged by a relay the DeerFlow operator does not run. What remains trusted is the authorship of kind-39000 channel metadata: any member can sign one, and because per-channel subscriptions are now driven by discovery, a forged kind-39000 has two effects rather than one — it can mark a channel type: "dm" (relaxing require_mention for that channel) and it can induce a chat subscription for a channel of the forger's choosing, since the channels we listen to are exactly the channels we hold metadata for. Neither makes anything be acted on: allowed_users and per-event signature verification are independent gates, so an induced subscription only means the relay reads its own traffic back to a subscriber that drops it, bounded by MAX_CHANNEL_SUBSCRIPTIONS (which refuses rather than evicts, so it cannot displace a real channel). Same for a forged kind-44100, except its p tag is re-checked locally so it must at least name us. Closing this needs a configured trusted relay pubkey, which relay_url is not. allowed_users is deny-by-default (empty = nobody, unlike siblings' empty = everyone), so start() logs a WARNING when it is empty and each drop logs at DEBUG. The resubscribe cursor (since) is per channel, advances only for events that were actually processed, and never past now + MAX_FUTURE_SKEW_SECONDS, because it is peer-supplied (created_at) and a single future-dated event otherwise made the connector permanently deaf. Per channel rather than global is the safety-critical half: one shared cursor is the newest event seen in any channel, so a busy channel would drag it past a quiet channel's unread messages, skipping them on the next reconnect. Per-channel cursors can only ever cost duplicate delivery (absorbed by the manager's event_id dedupe), and an evicted cursor degrades to "no since", i.e. the relay's default backlog — both fail toward replay, never toward a miss. Streaming tracks every oversize chunk index (_stream_targets for chunk 0, _stream_tails for the rest), since the manager republishes cumulative text and reposting chunks[1:] per update flooded the channel; all of it is per-connection state cleared by stop(), and the remote-fed kind-39000 cache is capped at MAX_CACHED_CHANNELS. send() refuses outright to publish text carrying a hidden model-context wrapper (<memory>, <durable_context_data>, <system-reminder> — _HIDDEN_CONTEXT_MARKERS), logging at ERROR and clearing the stream bookkeeping on a blocked is_final. This is defense in depth behind the manager's allowlist, and it lives here rather than in a sibling connector because on Buzz a leak is permanent: every streaming update is an immutable public Nostr event, so a corrective edit only changes what clients render while the original leaked event stays on the relay. Matching is on the literal opening tag, so a reply that merely talks about memory is still published Buzz seen-event shutdown stays bounded and retryable: aflush() awaits any in-flight write, attempts at most one final snapshot, leaves a still-changing generation dirty for fail-open replay, and returns without a live persistence timer. A Gateway cancellation does not cancel the underlying worker-thread write; BuzzChannel.stop() tracks cleanup completion separately from transport admission so ChannelService can retry the retained channel without racing a second snapshot against that write. The store is quiesced before stop (including the already-stopped guard), so a timed-out relay task that records after stop only marks data dirty and cannot schedule detached file work; a repeated stop() still drains that dirty state, while BuzzChannel.start() explicitly resumes scheduling and flushes it automatically.
• github.py - Webhook-driven GitHub channel. Inbound messages come from POST /api/webhooks/github; outbound is log-only because GitHub agents post explicitly with gh from their sandbox when they choose to comment or create a PR
• app/gateway/routers/channel_connections.py - Browser-facing user connection and disconnect APIs
• deerflow.persistence.channel_connections - SQL-backed user-owned connection, optional credential, connect state, and conversation store

Message Flow:

1. External platform -> Channel impl -> MessageBus.publish_inbound()
   • For GitHub, the webhook router verifies the delivery then calls fanout_event(bus, ...); matching agent bindings publish one InboundMessage each instead of a long-polling channel worker.
   • Telegram photo/document updates use the largest photo size or document metadata, preserve message.caption, enforce the hosted Bot API's 20,000,000-byte download ceiling before and after download, and never expose the token-bearing Bot API file URL. Downloaded bytes cross the adapter/manager boundary only through message_bus.INBOUND_FILE_CONTENT_KEY; the manager consumes that transient field before persisting safe upload metadata.
   • Feishu/Lark inbound image/file downloads read at most 20,000,001 bytes and reject anything above 20,000,000 bytes before persistence or sandbox sync. Oversize, unsafe-path, and path-resolution failures rewrite only that attachment placeholder to Failed to obtain the [type] so later attachments in the same message can still load.
   • WeChat iLink inbound image/file downloads stream with an in-flight cap derived from channels.wechat.max_inbound_image_bytes / max_inbound_file_bytes (20 MB / 50 MB defaults) that aborts before the full read, and the payload-supplied media.full_url must pass a scheme + dot-boundary host-suffix allowlist (channels.wechat.allowed_media_hosts, defaulting to the qq.com family plus the configured cdn_base_url host) before anything is fetched. The limits are PLAINTEXT limits while the stream measures CIPHERTEXT, so the in-flight cap is the PKCS#7-padded size of exactly-limit plaintext (WechatChannel._stream_cap_for); the exact post-decryption check stays as the authority. There is deliberately no URL re-fetch fallback: _read_wechat_inbound_file (manager) only reads the locally staged path — the channel always stages one before publishing — so nothing fetches a WeChat URL outside the channel-side gate. WeCom media URLs (frame-supplied url, no per-channel size config) get the same style of host gate — qq.com suffixes plus the COS shape ww-aibot-img-<APPID>.cos.<region>.myqcloud.com WeCom serves signed links from, where the numeric suffix must be one of the verified WeCom-owned Tencent Cloud APPIDs (_WECOM_MEDIA_COS_APPIDS, currently only the 1258476243 observed in Tencent's published callback examples) because bucket names are user-chosen and any Tencent Cloud account can register a lookalike ww-aibot-img-* bucket; other accounts go through operator suffixes from channels.wecom.allowed_media_hosts resolved from the live channel at read time — and the manager-level MAX_INBOUND_URL_FILE_BYTES (50 MB) streaming cap. Both URL downloads and _download_cdn_bytes request Accept-Encoding: identity, refuse a response with any residual Content-Encoding before reading, and iterate aiter_raw(): httpx's transparent decoding would allocate the full decompressed body before the byte cap sees a chunk. No URL-based inbound-media log line may contain any part of the URL (signed links carry credentials in path and query): failure labels use the attachment filename or host only (manager._inbound_file_label), and reader failures log a sanitized exception summary — class name plus HTTP status (manager._reader_error_summary, wechat._media_download_error_summary), never the URL-bearing message or traceback. The rule extends to the HTTP client libraries' own records: httpx emits HTTP Request: GET <full URL> at INFO via the bare httpx logger, and urllib3 — enumerated against the installed source; every remaining emitter logs host:port only (connection establishment/reset) or one absolute URL (the header-parse warning) that the generic pattern rewrites — renders a request target in five shapes, each needing a dedicated redaction pattern because the generic absolute-URL pass cannot see a target with no scheme: Redirecting <target> -> <target> at INFO via urllib3.poolmanager and DEBUG via urllib3.connectionpool (either slot may be an origin-form relative Location, which pool/connection callers pass or the Location header carries; the shape keeps the ^Redirecting prefix anchor because -> /path arrows also appear in non-URL logs — sandbox mount mappings — and must pass through untouched, while both slots consume through end-of-line: the raw Location header may carry interior spaces, and the recursive urlopen frame can pass it on as the next request target. A slot is kept verbatim only when the generic pass consumes it whole (logging_config._url_pass_consumes_slot asks that pass itself, so the kept set cannot drift from its host/rest stop conditions) — every other RFC 3986 relative-reference form (slash-less paths, query-only, fragments, network-path), non-hierarchical scheme, space-carrying slot, quote-at-boundary slot (https://h/a')b?sig=…, where rest stops at the closing mark), and empty-host absolute (https:///path?sig=…, which host never matches) collapses, so neither download?sign=… nor other page?sig=… leaks its signed query), the per-request scheme://host:port "METHOD /path?query HTTP/x.x" DEBUG line, Retry: <target> DEBUG and Retrying (…) after connection broken by '<error>': <target> WARNING — above the INFO root, so reachable at the default log level — via urllib3.connectionpool, and Incremented Retry for (url='<target>') DEBUG via urllib3.util.retry (origin-form on the request path, absolute on the redirect path). The generic pass's rest treats a quote as a closing mark only at a boundary — followed by whitespace, a closing parenthesis, or end of string — so urllib3's Incremented Retry for (url='…') keeps its ') scaffolding while a quote embedded in a URL stays consumed and redacted. Logger filters only see records emitted through that exact logger — child loggers neither inherit them nor trigger them on propagation — so UrlRedactionFilter is attached to the httpx logger AND to every root handler by configure_logging (handler-level filters see propagated records), which also keeps Telegram's token-bearing Bot API paths (api.telegram.org/bot<token>/..., emitted by python-telegram-bot's httpx client) out of the logs.
2. ChannelManager._dispatch_loop() consumes from queue
3. For user-owned channel connections, incoming messages carry connection_id, owner_user_id, and workspace_id; owner_user_id becomes the DeerFlow run user_id, while the raw platform user id remains channel_user_id. The Gateway accepts channel_user_id only from an internally authenticated channel caller's top-level body.context, clears it from both free-form body.config sections, and writes it into runtime context only (never configurable, which is checkpointed). bash_tool exposes it to sandbox commands as the fixed env var DEERFLOW_CHANNEL_USER_ID — via a shell-quoted command-string prefix, NOT the execute_command(env=...) channel, which is reserved for request-scoped secrets and would switch AioSandbox onto the bash.exec path (image >= 1.9.3, fresh session per call). Per-call injection keeps group-chat identity correct (one thread/sandbox, many senders) without depending on the AIO shell's session semantics: every IM-channel command carries an explicit export VAR=<id>; (valid id) or unset VAR; (empty / non-str / over the 256-char cap). The AIO no-env path reuses a persistent shell session (the reason for the class lock, #1433), so a bare command could otherwise resolve a stale id an earlier sender exported; the unset closes the window the length/type guard would open (a dropped id would inherit the previous sender's value). Non-IM runs (no channel_user_id in context) are left untouched. A second, always-present prefix carries the authenticated DeerFlow user id (#3919) as the fixed env var DEERFLOW_USER_ID, emitted outermost of the two (export DEERFLOW_USER_ID=<id>; export DEERFLOW_CHANNEL_USER_ID=<id>; cd ...), so a reader sees which user the command runs as before which IM sender it came from; unlike the channel id it is never skipped for a non-IM run, because resolve_runtime_user_id falls back to default (same export/unset discipline and 256-char cap). Neither prefix is emitted on the Windows local sandbox (its PowerShell/cmd.exe fallback has no export/unset): there the user id is published through execute_command(env=...) instead — LocalSandbox layers env into the per-process environment and, unlike AioSandbox, has no persistent session to keep honest — while the channel id is simply absent. Propagates across task delegation: task_tool captures the dispatching turn's id and the subagent executor forwards it into the subagent's runtime context, same as the guardrail attribution fields. The runtime-context value is authorization-grade at the Gateway/guardrail boundary, but both exported shell variables (DEERFLOW_CHANNEL_USER_ID, DEERFLOW_USER_ID) remain informational because any bash command can overwrite its own environment, and anything the command sources or launches (a dependency, a sourced rc file on the host-bash path) can silently re-export a different id before a skill reads it — only the outer prefix on the next bash_tool call re-asserts the true value; skills must not treat the shell variable itself as authenticated identity. Tests: tests/test_gateway_services.py, tests/test_channel_user_id_env.py
4. For chat: look up/create thread through Gateway's LangGraph-compatible API
   • First-use creation is serialized by a waiter-aware keyed lock over (channel_name, chat_id, topic_id). Holders and queued callers share one lock generation through failure and cancellation; idle entries are reclaimed only after the final participant leaves. This prevents a late inbound message from bypassing a queued creator and splitting one conversation across duplicate Gateway threads.
5. Feishu/Telegram chat: runs.stream() → accumulate AI text → publish multiple outbound updates (is_final=False) → publish final outbound (is_final=True)
6. Slack/Discord chat: runs.wait() → extract final response → publish outbound 6b. GitHub chat (ChannelRunPolicy.fire_and_forget=True): runs.create() returns once the run is pending; the manager does not wait for the final state and does not publish an outbound; the agent posts its own reply via sandbox gh. ConflictError on a busy thread still trips the standard THREAD_BUSY_MESSAGE path (log-only on GitHub); with buffer_followups_on_busy=True (GitHub's default) the triggering message is captured into a per-thread buffer so a concurrent comment is not silently dropped (see "Follow-up buffering while busy").
7. Feishu patches one card per source (config.update_multi=true), with topic previews and queued → running → final states. Opt-in cards show context/question; bounded tokens preserve request ID/source. Originator/option checks consume once; consumption fails closed across workers (feishu_card_interaction.py). Card replies keep bound aliases in fields/metadata; ordinary replies preserve /new. Accepted intake retires only that card's buttons/text hint before terminal PATCH; rejected intake keeps text fallback.
8. Telegram edits the "Working on it..." stream target: 1s private/3s group throttle (20 msg/min), 4096-char truncation, rate-limited updates dropped; final edits split overflow. Text/preview/photo/document reply sends use allow_sending_without_reply only for chat_id > 0: missing private targets degrade atomically; group/forum routing and transport retries stay unchanged.
9. DingTalk AI Cards (card_template_id): runs.stream() → create card with initial text → PUT /v1.0/card/streaming → finalize (is_final=True). Creation/streaming failure uses sampleMarkdown; tables preserve escaped pipes and empty edge cells.
10. For commands (/new, /status, /models, /model, /memory, /goal, /agent, /help): handle locally or query Gateway API. /agent list reads the effective owner's Custom Agents; /agent use <name> validates in the same owner bucket, creates a new thread, and persists channel_agent_name in its metadata. A Custom Agent writes canonical metadata.agent_name so thread-search routes Web continuation to /workspace/agents/<name>/chats/<thread_id>; lead_agent omits it and stays on the ordinary chat route. /model <name> persists channel_model_name in the current thread's metadata (in place, no fresh thread); /model reports the effective model, /model default clears it (stale pins self-clear). The channel_agent_name cache reloads after restart before a resumed turn routes. An explicit selection also normalizes agent_name across the top-level run context and both RunnableConfig carriers (context, configurable) — cleared for lead_agent — before Gateway's setdefault merge, so stale channel defaults cannot silently win.
11. Outbound → channel callbacks → platform reply
    • GitHub is the exception: the channel logs the final assistant message and does not auto-post it to GitHub. Agents use the sandbox gh CLI (gh issue comment, gh pr comment, gh pr create, etc.) for intentional writeback, so silence is cheap when several agents fan out on the same event.

Owner-scoped file storage: inbound files, uploads, and output artifacts are staged under the DeerFlow owner's bucket so they land where the agent run reads/writes (users/{user_id}/threads/{thread_id}/user-data/{uploads,outputs}). ChannelManager._handle_chat resolves the storage owner once via _channel_storage_user_id(msg) (sanitized owner id, falling back to safe(msg.user_id) for unbound auth-enabled channels — mirroring _resolve_run_params's run identity; None only when no identity is available) and threads it as the user_id= kwarg through the file pipeline:

• Channel.receive_file(msg, thread_id, user_id=...) — owner-bound channels persist downloaded files under the owner's bucket instead of the default bucket
• FeishuChannel._receive_single_file(...) / DingTalkChannel._receive_single_file(...) — normalize filenames and claim/write through write_upload_file_no_symlink(..., exclusive=True), retrying FileExistsError under the channel lock; the final basename drives agent-visible paths and sandbox sync
• sandbox_files.py — non-mounted Feishu/DingTalk syncs acquire unique execution holders, drain blocking update_file workers across repeated cancellation, and release only after the last sandbox operation, so a parallel run cannot close the shared client mid-upload. Both channels request provider release after the final holder exits, parking standalone temporary sandboxes.
• _ingest_inbound_files(...) and the underlying ensure_uploads_dir / get_uploads_dir — owner-scoped via the same kwarg
• _resolve_attachments / _prepare_artifact_delivery — resolve output artifacts from the bound owner's bucket through app.gateway.path_utils.resolve_outputs_confined_path, the same outputs-only rule the artifact editor uses, so a sibling uploads//workspace/ path or a symlink planted in outputs/ is skipped with a warning The cached value is reused for both the blocking (runs.wait) and streaming (_handle_streaming_chat) paths, so uploads and artifact delivery always target the same bucket even if a channel returns a rewritten InboundMessage from receive_file. The bucket id matches the memory bucket resolved by _resolve_memory_user_id (both normalize through make_safe_user_id).

Configuration (config.yaml -> channels):

• langgraph_url - LangGraph-compatible Gateway API base URL (default: http://localhost:8001/api)
• gateway_url - Gateway API URL for auxiliary commands (default: http://localhost:8001)
• In Docker Compose, IM channels run inside the gateway container, so localhost points back to that container. Use http://gateway:8001/api for langgraph_url and http://gateway:8001 for gateway_url, or set DEER_FLOW_CHANNELS_LANGGRAPH_URL / DEER_FLOW_CHANNELS_GATEWAY_URL.
• Per-channel configs: feishu (app_id, app_secret), slack (bot_token, app_token), telegram (bot_token, optional rich_messages for final Markdown Rich Messages; allowed_users takes numeric IDs or one ID, warns on unparseable entries like @names, and denies everyone if none parse), dingtalk (client_id, client_secret, optional card_template_id for AI Card streaming), github (operator kill-switch enabled, plus default_mention_login for mention-required GitHub triggers), buzz (relay_url, private_key)

User-owned channel connections (config.yaml -> channel_connections):

• Disabled by default. It is a user-binding layer on top of the existing channels.* runtime config, not a replacement for provider bot credentials.
• No public IP, OAuth callback URL, or provider webhook route is required by the current implementation.
• Telegram binds via deep-link /start <code> over long polling; Slack, Discord, Feishu/Lark, DingTalk, WeChat, and WeCom use /connect <code> through their outbound workers.
• WeChat polling_timeout, polling_retry_delay, qrcode_poll_interval, and qrcode_poll_timeout require positive finite seconds; invalid values use defaults to prevent hot loops or endless sleeps.
• allowed_users.py: QQ/WeChat/Telegram/Slack/DingTalk empty allows all; invalid warns; all-invalid denies. WeChat: int IDs, gate downloads. Buzz/Discord parse_channel_ids: trim; no splitting/mapping keys; empty = no exemption. Separator warnings: Buzz/QQ/WeChat/Slack/DingTalk.
• WeCom serializes start() and stop() for each channel instance. The SDK connect() task covers connection setup only; after the handshake, the SDK owns a separate receive task. Shutdown cancels an in-progress connection attempt and awaits the SDK's actual asynchronous receive-task/socket cleanup before releasing lifecycle state or allowing a restart. Cancellation of stop() still propagates, but only after owned cleanup finishes and lifecycle references are cleared; real connection failures remain reported by _on_ws_task_done.
• WeCom outbound content is capped at the protocol's 20480 UTF-8 bytes: stream replies clip on a character boundary with a truncation marker (one stream carries the whole reply and cannot split mid-way), while proactive pushes split into at most 10 sequential markdown messages per push, with the remaining tail clipped and marker-terminated. Both paths measure bytes, not characters. A per-chat send lock serializes each split batch end to end, because manager workers run concurrently and two long pushes to the same chat would otherwise interleave chunks; locks are reference-counted and reclaimed once no sender holds or waits on them, so the registry does not grow for the life of the Gateway.
• Frontend APIs: GET /api/channels/providers, GET /api/channels/connections, POST /api/channels/{provider}/connect, and DELETE /api/channels/connections/{connection_id}.
• Browser APIs remain protected by normal Gateway auth/CSRF. Provider messages arrive through the already-configured channel workers.
• Runtime credentials (runtime_config_store.py): encrypted channel_runtime_configs rows (JSON only for memory), re-read per request, never cached on app.state; a save is one cancellation-owned commit.
• The JSON runtime-config sidecar uses utils.file_lock.retry_until_locked on Windows: retry only EACCES/EAGAIN/EDEADLK every 0.2s without a deadline; propagate other errors before entering the mutation or unlocking. Warn once per acquisition after 50 contention retries with the sidecar path; logging failures must not interrupt the wait. POSIX keeps blocking flock. Coverage: tests/test_channel_runtime_config_store_lock.py.
• Provider-level connection_status reflects the user's newest connection row. With no binding it is not_connected, except in auth-disabled local mode where a configured running channel reports connected because all channel messages already route to the default user.
• Slack replies use the configured operator bot token from channels.slack unless per-connection credentials are present; unreadable or corrupt stored credentials are treated as unavailable.
• Telegram, Slack, Discord, Feishu/Lark, DingTalk, WeChat, and WeCom workers resolve incoming platform identities to connection records before reaching ChannelManager.
• Connect-code ordering: /connect <code> (Telegram /start <code>) precedes allowed_users, so even non-allowlisted senders can bind with a valid code. The allowlist gates ordinary messages. Codes rely on secrecy: secrets.token_urlsafe(16), 600 s TTL, one-time consume_oauth_state, shown only in the initiating browser, never echoed to chat.
• Single-active-owner transfer semantics: an external identity is keyed by (provider, external_account_id, workspace_id). The latest successful bind wins — upsert_connection revokes other owners' active rows for the same identity (ownership transfer). This invariant is enforced at the DB layer by the partial unique index uq_channel_connection_active_identity (WHERE status != 'revoked'), so concurrent connects from different owners cannot both end connected; the losing writer retries against the now-visible state.
• See backend/docs/IM_CHANNEL_CONNECTIONS.md for provider setup, operational notes, and the architecture diagrams (connect-code flow, single-active-owner transfer, sync vs streaming dispatch, owner-scoped file storage pipeline).

GitHub event-driven agents (webhook-driven IM channel):

• Custom agents declare a github: block in their config.yaml to bind to repos and event triggers; the webhook route is fail-closed by default (mounted only when GITHUB_WEBHOOK_SECRET is set) and exempt from auth/CSRF because authenticity is enforced by HMAC.
• Registry caching is keyed by the configured agent store's opaque signature: file storage uses agent config mtimes, while database storage hashes the ordered owner/name/config/soul contents so same-timestamp writes still invalidate webhook routing.
• Outbound is log-only by design: each agent posts its own reply mid-run via the gh CLI from its sandbox, so the manager uses fire_and_forget=True and runs.create() returns once pending.
• Busy follow-ups (#4121): ChannelRunPolicy.buffer_followups_on_busy=True (GitHub default) buffers runs.create() conflicts per thread, deduplicated by delivery ID. Cap at 20 entries, dropping the oldest with WARNING. Watch each successful run's StreamBridge; END_SENTINEL drains up to 10 entries into <followups-while-busy> and watches the successor. Requeue drain conflicts until this manager next creates and watches a run. Buffers/watchers are process-local; GitHub has no leader-lease arbitration. Without the zero-arg lifespan → service → manager bridge closure, buffering has no auto-drain. Coalesce silently without reactions. Before changing buffering or watcher plumbing, read the full contract.
• See backend/docs/GITHUB_AGENTS.md for the architecture diagrams: webhook → fan-out → InboundMessage dispatch, preferred_thread_id = UUID5(repo, number, agent_name) thread determinism, mention-handle precedence chain, GH token lifecycle via GH_TOKEN/GITHUB_TOKEN per-call extra_env, and the narrow ConflictError (HTTP 409) thread-create race recovery.

QQ WebSocket channel

qq.py handles text C2C/group @mentions via httpx/websockets and the shared non-streaming manager. Keep service, provider, credential and config registration aligned. Binding scopes: <app_id>:c2c or <app_id>:group:<group_openid>; topics retain the sender. Bind commands precede allowlists; reserve intake before identity lookup.

Start waits for READY; stop drains tasks/HTTP. Keep reads/heartbeats independent of intake and SSL/client setup off-loop. Accept only TLS/443 gateways at api.bot.qq.com or the sgroup.qq.com family; reject URL credentials/redirects. Logs exclude secrets, content, provider bodies and raw exceptions.

thread_ts is the source message ID. Lock each source; preserve expiry, sequence and quota (C2C: four/60 min; group: five/5 min) across redelivery. Ignore progress; reuse sequences only after definite auth failure, never ambiguous I/O. Unknown timestamps use a full advisory window with content-free, rate-limited warnings. Evict expired contexts first; caches are bounded/process-local. One worker per bot.

Tests: test_qq_{channel,lifecycle,binding}.py, connection router and blocking-I/O. Setup: IM_CHANNEL_CONNECTIONS.md.