103 Commits

Author	SHA1	Message	Date
Willem Jiang	dbd72df405	test(channels): await artifact replies between sequential turns (#7616)	2026-10-27 17:51:11 +08:00
Hyeonsang Cho	137c721d2b	fix(channels): write each Feishu thread alias once per streamed reply (#7593) ... * fix(channels): write each Feishu thread alias once per streamed reply Every outbound to a Feishu running card re-sent set_thread_id for all of the reply's aliases (source message, card, topic, root, parent and Feishu thread ids), although only updated_at changed after the first chunk. That was about five store writes per streamed chunk: a full store.json rewrite under the store lock each, or a database upsert. Remember the aliases written for each in-flight reply and skip them on non-final chunks. A failed write is not remembered, so the next chunk retries it, and the final outbound still writes every alias, so the store ends up as before. * docs(changelog): reference #7593 for the Feishu streamed thread alias writes --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-27 10:56:31 +08:00
Totoro	c48447a4ae	fix(runtime): present outputs before a loop wrap-up and explain step-limit stops (#7588) ... * fix(runtime): present outputs before a loop wrap-up and explain step-limit stops Loop-detection warnings now ask the agent to present the files it already created in /mnt/user-data/outputs before its final answer, when present_files is available. A run that exhausts recursion_limit now ends with stop_reason recursion_capped and a plain message that lists outputs it created but did not present. An active goal stands down without an evaluator call. The /wait error envelope keeps the GraphRecursionError name, and IM channels reply with the step-limit message instead of a generic failure. * docs(changelog): note run-end delivery and step-limit stops (#7588) * test(runtime): stop pinning LangGraph's step count in the step-limit test The produced paths and the message already follow the observed model calls, so the exact count only pinned LangGraph's per-step accounting. Also note on the step-limit goal branch that its caller passes no evaluator factory, so it has to stay ahead of the evaluator section. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-27 10:34:34 +08:00
Totoro	ab8cd4e10f	fix(channels): keep IM binding codes on screen and report goal state in replies (#7591) ... * fix(channels): keep IM binding codes on screen and report goal state in replies Connecting Slack, Discord, Feishu, DingTalk, WeCom, Buzz or QQ now opens a dialog that keeps the /connect command on screen with a copy button, its expiry time and the binding status, instead of a toast that disappears. The bots' /connect replies and the IM /goal replies now use the owner's web UI language (else channel_connections.notification_locale), and /goal reports the goal's state as the web goal bar does: in progress, why it stopped, or that it was met. * docs(changelog): note the IM binding dialog and localized IM replies (#7591) * test(channels): check IM reply tables against every notification locale The connect-reply check compared a fixed en-US/zh-CN pair, and the /goal reply table had none. Both now match NOTIFICATION_LOCALES with the same keys in each, like the existing _GOAL_REASON_TEXT check, so a new locale fails here instead of raising KeyError at reply time. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>	2026-10-27 09:46:58 +08:00
Totoro	c36bb699de	fix(channels): honor the configured recursion limit in IM runs (#7521) ... * fix(channels): honor the configured recursion limit in IM runs Every IM channel run asked the Gateway for recursion_limit 100, because the channel manager seeded each run config with it. The Gateway honors a request value, so the top-level recursion_limit never reached IM runs, and a budget of 100 could stop a run after a handful of tool-calling turns (six in one measurement with the default lead-agent middleware; the count changes with the middleware chain). IM runs now send no recursion_limit of their own and get the Gateway's configured default, clamped by max_recursion_limit, like Gateway API runs. A session config value still takes precedence, and GitHub channel runs keep their floor of 250. The README and config.example.yaml IM examples no longer pin session.config.recursion_limit to 100. * docs(changelog): reference #7521 for the IM recursion-limit fix --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-26 12:16:26 +08:00
Nan Gao	085924a3d0	fix(channels): run each IM message once across instances, restarts and late redeliveries (#7104) ... * fix(channels): run each IM message once across instances, restarts and late redeliveries Inbound IM messages were deduplicated only by the 10-minute inbound dedupe store (#7103). Discord direct messages had no key at all, a Discord thread reply got an instance-dependent key, a Discord thread was created before dedupe ran, a redelivery past the window started a second run, and two instances racing on a conversation's first messages bound it to two threads. - Discord messages are keyed (discord, guild_id or "dm", posting channel, message_id); the posting channel is the same on every instance. - MessageBus.claim_inbound lets an adapter record the manager's dedupe key before a side effect; Discord claims it on the Gateway loop before _create_thread, marks the published message dedupe_claimed, and gives the claim back when the message is never committed. - runs.wait / runs.stream / runs.create send an Idempotency-Key derived from the dedupe key, and keyed human message ids are uuid5 of it (a uuid4 per attempt made every keyed retry a 409). A reused run's status-only /wait or stream gap is resolved to this message's own turn via threads.get_state; 409s about the message's own keyed run keep the key instead of reading as a busy thread; an ended reused GitHub run gets no follow-up watcher. - A conversation's first thread binding is claimed insert-if-absent (claim_thread_id on channel_conversations and channel_thread_bindings); the loser adopts the winner and deletes the empty thread it minted. /new and /agent use still rebind. Docs: IM_CHANNEL_CONNECTIONS.md, CONFIGURATION.md, channels AGENTS.md, CHANGELOG (en/zh). * fix(channels): give back Discord dedupe claims for messages never processed Review round 1 on #7104. A Discord claim records the manager's dedupe key before the message is queued, so every path that drops the message unrun must release it, or a peer refuses the redelivery for the rest of the TTL: - The hand-off task releases the claim on cancellation too (stop() drains it while it resolves the connection identity), via await_drained so the cancellation still propagates. - ChannelManager._discard_pending_inbound releases the key of a claimed message that shutdown drops from the queue, after the grace period and on a cancelled shutdown. - The adapter tracks claims until they are committed. A release refused because stop() already closed cross-loop intake, or a hand-off cancelled before it ran, leaves the claim tracked, and stop() releases it last. * fix(gateway): keep IM run idempotency stable when /new rebinds the conversation Review round 2 on #7104. The Gateway scoped every Idempotency-Key by owner and thread. /new rebinds an IM conversation to a fresh thread, so a late redelivery of a message admitted on the old thread was requested on the new one, got a different scoped key and started a second run. - Keys from trusted internal callers (the IM channel manager) are scoped by owner only (`internal-run:` prefix); browser and API keys keep the owner-and-thread scope (`http-run:`). - A key that resolves to a run of another thread or user raises RunIdempotencyKeyMismatch, a RuntimeError subclass so in-process launchers see the same type, and start_run answers it with 409 "Idempotency-Key already used with a different request" instead of a 500. The channel manager already treats that 409 as the message's admitted run. Docs: API.md, IM_CHANNEL_CONNECTIONS.md, gateway AGENTS.md (reworded to stay under its budget), CHANGELOG (en/zh). --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-19 22:57:19 +08:00
Nan Gao	33b8e84f11	fix(channels): keep IM runs alive when the creating request disconnects (#6876) ... * fix(channels): keep IM runs alive when the creating request disconnects The channel manager holds a `runs.wait` / `runs.stream` request open for the whole run, and that request dies with the Gateway instance that received the IM message: a leader hand-off, a rolling restart, a crash. The Gateway cancels a run whose creating request disconnects unless the request asked for `on_disconnect: continue`, so every IM run in flight was cancelled on failover. Channel runs now ask for `continue`, like scheduled-task and MCP-task runs already do. The run finishes on the Gateway executing it and its thread state is persisted, so the answer stays readable in the web UI even though the IM reply is lost with the old transport. The key is set after the fire-and-forget branch because the SDK's `runs.create` has no `on_disconnect` parameter. Docs: CONFIGURATION.md and the Helm/deployment guides no longer list "failover cancels IM runs" as a known limit; IM_CHANNEL_CONNECTIONS.md and the channels AGENTS.md describe the policy. * docs(changelog): link #6876 * fix(channels): keep the inbound dedupe key once a run may be executing With `on_disconnect: continue` a run outlives the manager worker that requested it, but the worker loop released the inbound dedupe key on cancellation, so a provider redelivery on the replacement instance admitted the same event again and executed the turn twice (review on #6876). Each inbound message now carries an admission record: `requested` flips before the run request leaves the manager, `confirmed` once the Gateway proved it admitted the run (a run id, a stream frame, a completed response). The generic release paths (worker cancellation, worker/handler exceptions, a broken stream) keep the key when the run may continue: after confirmation, on cancellation after the request was sent, and on post-request transport failures (read timeout, read error, protocol error). Pre-request failures, Gateway rejections (busy thread, stale pin, status errors, connection failures) and the stream's terminal `error` frame still release it so the redelivery can retry.	2026-10-14 22:07:07 +08:00
Nan Gao	e8c7df9042	feat(channels): arbitrate IM transports with a per-provider leader lease (#6843) ... * feat(channels): arbitrate IM transports with a per-provider leader lease Every Gateway instance connected every enabled IM platform with the same bot credentials, so channels could not run on more than one instance (Telegram 409 storms, Discord answering twice, scattered per-process state). channels.leader_lease was reserved by #6746; this implements it. - channel_leases table (migration 0041): one row per provider. Acquire is one dialect-native upsert decided against the database clock and bumps a fencing epoch; renew extends only a live lease at the holder's epoch; release sets the expiry to 0 so a peer takes over on its next tick. - app/channels/leadership.py: the holder's local monotonic deadline (send time of the last confirmed renewal + lease_seconds); holds() turns false one renewal interval early, so the transport stops at least grace_seconds before a peer may connect. DB calls are bounded. - ChannelService: start fence in _start_channel (no DB call) behind every start path; fixed-rate supervisor renews/acquires and runs transport starts/stops as per-provider tasks; takeover reloads the provider's saved config; failed starts back off; shutdown releases only leases whose transport stopped. get_status reports leader/leader_here and the holder's transport state on every instance. - Notification worker leaves rows of providers another instance holds. - Gate: the enabled=false refusal now recommends auto/true; an explicit true with database.backend memory is refused when a channel is enabled. - Docs, Helm, deployment guides, AGENTS.md and CHANGELOG updated. * docs(changelog): link the channel leader lease entry to #6843 * fix(channels): see peer-held leases for providers absent from local config Review on #6843: a provider connected from the web UI on a peer never enters this instance's channel config, so the notification worker here still claimed its rows (and parked them for 900s) and GET /api/channels/ reported no leader for it. Every supervision round, and startup, now also reads the whole lease table once (bounded), and notification exclusions and status use those observations: a provider whose live lease a peer holds is left to it even without local config. Running that read concurrently with renewals exposed that, through SQLAlchemy's aiosqlite adapter, an UPDATE ... RETURNING racing a read on another connection can hand back one of its own bind values instead of the column (stored rows stay correct; plain sqlite3 and raw aiosqlite do not show it). A lease must never take such a row for a confirmed renewal, so acquire/renew/release now decide by affected-row count and acquire reads its epoch back in the same transaction. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-14 13:51:42 +08:00
Zhengxu (Joshua) Jin	70e8f8974f	fix(channels): parse Slack and DingTalk allowed_users fail-closed (#6742) ... * fix(channels): parse Slack and DingTalk allowed_users fail-closed Slack and DingTalk still built their allowlists with their own _normalize_allowed_users: every entry went through str() without stripping, only list/tuple/set counted as collections, and the gate was `if self._allowed_users and ...`. A configured list whose entries are all blank, such as `allowed_users: [$SLACK_USER_ID]` with the variable set to an empty string, became an empty set and let every sender through. " U123 " never matched U123, and a frozenset was stringified into one ID. Parse both with the shared parse_allowed_users helper that Telegram, WeChat and QQ use, and gate on `is not None`: unset, null, empty collections and a blank string allow everyone; a scalar is one entry; strings are stripped and integers keep matching their string form, as before; booleans, floats, None and mappings are rejected (a mapping as a whole); a configured value with no valid ID denies every sender and logs an error. A scalar containing commas or whitespace warns that it stays one literal ID. * refactor(channels): share the Slack and DingTalk allowlist coercer and separator warning Slack and DingTalk had byte-identical ID coercers and their own copies of the scalar-separator warning. Move the coercer into allowed_users.py as parse_string_or_int_user_id and opt both channels into warn_on_scalar_separators, as QQ and WeChat do. Log messages are unchanged. Pin that a separator-only string such as "," stays one literal ID (denying everyone else) and still warns. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-14 10:56:31 +08:00
ooiuuii	499fbe1142	fix(telegram): allow deleted reply targets in private chats (#6791) ... * fix(telegram): allow deleted reply targets in private chats * fix(telegram): preserve previews after reply target deletion	2026-10-14 10:32:30 +08:00
Totoro	c5bbaea02f	fix(channels): clean up interrupted Telegram initialization (#6810) ... * fix(channels): clean up interrupted Telegram initialization * fix(channels): preserve Telegram shutdown failures and finalize generators --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>	2026-10-14 06:20:33 +08:00
FanouZeng-TT	8319e3384b	fix(telegram): retain loop cleanup across bounded stops (#6772) ... * fix(telegram): retain loop cleanup across bounded stops * fix(telegram): release abandoned cleanup generations --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-13 08:00:41 +08:00
Storm	3f7a076972	feat(channels): Feishu interactive clarification cards (#6670) ... * feat(channels): render Feishu option clarifications as button cards When the agent's ask_clarification carries options, the Feishu channel now renders the question as an interactive card with one button per option instead of plain text. This is the outbound half: card construction plus a process-local pending registry that a later card-action callback handler will validate clicks against. - New feishu_card_interaction module owns request extraction, button card rendering, and the pending registry (30-minute TTL, matching the typed-reply clarification window) - Manager attaches the clarification's structured human_input payload to the final outbound's metadata so channels can render richer affordances without touching the Gateway protocol - Feishu channel renders the button card only when card_interaction_enabled is set (default off); free-text, form, and over-four-option clarifications keep the plain-text path - Button cards bypass the markdown wrapper via a raw_card_content flag on the card send/patch helpers; plain-text call signatures unchanged * feat(channels): close the Feishu button-card click loop The inbound half of interactive clarification cards: a click on an option button validates, resolves the clarification, and marks the card in place. - Register the card-action callback only when card_interaction_enabled is set; clicks arrive as WS EVENT frames through the SDK's callback processor map, and the synchronous return value becomes the toast ack - consume_if_originator atomically validates operator identity against the pending entry: non-originators get a refusal toast and the entry survives; unknown or expired requests are toast-only and dropped - A validated click consumes once, updates the card to a terminal resolved state, and injects a hidden inbound carrying the human_input_response so the run resumes on the originating thread with Web-card parity - Manager translates the inbound's human_input_response metadata into the human message's additional_kwargs with hide_from_ui set * docs(channels): document Feishu interactive clarification cards - README / README_zh: optional card_interaction_enabled setup, calling out that card.action.trigger lives under the console's callback configuration tab (separate from event subscriptions) and requires publishing a new app version - config.example.yaml: commented flag with the same console caveat - backend/AGENTS.md: module ownership note for feishu_card_interaction * fix(channels): close Feishu card-click routing and ack gaps from review Address review on #6670: - Carry the routing identity (topic_id, connection_id, owner_user_id) on the pending entry and the synthetic inbound, so clicks resolve to the clarified thread in topic-scoped chats and pass bound-identity admission instead of opening a new thread or being rejected - Validate option_value against the registered options at consume time: an unknown value is treated like a stale card and the entry is kept - Return a warning toast when a click cannot be dispatched, and flip the card to its resolved state only after the inbound is durably in the bus — never display a choice that was not recorded - Drop the dead mark_sent/consume registry APIs and pass the payload's own option ids through to the buttons - Add an end-to-end test through the manager's thread resolution pinning that a card answer reuses the clarified thread * fix(channels): carry workspace_id on the Feishu card-click inbound The connections-enabled admission path re-reads the binding with the message's workspace_id, and Feishu connections are bound with workspace_id = chat_id. The synthetic inbound set connection_id and owner_user_id but never workspace_id, so the re-read missed and every click was rejected as unbound after the entry was consumed. - Set inbound.workspace_id from the entry's chat_id (Feishu's chat-as-workspace binding convention) - Extend the card-click e2e to require_bound_identity=True, driven through the channel's real _resolve_card_click production * fix(channels): stabilize Feishu clarification delivery and routing Option cards could fail through the real reply SDK, resume another conversation, or leave a stale text hint after the answer was accepted. Unify raw-card delivery and persist trusted thread aliases before showing buttons. Keep protocol request IDs separate from bounded, one-time callback tokens; validate originators and option pairs, and retire only the accepted card's text/button state. Treat valid hidden human answers as turn boundaries so old replies and artifacts do not leak into the next answer. Cover consecutive clarifications and /new routing through real SDK request models, stores, and the run journal. Validation: 147 focused tests and 29,421 offline tests passed (405 skipped); strict blocking-I/O passed 245 tests (1 skipped). Ruff and guidance checks passed. * fix(channels): preserve clarification context in Feishu cards Button cards replaced the full clarification text but omitted its background, leaving users without the context needed to choose an option. Carry optional context from request metadata and render its original Markdown before the question. Keep question-only cards for absent, blank, or invalid context. Cover rendering and all real SDK delivery paths, and update both user guides. Validation: 156 focused tests, 245 strict blocking-I/O tests, Ruff, and guidance checks passed. The offline run recorded 29,381 passed, 49 failed, and 405 skipped; 48 import failures reproduced before the fix. All 49 cases passed on both versions in isolated environments, and the Feishu-to-export order check passed 13 tests.	2026-10-11 23:58:54 +08:00
Jason Cat	37ff61e87b	fix(tests): make oversized-payload tests pass on Windows hosts (#6654) ... test_skillscan_env_launcher.py and test_channels.py put payloads of up to 52 KB directly in parametrize values, so pytest's generated item ids exceed the 32767-character Windows environment-variable limit: PYTEST_CURRENT_TEST raises ValueError at setup and teardown even though the products under test behave correctly, and the giant ids flood -q reports. The skillscan linearity test additionally spawned one child process per chain. A fresh child spends 9-18 s importing the skills stack before the 4-168 ms scan, so its fixed 10 s timeout measured the import instead of the scan and flipped to subprocess.TimeoutExpired on a loaded host. Give the oversized cases short explicit ids, scan every chain in one child process fed through stdin, and enforce the linearity bound on each scan inside that child (5 s) while a 60 s outer timeout covers the import.	2026-10-11 17:47:20 +08:00
Nan Gao	db7340d2fa	fix(security): add DEER_FLOW_CREDENTIALS_KEY and wire the channel credentials cipher (#6611) ... * fix(security): add DEER_FLOW_CREDENTIALS_KEY for at-rest credential encryption There was no at-rest encryption key. ChannelCredentialCipher had no production caller: the channel service, the scheduled-task notification outbox and the channel-connections router all built ChannelConnectionRepository without a cipher, so store_credentials refused (RuntimeError) and get_credentials always returned None. - New harness module deerflow.config.credentials_key: env-only DEER_FLOW_CREDENTIALS_KEY (comma-separated Fernet keys; first encrypts, all decrypt via MultiFernet), CredentialsCipher with encrypt_text / decrypt_text / rotate_text, a fernet:v2: prefix with legacy fernet:v1: values still readable, and get_credentials_cipher() as the process-wide entry point. Malformed keys are refused without echoing key material. Without the variable a single instance generates {base_dir}/.credentials_key. Never derived from AUTH_JWT_SECRET. - New deerflow.config.secret_file.read_or_create_secret_file: O_EXCL create (0600) plus read-back with a bounded settle window, shared by .credentials_key, .jwt_secret (fixes two replicas cold-starting on a shared volume keeping different JWT secrets) and the managed-model key. - ChannelCredentialCipher now builds on CredentialsCipher (same constructors); the Gateway loads the cipher off the event loop at startup when channel_connections is enabled and passes it to every ChannelConnectionRepository. Undecryptable values stay "missing + warn". - Startup gate: a malformed key stops startup in any topology; a declared multi-instance deployment with channel_connections enabled and no key is refused. Plain multi-worker shares the generated file and is allowed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(deploy): generate and inject DEER_FLOW_CREDENTIALS_KEY in Helm, compose and deploy.sh - Helm: the app Secret generates a Fernet key (randBytes 32 translated to the urlsafe alphabet) and preserves it across upgrades via lookup like AUTH_JWT_SECRET; the gateway reads it through an optional secretKeyRef so a user-managed existingAppSecret without it still boots (the Gateway gate refuses only multi-instance deployments that store credentials). README, NOTES and values document backup, rotation and existingAppSecret. - scripts/deploy.sh resolves the key shell -> .env -> persisted file -> generated, persisting it as $DEER_FLOW_HOME/.credentials_key (0600, noclobber/O_EXCL with read-back): the same file the Gateway generates in its runtime home, so either side's key is the one both keep. - Both compose files pass DEER_FLOW_CREDENTIALS_KEY=${...:-} to the gateway. - .env.example entry with a generation hint; README and CHANGELOG notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(security): make abandoned secret-file replacement single-winner A rename is last-writer-wins, so replicas that restart together and all find the same abandoned empty secret file could each replace it and keep a different value. The process that exclusively creates <name>.replacing now owns the replacement, re-checks the file is still empty, and publishes by renaming the claim over it; peers wait for that value, bounded so a crashed replacer cannot block startup forever. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): link DEER_FLOW_CREDENTIALS_KEY entries to #6611 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(security): name the setting that turns off a credentials-key consumer The multi-instance refusal ended with "or disable channel_connections.enabled=true"; it now says "or set channel_connections.enabled=false". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(security): never expose a secret file before its value is written O_EXCL published the name before its content, so a creator that stalled mid-write left an empty file a peer replaced after the settle window; the resumed creator then returned a value nobody persisted, or deleted the peer's valid file when its own write failed. Creators now write and sync a private temporary file and hard-link it to the name, which fails if the name exists; a failed creator only removes its own temporary file. Filesystems without hard links publish through the same single-winner <name>.replacing claim that replaces abandoned empty files, so creation and replacement are serialized there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(models): recover an empty managed-model key when no catalog exists An interrupted first save could leave an empty key file; every later save then passed empty bytes to Fernet and the endpoint returned 503 until the file was removed by hand. An empty key with no catalog encrypted nothing, so it now goes through the secret-file recovery path; next to an existing catalog it is still refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(docker): pass the dotenv DEER_FLOW_CREDENTIALS_KEY through make docker-start scripts/docker.sh runs Compose from docker/ without --env-file, so the dev gateway's DEER_FLOW_CREDENTIALS_KEY=${DEER_FLOW_CREDENTIALS_KEY:-} entry interpolated to empty and overrode the key the checkout .env supplied via env_file. Export it from the dotenv like AUTH_TRUSTED_PROXIES. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 20:51:09 +08:00
Nan Gao	6c4c74a504	fix(channels): share chat-to-thread bindings through the database (#6558) ... * fix(channels): share chat-to-thread bindings through the database ChannelManager kept its IM chat-to-thread bindings (channel_name:chat_id[:topic_id] -> thread_id) in a per-process JSON file, channels/store.json, loaded once and rewritten whole on every change. With several Gateway processes a binding created on one replica was invisible to the others (the next message for the same chat opened a second thread) and concurrent writers clobbered each other's file. - New channel_thread_bindings table (migration 0036_channel_thread_bindings) and harness package deerflow.persistence.channel_thread_bindings: key PK (the legacy composite string), channel_name, chat_id, nullable topic_id, thread_id, user_id, epoch created_at/updated_at; index (channel_name, chat_id). Dialect upsert keeps created_at; prefix remove and batched ON CONFLICT DO NOTHING import. - app.channels.store.ChannelStore is an async protocol: SqlChannelStore for sqlite/postgres, JsonChannelStore (file I/O via asyncio.to_thread) for database.backend: memory. resolve_channel_store picks at ChannelService construction; ChannelService.start() imports an existing store.json once into an empty table and renames it store.json.migrated (a concurrently importing peer neither duplicates nor clobbers; a populated table leaves the file alone). - Callers updated: manager, connection_identity.lookup_thread_id, Feishu (send() awaits the mapping writer; the synchronous lark callback bridges its two store calls to the Gateway loop with a bounded wait), Discord (channels_dir). - Head pins in the 0027_0028/0029/0032/0034/0035 migration tests point at the new head; docs (channels and migrations AGENTS.md, CONFIGURATION.md, README) and CHANGELOG updated. * docs(changelog): link the channel thread bindings entry to #6558 * fix(channels): reject only the Gateway loop in the Feishu store bridge lark-oapi 1.5.5 invokes the message callback synchronously from its async frame handler, on the private event loop that FeishuChannel._run_ws starts for the SDK thread. The bridge's guard treated any running loop as unsafe, so every real inbound message skipped the stored-topic lookup and the pending-mapping restore: a reply whose parent card was mapped but whose root was not fell back to the unknown root, and could consume an unrelated pending clarification. Only a wait issued from the Gateway loop itself (the loop that executes the submitted coroutine) is refused now; the SDK's own loop may block for the bounded wait, as that thread blocked on the JSON rewrite before. Tests drive _on_message from a worker thread running its own loop, the production shape, for both the lookup and the write path, and pin the Gateway-loop refusal. * fix(channels): defer unresolved Feishu lookups and harden the store.json import Review follow-ups on the chat-to-thread binding store (#6558): - Import: a legacy entry whose key, channel_name, chat_id, topic_id, thread_id or user_id exceeds its column width is skipped and counted with the malformed ones instead of failing the whole batch on PostgreSQL (SQLite silently accepted the same row, so the backends diverged on one file). The corrupt-file warning now says what each caller does: the JSON store starts fresh, the import is skipped and the file kept. - Feishu bridge: store calls run on the Gateway loop under a 3s asyncio.wait_for (the SDK loop also services ws keepalives; the old 10s bound could stall it per message), and the lark-thread wait is that budget plus 1s. A deadline or store error is StoreLookupUnresolved, never a miss: _on_message no longer routes such a reply onto its unknown root (which created a second thread and later overwrote the parent alias). The message joins a per-chat FIFO drained on the Gateway loop with backoff for up to 30s, then is dropped with a warning. _on_message is split into parse / resolve / dispatch so the deferred path dispatches the same way; later messages for the chat queue behind the deferred one. * fix(tests): expect follow_redirects in the Jina client kwargs after #5808 #5808 made JinaClient pass follow_redirects=True to httpx.AsyncClient and updated test_jina_client.py, but test_jina_response_limit.py and test_jina_retry_after.py still pin the exact client kwargs to proxy and trust_env, so every backend-unit-tests shard that lands on them fails on main (the backend unit-test workflow only runs on pull requests, which is why the push to main did not surface it). Update the two expectations to include the new keyword. (cherry picked from commit d5aa23156ce19ce36d95df03953442f83ae6d050) * Revert "fix(tests): expect follow_redirects in the Jina client kwargs after #5808" This reverts commit 5bee00bfb09c65a30f77fb4df47a1f82c47ecef7. * fix(channels): bound deferred Feishu routing by the inbound queue capacity Review follow-up on #6558: a store outage let every subsequent message for a chat join the deferred queue with no capacity check (the bus reservation was only taken after routing succeeded), and each queued message started a fresh 30s retry budget when it reached the head, so a burst could be retained for N x 30s and grow without bound. - Intake is reserved BEFORE routing for every store-backed message (bus.reserve_inbound is thread-safe and already used from SDK threads), so deferred and live messages share the one configured bound, channels.inbound_queue_maxsize; a message that cannot be admitted is dropped at once with a warning, never queued or routed as a miss. - The reservation travels with the message into dispatch (committed by _prepare_inbound), is released on the batch path (the flush reserves at publish time), on drop after the budget, on a failed deferral, on dispatch failure and for every leftover when the drain task is cancelled by stop(). - The retry budget runs from the message's arrival: a queued message whose budget is spent when it reaches the head is dropped without a lookup. - The bounded resolver returns the unresolved outcome instead of raising it through the tracked task, which logged an ERROR for an expected, handled store timeout. - Upstream's new blocking-io test constructs the store by its old class name; it now uses JsonChannelStore. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-10 14:00:00 +08:00
Storm	061a7dfc14	feat(channels): select the model per conversation via /model (#6534) ... * feat(channels): select the model per conversation via /model IM channels could list models with /models but had no way to choose one: the model for a conversation came only from channel/session config or the server default, so a user who needed a stronger or cheaper model for one task had to reconfigure the whole channel or move to the web UI. - /model <name> pins a model to the current conversation, validated against the caller-visible /api/models list (list membership doubles as the authorization check), persisted in thread metadata so it survives a Gateway restart and applies from the next message in place — unlike /agent use, which must start a new conversation. - The pin is written to all three runtime carriers (run context, configurable, context) after the session layers merge, so it wins over channel/session configured models; like any explicitly requested model it also takes precedence over a Custom Agent's configured model. - /model (no args) reports the effective model and its source through one resolver that mirrors the run merge order (default -> channel -> user, context carrier over configurable, both session-config carriers scanned); /model default clears the pin; /models names the pinned model in a footer. - Pin lookups degrade instead of failing messages: the status query reports "Current model: unknown" on a lookup failure (matching the /models footer), and the reuse-thread preload leaves the in-process cache untouched so the next turn retries a transient threads.get. - Reserve "model" in the slash-skill reserved-name contract (backend, contract fixture, frontend), register the Telegram handler, and document the command in /help, README, and README_zh. * fix(channels): fence model pin cache and mirror runtime model resolution Review follow-up for /model: - Serialize cold pin loads with persist+cache publication under a per-manager lock, so a slow threads.get that captured pre-pin metadata can never overwrite a concurrently committed selection (commands are not run-serialized). - _resolve_configured_model_name now replicates the run path's shallow session merges and applies the real runtime precedence — config.context (preserved by build_run_config, unbeatable by the setdefault-filled body context) > top-level context > configurable — instead of scanning per-layer carriers; a later layer replacing a nested carrier wholesale no longer leaks a stale value into the status reply. - /api/models resolves the trusted bound owner for internal channel calls and filters the list with the owner's role, so /model selection gates on the same principal that run admission admits for that account. Tests: pin-write-vs-in-flight-load race, status compared against the real merged runtime config (parametrized across carriers), nested-carrier replacement, and owner-role filtering with the real RBAC provider; each verified red by reverting exactly its guard. * fix(channels): share model resolvers with the run path and harden the pin lifecycle The /model status reply re-implemented the session-layer merge and the Gateway carrier precedence, and it ignored the custom-agent model fallback — on a thread pinned to a custom agent whose config sets ``model:`` it reported "server default" while the run actually used the agent's model. Review also found the pin lock spanned a network call and a stale pin bricked its conversation behind the generic error fallback. - Extract ``_select_carried_model_name`` as the single carrier-precedence rule; the status reply reuses the run path's exact ``_merge_dicts`` layer shapes, so a future layering change has one place to touch. - Extract ``_resolve_message_assistant_id`` / ``_resolve_assistant_id`` and share the assistant precedence chain between the run path and the status reply. - Fall back to the resolved custom agent's configured ``model`` when no session layer carries one, mirroring the agent factory's ``_resolve_model_name(requested or agent_model_name)``; the reply now distinguishes "(from channel configuration)" and "(from agent configuration)". - Run the cold ``threads.get`` in ``_load_thread_model`` outside ``_model_pin_lock``; only cache publication takes the lock, with a post-GET re-check so an in-flight /model write still wins over the older snapshot. - Detect the run-admission allowlist rejection in ``_handle_message`` and recover: clear the stale pin (durable write plus cache under the lock) and reply with the reset and the /model recovery instead of the generic internal-error fallback. - Document /model and the ``channel_model_name`` pin semantics in backend/app/channels/AGENTS.md. * fix(channels): fence stale-pin recovery and cover streaming admission failures Review round 4 found the stale-pin recovery itself unsafe and incomplete: - Recovery cleared whatever pin was current when the delayed admission rejection arrived, so a rejection for an old pin could erase a replacement selection committed mid-flight (and tell the user the new model was unavailable). ``_stale_pin_recovery`` now takes the failed run's own selection (``expected_pin``, read from its run context) and clears only while that selection is still current under the lock; a replaced or already-reset selection gets a non-destructive reply instead. - Feishu/Telegram/DingTalk streaming turns never reached the recovery hook because ``_handle_streaming_chat`` swallows start_run's 400 into its terminal-error reply. The recovery now runs in the streaming terminal-error path too, delivered as the single final outbound with the same dedupe cleanup; blocking, streaming, and fire-and-forget run-creation paths all handle the rejection at the site that knows the submitted selection. * docs(channels): fit the /model entry within the agent-guidance size budget The base file sits one byte under the AGENTS.md hard limit, so documenting /model requires net-zero growth: keep the command-inventory addition and a compact pin-semantics sentence, and losslessly tighten the surrounding prose (the file was already at the limit). * fix(channels): apply the review suggestions on /model replies Three non-blocking findings from the latest review: - The /models pin footer decorated the failure string too, advertising a pin next to an error that lists nothing. The command now fetches the listing through ``_fetch_gateway_json`` and appends the footer only on a successful fetch. - The /model status reply labeled every session-layer model "from channel configuration", misreporting user- and default-layer models. The label is now "session configuration" for every session layer. - Stale-pin recovery replies read as if the turn went through, but the run was rejected before starting and the triggering message never reached the agent. All recovery variants now end with an explicit "Your message was not processed — please send it again." * fix(channels): recover stale model pins on the follow-up drain path The drain is the fourth run-creation path and the only one without stale-pin recovery: a /model pin that left the allowlist while follow-ups were buffered made every drained runs.create fail admission, requeueing the batch into the same 400 on every cycle with no user-facing recovery — contradicting the documented "stale pins self-clear" invariant. - Compute expected_pin from the drain's run context (mirroring _handle_chat_on_thread, initialized None since _resolve_run_params is itself inside the guarded try) and run _stale_pin_recovery on drain failure before requeueing; the pin self-clears and the requeued batch succeeds on the next cycle. - Pass the caller-authoritative thread_id into _stale_pin_recovery: every call site already holds it, so recovery never re-derives it from the message — a store-mapping rewrite between dispatch and recovery can never clear the wrong conversation's pin, and the lookup's failure mode (and its guard) disappears entirely. - Return StalePinRecovery(text, resolved) so requeueing callers can tell a cleared pin from a repeat failure: the drain logs INFO when the next cycle is free of this rejection and WARNING when recovery could not remove it (persist failed, or the selection came from session configuration, which still carries it) — no silent infinite requeue loop. Drain logs carry structured facts only: the reply text ends with a "please resend" note that would double-process buffered messages retried automatically (GitHub, the only buffering channel, sends log-only anyway). - Reword the config-carried recovery reply: with no pin held, the rejected selection came from session configuration — name that source and the fix instead of claiming a default takeover that would not happen. - Correct the _load_thread_model lock docstring (writes deliberately hold _model_pin_lock across threads.update) and read the agent config for /model status off the event loop via asyncio.to_thread, with a Blockbuster-gated blocking_io regression test. - Pin the under-lock re-check in _stale_pin_recovery with a concurrency test (a /model committed during the lock-wait window wins; recovery persists nothing), mutation-verified — the drain is the only caller running on a background task without the serialized-run lock, so it is the path most exposed to that window. * Revert "fix(channels): recover stale model pins on the follow-up drain path" This reverts commit 0b4652c689afb56791e5e670901c97a4bf44a156. The drain recovery work moves to a dedicated PR stacked on this branch, keeping #6534 scoped to the /model feature itself. * fix(channels): recover stale model pins on the follow-up drain path The drain is the fourth run-creation path and the only one without stale-pin recovery: a /model pin that left the allowlist while follow-ups were buffered made every drained runs.create fail admission, requeueing the batch into the same 400 on every cycle with no user-facing recovery — contradicting the documented "stale pins self-clear" invariant. - Compute expected_pin from the drain's run context (mirroring _handle_chat_on_thread, initialized None since _resolve_run_params is itself inside the guarded try) and run _stale_pin_recovery on drain failure before requeueing; the pin self-clears and the requeued batch succeeds on the next cycle. - Pass the caller-authoritative thread_id into _stale_pin_recovery: every call site already holds it, so recovery never re-derives it from the message — a store-mapping rewrite between dispatch and recovery can never clear the wrong conversation's pin, and the lookup's failure mode (and its guard) disappears entirely. - Return StalePinRecovery(text, resolved) so requeueing callers can tell a cleared pin from a repeat failure: the drain logs INFO when the next cycle is free of this rejection and WARNING when recovery could not remove it (persist failed, or the selection came from session configuration, which still carries it) — no silent infinite requeue loop. Drain logs carry structured facts only: the reply text ends with a "please resend" note that would double-process buffered messages retried automatically (GitHub, the only buffering channel, sends log-only anyway). - Reword the config-carried recovery reply: with no pin held, the rejected selection came from session configuration — name that source and the fix instead of claiming a default takeover that would not happen. - Correct the _load_thread_model lock docstring (writes deliberately hold _model_pin_lock across threads.update) and read the agent config for /model status off the event loop via asyncio.to_thread, with a Blockbuster-gated blocking_io regression test. - Pin the under-lock re-check in _stale_pin_recovery with a concurrency test (a /model committed during the lock-wait window wins; recovery persists nothing), mutation-verified — the drain is the only caller running on a background task without the serialized-run lock, so it is the path most exposed to that window. (cherry picked from commit 0b4652c689afb56791e5e670901c97a4bf44a156) * fix(channels): apply the third review round on /model command replies - Delete the dead kind == "models" branch in _fetch_gateway: /models has rendered the listing inline since the pin-footer fix, so the helper's copy was unreachable and had already diverged (unguarded m["name"], no pin footer). The internal-auth-header test now drives the surviving /memory kind. - Guard the inline /models listing comprehension like the _handle_model_command sibling (isinstance(m, Mapping) and isinstance(m.get("name"), str)), so a malformed payload entry can no longer turn the command into the generic internal-error reply; regression: test_models_command_skips_malformed_entries. - Drop the unused user parameter from _resolve_bound_owner_for_model_list: the resolved owner comes solely from the trusted header, and the parameter implied the caller's own principal shaped the filter. * fix(channels): apply the fourth review round on the /model lifecycle - Preserve fresh model pins across cache eviction: a cold threads.get racing a successful /model write could still publish its stale snapshot after clear-on-pressure evicted the fresh entry — the in-cache re-check cannot see an eviction. The model-pin cache now carries a write generation; a load whose generation moved during the GET refetches once instead of publishing over the committed selection. Regression: test_fresh_model_pin_survives_cache_eviction_during_cold_load (blocked GET, write, eviction, then the refetch wins), mutation-verified. - Resolve context-carried agents before reporting their model: /model status now mirrors the run path's effective-agent rule — explicit message/thread choice overwrites every carrier, then the merged carriers' agent_name (new _select_carried_agent_name, sharing the model-name precedence via _select_carried_value), then the session-layer assistant_id that only setdefaults into context. A session with context.agent_name=coder now reports coder-model instead of server default, and assistant_id=writer alongside context.agent_name=coder reports coder-model instead of writer-model. Regressions: context-carried, context-beats-assistant_id, and thread-pin-beats-context status tests. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-10 09:45:17 +08:00
Hyeonsang Cho	b842e06be1	fix(channels): send the CSRF pair on IM /goal writes (#6537) ... * fix(channels): send the CSRF pair on IM /goal writes `/goal <objective>` and `/goal clear` from an IM channel sent their PUT/DELETE to the Gateway with only the internal auth headers. The Gateway's CSRF middleware does not exempt internal auth, so with auth enabled (the default) both writes were rejected with 403 and the channel replied "Failed to set goal." / "Failed to clear goal.". `_goal_request` now sends the same double-submit cookie/header pair as the channel's LangGraph SDK client; both build it from one helper, `ChannelManager._csrf_headers()`. * docs(changelog): reference #6537 --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-09 17:58:09 +08:00
Totoro	35cdcab1c5	feat(scheduled-tasks): signals, honest notices and fair queueing for server-started runs (RFC #6340, 2/2) (#6393) ... * feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks - Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal streak boundary) and stop_condition (the user's stop rule, own column). - Stop rule is normalized (stop_rule.py) and appended only at launch; the stored prompt never contains it. Wording adapts when scheduler.tool_enabled is off. - Willem 1: page-created tasks get goal evaluation, <standing_notes>, the previous-run reference and own-schedule stop; request_stop and the scheduled capability no longer require an origin conversation. - Repository: single reactivation choke point raising ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop condition/note changes (not on resume); goal-check failures neither count nor reset the streak; pause of a finished task returns "finished"; trials on a paused task keep the host pause marker. - New repository reads: automatic_runs_used_for, active_run_status_for, list_manageable_from_thread, thread_relation/thread_run on thread tasks, task_ids_for_thread, run_number, and run_number/total_tokens/summary on run history rows. - REST: goal/max_runs/end_at/stop_condition on create and PATCH (null clears, naive end_at = task timezone), resume recomputes from now with an optional renewal body, once_time_passed, task_finished, limits_exhausted, scheduler_not_running on create, trigger returns outcome/existing/thread_id, task responses add automatic_runs_used and active_run_status. - Coded errors {code, message, params} for routes, shared validation and the chat capability (scheduled_task_errors.py), pinned by contracts/scheduled_task_errors_contract.json; goal notes contract v2 with check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py). - /api/features adds scheduled_tasks {available, running, tool_enabled, min_interval_seconds}; ScheduledTaskService.is_running. - Docs: six README scheduler sections, API.md scheduled tasks section (breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs. - Tests: migration 0031, stop rule, error contract, run listing, lifecycle over HTTP, repository, service and capability cases; updated tests for the new error shape, finished-task pause and head revision. * feat(scheduler): conversation update/resume, run-conversation authority and readable launches - schedule_task: update and resume actions, stop_condition and clear_fields parameters, plain parameter types (value rules come back as coded results), rewritten docstrings for schedule_task and stop_scheduled_task. - Capability: update/resume share the REST validation and error codes; interactive turns in a task's run conversation may manage that task; scheduled runs keep only own-stop; grants require a running scheduler. - Results: compact JSON-safe task views with local times, display card/text, last run in list, timezone_source/browser_timezone, now_local; no ISO, user, lease or origin fields; stop_instructions removed. - Timezone: explicit > browser (context.client_timezone, validated in start_run and never forwarded to the run config) > not needed for intervals and offset run_at > timezone_required. - Trial gate accepts natural direct requests after acknowledgements and particles; trial results report thread_id and existing. - Scheduled launches: stable message id scheduled-<task_run_id>, server-owned deerflow_scheduled_origin metadata with the user-language parts, run thread title for fresh-thread tasks. - Worker records goal continuations in the verdict; lead prompt language rule covers stored task text and scheduled runs. - Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs. * feat(frontend): scheduled-task core types, formatting, errors and glossary - throwGatewayApiError now throws GatewayApiError (status, code, params, rawMessage); message stays the string detail or detail.message - scheduled-task types gain stop_condition, automatic_runs_used, active_run_status, origin/notes, run_number/total_tokens/summary, thread relation, trigger result, renewal and scheduled origin; isTaskBusy and hasActiveRun helpers - api: fetchScheduledTask, resume with optional renewal body, typed trigger result and update payload with null clears - hooks: polling at 3 s while a run is active (list, thread list, run history), useScheduledTask with live gating and final 404, useInView, localized error toasts; run history refreshes tasks when a run finishes - new pure modules: errors.ts (contract code to localized copy), format.ts (relative task times, viewer time, display zone), status.ts, actions.ts, run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked goal checks, describeTaskOutcome and parseStopRunId - features: scheduled_tasks block with defaults when missing - buildRunContext sends client_timezone - titleOfThread takes a localized untitled label; breadcrumb names scheduled tasks and agents; reference chips use the localized untitled label - schedule input: shared browserTimeZone, labelled selects, aria-pressed type toggles, interval minimum from the hint template - i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes, recipe instructions moved into i18n - e2e mocks follow the new task, trigger, resume, thread-task and features shapes; expectNoRawIdentifiers helpers for e2e and DOM tests - tests for errors, format, cron, status, actions, goal outcome, run errors, hooks, run context, thread titles and features; scheduled e2e copy updated * feat(frontend): list-first scheduled tasks page with outcomes and editable goals - page.tsx is now a thin composition of components/workspace/scheduled-tasks: status tabs (All/Active/Paused/Finished) and search over the task list, ?task_id= deep link with router.replace on selection, ?thread_id= chip with Show all, empty state, scheduler-off notice and unavailable state - task list rows show the schedule in words and one status line (next run, running now, paused by agent, auto-paused, finished by its cap) - task detail: status badge, actions from availableActions with reasons, More actions menu (Duplicate, Copy task ID, Delete), outcome notice, Runs / Stops when / Each run's goal / Does / Notes from chat / History, origin chat link; success toasts for every action - run rows: run number or Trial run, summary, goal and assumption badges, continuations, tokens, Open chat link; raw errors and the evaluator reason only behind Details - create/edit/duplicate dialog with goal, stop condition, safety caps and advanced agent/context options; PATCH sends only changed fields, null clears a cap - renew dialog raises or removes caps and resumes in one request; a limits_exhausted Resume opens it - removed i18n keys the old page used; added feedback.resumedNoTime - recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for the new page; DOM tests for the detail and the form dialog - CHANGELOG entries and the frontend AGENTS.md scheduled-task note * feat(frontend): live schedule card, run conversation provenance and header entry - schedule_task results (create, update, pause, resume, trial, delete) become an assistant:scheduled-task message group rendered as a live card: schedule, next run, stop condition with the safety cap, goal, where results go, Run once now, Pause/Resume and Open task; the card polls the task only while it is on screen and shows deleted, scheduler-off and limits-exhausted states - one card per task per assistant turn; the card group is not a turn boundary for usage, branching, editing, outline or skill attribution - a human message with deerflow_scheduled_origin stays a human group with scheduledOrigin and renders as a "Scheduled run / Trial run" block with the task instructions, stop condition and notes collapsed; the launched text is never shown and the message is not editable - readable step labels for schedule_task actions and stop_scheduled_task - ThreadScheduledTasksButton replaces the header link: hidden without tasks, shows the count, links to the task (or the chat filter), and a run's chat links to its task - recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card, run block, header button, step labels and reconnect hydration - i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and frontend AGENTS.md notes * fix(scheduler): address verification findings Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check. * fix(scheduler): address review findings * fix(gateway): no-transform on SSE responses All four text/event-stream responses (thread run create stream, join, existing-run GET/POST stream, stateless create stream) now take their headers from app/gateway/sse_headers.py and send Cache-Control: no-cache, no-transform, so compressing proxies such as Next's rewrite proxy stop buffering streams. Content-Location and X-Accel-Buffering are unchanged; scheduled and MCP-notification runs keep stream_mode=None. Tests pin the headers on every SSE route and fail on a literal no-cache header left in routers. Docs: API.md SSE headers note, CHANGELOGs. * feat(scheduler): per-owner run cap and fair queue selection New scheduler.max_concurrent_runs_per_user (default 2, 0 = off, effective min(cap, max_concurrent_runs)) caps launching/running scheduled runs per task owner. claim_queued_run checks it right after the global count under the same budget lock, so it is atomic across workers and Pods. list_queued_runs ranks same-thread FIFO heads per owner with ROW_NUMBER() OVER (PARTITION BY owner) and orders the batch by that rank, so the drain is owner-fair even with the cap off; owners at their cap drop out before the batch limit. Orphan rows (task deleted while queued) are never held back by the cap. Gateway lifespan passes the startup value; the service constructor default stays 0. Docs: config.example.yaml (config_version 54), CONFIGURATION.md, the six READMEs, backend AGENTS.md, CHANGELOGs (behavior change flagged). The live-task quota is untouched. * feat(scheduler): durable lifecycle events for the originating chat When a chat-created schedule is paused by the agent, auto-paused or finishes, finalization now reports task_stopped / task_paused / task_finished, only on a live -> paused/finished transition. The new finish_task_at_end_condition replaces the three inline "mark completed" blocks (admission rejected as ended, claim-time skip, complete_if_ended), so every site emits exactly once. The scheduler's finalization observer is always installed. It writes scheduled_task_events rows for the originating chat in the same transaction as the state change, deduplicated on (task_id, anchor, event), with a title snapshot, stop condition, run number, last run outcome and the chat's newest run as placement anchor. Lookups fall back on bad stored data; only database errors propagate. Manual trials drop only run_* events. The IM step keeps PR1's selection until B4. New GET /api/threads/{thread_id}/scheduled-task-events (owner check, caller's rows, oldest first); rows survive task deletion and go with the chat. Migration 0032_activity_and_task_events adds the events table plus the run-origin column/index and read-state tables used later, each object guarded on its own. Contract scheduled_goal_notes v3 pins the lifecycle vocabulary. Docs: API.md, migrations chain, backend AGENTS.md, six READMEs, CHANGELOGs. * feat(scheduler): honest, localized and self-contained IM notices Every scheduled-task IM notice is now enqueued by the finalization observer in the transaction that records the outcome, one message per occurrence (task_stopped > task_paused > task_finished > run_failed > run_unmet > run_completed; a once task sends only its run event). The merged auto-pause replaces the separate unmet + pause messages, agent stops and finishes now notify, and recovery-finalized runs notify exactly once. Idle finishes use a deterministic idle: key derived from the lifecycle anchor. The old non-transactional enqueue in handle_run_completion is removed. New app/channels/capabilities.py holds CHANNEL_CAPABILITIES with a proactive_notifications flag (adds the missing qq entry); only push-capable providers (WeCom) get rows, /api/channels/providers returns the flag, and the worker ends legacy rows for other providers as terminal failed. New app/scheduler/notification_text.py renders en-US/zh-CN text: title, what happened (plus the last run's outcome), the tasks page's one-line run summary (redacted) when the agent replied, and a closing line. No IDs, no links, no public_base_url. Locale is the owner's new `locale` preference, else the new channel_connections.notification_locale (default en-US). Docs: config.example.yaml (config_version 55), CONFIGURATION.md, API.md, channels AGENTS.md, six READMEs, CHANGELOGs (behavior change flagged). * feat(gateway): thread origin, activity feed and read markers for server-created threads Runs started by the server for a user (scheduler, IM channels, GitHub agents, extension handles, MCP notifications) now carry a server-owned metadata.deerflow_origin {kind, provider?, namespace?} (new harness module deerflow.runtime.run_origin, pinned by contracts/thread_origin_contract.json). It is stamped only from request.state.run_origin or by the internal channel caller; run admission drops client copies from both metadata and config.metadata, and thread create/patch strip it. RunRepository derives runs.origin_kind on put and on create_thread_operation_atomic (agent runs only), and gains latest_change(). New ThreadReadRepository keeps a per-user, never-decreasing read position (thread_read_markers) and a per-user read clock (thread_read_versions). POST /api/threads/{id}/read marks a thread read, thread search items carry unread, and thread delete removes the markers. GET /api/thread-activity pages the caller's run changes over the run-change clock, returns only threads changed by server-originated runs plus read_version, and 503s without SQL; /api/features reports thread_activity.available. Docs: API.md, gateway AGENTS.md (condensed to stay under its budget), CHANGELOGs. * fix(scheduler): adapt PR2 to PR1 follow-ups PR1 moved the scheduled-task lifecycle rules out of backend/AGENTS.md into the persistence guide and grew the sandbox AGENTS.md chain to within 267 bytes of its hard limit. The per-owner queue-fairness detail now lives in the persistence guide next to the lifecycle rules (backend/AGENTS.md keeps the cap and a link), so check_agent_guidance passes again. PR1's goal-outcome unit test pinned scheduled_goal_notes_contract version 2; PR2 bumps it to 3 (lifecycle vocabulary only, the v2 keys are unchanged). * fix(scheduled-tasks): polish from live acceptance - zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒, and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2 days) instead of 每 1 分钟 or 每 24 小时; several units keep the number - zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between the interpolated time and the Chinese after it; a copy test guards every scheduled-task zh string against a placeholder glued to Chinese - run summary: when the first line of the final reply only leads in to a list (ends with ":" or "："), the list items follow it as plain text (bullets, numbering, checkboxes, bold, code and links stripped), joined with "；" for CJK text and "; " otherwise, within the 160-character cap - a schedule_task result (create, update, pause, resume, trial, delete) that arrives in a chat invalidates that chat's task list, the tasks list and the named task, so the header button and cards update without waiting for the idle poll; results already in the loaded history do not refetch - e2e and unit fixtures are now the live acceptance recordings (DeepSeek Flash, demo data, fictional names): live-weekday.json (create, trial, edit, pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1 and run 2, paused by the agent), fetched with GET from the demo Gateway and trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card, run-thread and tasks-page specs run in both locales over the recorded data * docs(changelog): reference #6378 for the scheduled-tasks entries * docs(agents): shorten the scheduled-task pointer to stay under the guidance limit * test(scheduled-tasks): pin the live takes to their recorded time zone The live fixtures were recorded in Asia/Shanghai, and an interval task reads in the viewer's zone, so the 16:48 assertions failed on UTC CI runners. Pin the browser zone in the two live-take e2e specs and the zh live-take unit test. * feat(frontend): thread origin, activity polling, read markers and task events core Frontend core for server-created threads and schedule lifecycle events (PR2 F1); the sidebar markers (F2) and the chat event line (F3) build on it. - core/threads/origin.ts: threadOriginOf reads metadata.deerflow_origin, then channel_source (github -> GitHub origin), then a legacy scheduled_task_id; key and kinds pinned to contracts/thread_origin_contract.json. labelOfThreadOrigin and the now exported labelOfChannelProvider(provider, t) use the new localized threads.origin.providers names (飞书, 企业微信, ...), with the English map (gaining GitHub) only as the fallback. - core/threads/activity.ts: useThreadActivity polls /api/thread-activity every 15 s (focus refetch, never in the background, off unless features.thread_activity.available). The first response only seeds the cursor; server-origin threads, a truncated page or a higher read_version invalidate the thread lists, a scheduled thread also ["scheduled-tasks"]; an invalid cursor re-seeds. useMarkThreadRead debounces POST /read by 1 s, skips a thread whose cached unread is false, patches the search and infinite caches optimistically and stores the returned read_version so a tab's own read does not refetch its lists. AgentThread gains unread. - core/scheduled-tasks/events.ts: ScheduledTaskEvent, the events query (["scheduled-tasks","thread-events",id], one fetch per saved chat, not gated on the task list so a deleted task's lines stay, refetched when a task of the chat changes status or finishes a run), describeTaskEvent (en/zh copy, title and condition as segments, last-run suffix, See that run / Open task) and placeTaskEvents (end of the after_run_id turn, else the tail, ordered by created_at); lifecycle names pinned to contract v3. - features: thread_activity (default unavailable); channels: proactive_notifications on providers. - Locale sync: the I18nProvider reports the UI language and a signed-in preference session saves it as the new `locale` preference when the account's differs (and on every switch), so IM notices follow the UI language; nothing is written with auth disabled. - i18n: threads.* (unread, origin labels, provider names), scheduledTasks.events.*, history.waitingForSlot (now shown for queued run rows), channels.scheduledUpdates.*. - e2e mocks: thread activity feed and controller, POST /read, the events route, providers with proactive_notifications, search unread, features thread_activity (off by default). Preference e2e mocks return the UI locale so they keep asserting only their own edits. - Docs: frontend AGENTS.md, CHANGELOGs. * fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in - The edit, duplicate and renew dialogs keep a stored end time's instant while its field still shows it. A minute-precision wall value cannot keep seconds or name the later 01:30 of a DST fall-back, so a title-only edit moved the cap an hour earlier, and raising the run limit re-sent the end time without its seconds. - Creating or duplicating a task from a chat's task view drops the chat filter when the new task does not belong to that chat, instead of showing "This task isn't one of this chat's tasks". * feat(frontend): origin markers and unread state in the thread lists Server-created threads (PR2 F2) now say who created them and whether they are new, in the sidebar and on the Chats page, and reach an open sidebar without a reload. - ThreadOriginIcon (thread-channel-source.tsx): clock for a scheduled run, the provider mark for IM threads, the GitHub mark (new in ChannelProviderIcon) for GitHub threads, a puzzle piece for extensions; the localized "Scheduled run" / "From Feishu" / "From GitHub" / "From an extension" label is its accessible name and tooltip. ThreadChannelIcon stays exported as a wrapper, and the provider chip tooltip is localized (no more "Feishu channel"). - Unread dot after the title (aria-hidden dot plus sr-only "Unread"); the row aria-label becomes "{title}, unread" / "{title}，未读". Never shown for the open thread. - WorkspaceSidebar mounts useThreadActivity once. - useMarkOpenThreadRead (components/workspace/chats): the default and custom-agent chat pages mark the thread read once its metadata has loaded, when its cached unread flips to true while open, when the tab becomes visible, and from onFinish (a send or a joined run ended). Each trigger goes through useMarkThreadRead, so it is debounced and a no-op while the cached unread is false. - Tests: thread-origin-icon and recent-chat-list DOM tests (en/zh, active row, no raw identifiers), use-mark-open-thread-read DOM tests, e2e thread-activity.spec.ts (new scheduled run appears unread within one fake-clock poll without a reload, opening it posts /read and clears the dot, a read on another device clears it, IM and GitHub markers, zh labels, the Chats page, no polling when unavailable); thread-history.spec.ts now finds the localized origin icon. - Docs: frontend AGENTS.md, CHANGELOGs. * feat(frontend): schedule event line in the originating chat and channel notice support The chat that created a schedule now says when the schedule paused or ended, and the channel cards say which apps receive scheduled task updates (PR2 F3). - ScheduledTaskEventLine (mockup m2, marker 4): a dashed, muted row with the describeTaskEvent text (title in <strong>, stop condition clamped to one line with the full text in its tooltip, last-run suffix), the time via formatTaskTime and a right-aligned "See that run" (run chat) or "Open task" link. PauseCircle for paused, CircleCheck for finished and once-done, CircleX for once-failed. role="note", aria-label "Scheduled task update" / "定时任务通知", data-task-id and data-event-id only. - MessageList takes scheduledTaskEvents and renders placeTaskEvents output through a new VirtualMessageList.renderAfterGroup, so each line sits at the end of its after_run_id turn inside the group's measured row (also when the list virtualizes); lines with nowhere to go render at the tail. - Both chat pages load useThreadScheduledTaskEvents (not gated on the task list, so the lines stay after a task is deleted). The custom-agent page passes its agent so "See that run" opens the run on that agent's route. - ChannelScheduledUpdates: "Scheduled task updates: sent here" / "... not available for this app yet" (定时任务通知：会发送到这里 / 此应用暂不支持) on the Settings provider cards and under each sidebar channel row; nothing when an older backend does not report proactive_notifications. - e2e mock controller: setScheduledTasks. - Tests: scheduled-task-event-line and channel-scheduled-updates DOM tests (every kind in en and zh, hrefs, clamp, no raw identifiers, Chinese title), MessageList placement DOM tests; e2e scheduled-task-chat-events.spec.ts (the line appears with the card's "Paused by agent" without a reload, See that run opens the run chat, anchored vs tail placement, two events in time order, the line survives task deletion and Open task shows the task is gone; en-3 and zh-1) and channels.spec.ts (WeCom sent here, Feishu not yet; en and zh). - Docs: frontend AGENTS.md, CHANGELOGs. * fix(scheduled-tasks): tie pause notices to the pause they describe - Paused by agent links and times only the run named in last_error. When that run is not loaded, the notice shows no link and the task's own time instead of an earlier run that also asked to stop. - The outcome notice reads the latest history page whichever page is shown, so browsing older runs no longer swaps the auto-pause reason and "Open latest run" to an earlier miss. It shares that page's request. * fix(scheduler): address PR2 review findings - Event lines: hold an event whose anchor run is on an unloaded history page instead of placing it at the bottom (hasMoreHistory); tail fallback only for fully loaded (branched/pruned) history or no anchor. - Idle-finish anchor follows its reason: end:<iso> only for an end_at finish, seq:<n> otherwise, so a later end-time finish is not deduplicated away by an earlier max_runs finish. - Observer takes FOR SHARE on the originating threads_meta row, so a racing thread delete cannot leave an orphan event line. - One taskPagePath (core), re-exported by the card. - Notification locales derived from NotificationLocale; /preferences uses it. - IM notices drop generic reasons (goal_not_met_yet, unknown) instead of repeating the sentence. - Localized provider names on Settings cards, sidebar rows and setup dialog. - zh event lines: no space after a Chinese title (Latin-aware gap). - Unread row label keeps the origin marker; chats page keeps the time as a description. - Sidebar shows the scheduled-updates line only for connected apps; Settings says "available after you connect" for unconnected push apps. - Event line keeps a real space before the time. * fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes - The edit form compares a cron in the form its schedule input emits, so an untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only edit no longer re-arms or refuses a finished task. - Chat resume asks for a timezone, as update does, when a naive end_at would land in the UTC placeholder of a zone-free task. - Re-arming a task clears a host pause marker, as resume does, so a later manual pause no longer reads as "Paused by agent". * test(scheduler): consolidate PR2 backend test setup - Fold the repeated task, origin-chat and observer setup of the lifecycle-event repository tests into one lifecycle_task builder. - Build the per-owner-cap services through one _capped_service builder that keeps every cap explicit at the call site. - Table the five SSE route header checks into one parametrized test with the same routes, bodies and Content-Location expectations. - Reuse bind_wecom and hoist repeated local imports in the event tests. Every case and assertion is kept; collected cases are unchanged. * test(frontend): consolidate PR2 thread activity and origin test setup Extract shared builders and helpers in the frontend tests PR2 adds, with every case, input and assertion kept: - activity.test.ts: seededClient/apply helpers and one LIST_KEYS constant replace the per-test seed, spy and apply boilerplate. - mark-read.dom.test.tsx: newClient() and advance() replace the repeated QueryClient wrapper and fake-timer act blocks. - origin.test.ts: the English and Chinese label checks become tables; the no-English-name check now runs on every Chinese row. - thread-activity.spec.ts: a scheduledRun() builder replaces four inline copies of the scheduled run thread. - channels.spec.ts: the PR2 block uses setLocaleCookie and one card helper. Collected cases are unchanged (unit 211, e2e 60); no test was removed. * fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages useLatestScheduledTaskRuns kept React Query's default refetch on mount, focus and reconnect, so going offline and back while an older history page was shown fetched the latest page again. Refreshing stays with the history, which refreshes the latest page only while it is shown. * fix(scheduler): two gaps found in live acceptance - A fresh scheduled run thread is named when it launches. The worker copied the title to the thread list only when the run ended, so the sidebar, which now shows the new run thread at once, listed it as untitled while it ran. - A task capped at one run finishes with "its one run is done" / "唯一一次运行已完成" in the chat event line and the IM notice, instead of "all 1 runs are done". * test(scheduled-tasks): let the viewer's time carry its own day label Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days, so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the fixed "01:00 · 09:00 your time" assertion failed depending on the hour. * test(scheduler): expect the one-run finish copy in service notices * fix(gateway): create a scheduled run thread with its title Pre-setting the name after start_run raced the thread row, which the run worker creates asynchronously: live, one of two run threads still reached the sidebar untitled. Run admission now creates a new thread with the title its run input carries, which the worker would copy only at run end. * docs(changelog): reference #6393 for the scheduled-task signal entries * fix(scheduled-tasks): route event-line run links by the agent that ran them The originating chat's agent was used for "See that run", but a task's agent can change after the chat created it, so the link could open the run chat on another agent's route. Lifecycle events now record run_agent_name, the task's agent when the run finished (edits are refused while a run is active), and the line routes by it. * fix(scheduled-tasks): load up to the route's maximum of chat event lines The chat loads its lifecycle events once, and the route's default of 50 would silently drop the oldest lines of a chat with many schedules. Ask for the maximum, 200. * fix(threads): keep activity reads and the first cursor in step with the lists - Accepting the first activity cursor refreshes the thread lists once. The lists load on their own, possibly before a run the seed already counts, and later polls start after it, so such a thread never appeared. - A local read whose returned read clock jumps past that one read also covers reads on other devices; the lists are refreshed instead of the clock silently skipping those threads' stale unread dots. - An invalid cursor re-seeds through the same path, so it refreshes once. * docs(agents): bring the middlewares guidance chain back under the hard limit #6378 and #6363 each fit the budget, but together the middlewares AGENTS.md chain on main is 98,425 bytes, 121 over the 98,304 hard limit, so Lint Check fails on main. Move the scheduled-task recovery, launch-update and queue-timeout rules from backend/AGENTS.md into the persistence guide's lifecycle section, which backend/AGENTS.md already points to. The chain is now 98,067 bytes. * fix(helm): bump the chart's config_version to 55 config.example.yaml moved to 55 in this PR, so the Helm chart's validate-chart drift check failed. The chart's embedded config needs no field changes: the new scheduler and channel keys have defaults. --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-07 10:22:15 +08:00
Hyeonsang Cho	3f73801086	fix(channels): give channel run input messages an id (#6401) ... * fix(channels): give channel run input messages an id Channel runs sent their human message without an id. The Gateway stores run input as sent, so the run record kept it id-less, while add_messages gave the checkpoint copy a fresh uuid. When a web client opens the thread while the run is still going, its reconnect hydration rebuilds the input from both copies and matches them by id, so the user's message showed up twice until the run finished and the view fell back to the checkpoint. _human_input_message now stamps a uuid on every message it builds, so the run record and the checkpoint carry the same id. It covers both channel run paths: regular turns and the coalesced follow-up drain. * docs(changelog): reference #6401	2026-10-07 08:44:43 +08:00
Shxiao	a97772d016	fix(wechat): deny all users when allowed_users has no valid ID (#6376) ... * fix(wechat): deny all users when allowed_users has no valid ID A single iLink user ID was split into characters, and a list with no usable ID collapsed to an empty set that the inbound gate read as allow-all. * fix(wechat): fail closed on numeric and mapping allowed_users Integer-valued floats now match the sender's integer text. Fractions, non-finite numbers, and mappings no longer admit anyone, and a denied sender is dropped before inbound media is downloaded. * docs(channels): keep WeChat guidance within size budget * fix(wechat): warn on scalar allowlist separators * refactor(channels): share fail-closed allowlist parsing --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-06 21:17:11 +08:00
Shxiao	b85e4ae74b	fix(channels): serialize lifecycle mutations under per-channel lock (#6304) ... * fix(channels): serialize lifecycle mutations under per-channel lock * fix(channels): invalidate stale config snapshots across lifecycle mutations under per-channel lock	2026-10-05 14:35:51 +08:00
Hyeonsang Cho	12fb53e5de	fix(channels): follow Buzz threads mapped in the connection repository (#6232) ... * fix(channels): follow Buzz threads mapped in the connection repository With channel_connections enabled, ChannelManager stores a bound author's thread mapping only in the connection repository, but Buzz's thread-follow gate read only the JSON ChannelStore. engaged_thread was therefore always False for authors bound with /connect, and every unmentioned reply in a thread the bot was already answering was silently dropped. Buzz now resolves the connection identity before the mention gate and looks the thread up through lookup_thread_id, a helper extracted from ChannelManager._lookup_thread_id that both sides now share, so the gate and the manager can no longer disagree about which store owns a mapping. * fix(channels): drop the manager's own JSON-store read in the skill whitelist _resolve_available_skill_names fell back to reading the JSON ChannelStore when the caller's lookup_thread_id found nothing. For a bound message that lookup is repo-only, so a legacy unbound mapping for the same chat decided which custom agent's skill whitelist applied. The parameter is now required and the caller passes "" when nothing is mapped. Also pin the Buzz thread-follow fix against the real SQL connection repository (bound through a real /connect) instead of a fake. * docs(changelog): reference #6232 for the Buzz thread-follow fix * fix(channels): log Buzz drops of unmentioned replies with no engaged thread The thread-follow gate was the one drop in _handle_chat_event without a log line. Log it at DEBUG with the thread root and whether the author was bound, matching the allowlist drop, so a report that the bot stopped following replies can be told apart from a lost bind or missing mapping. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-04 20:38:43 +08:00
Hyeonsang Cho	9bcf93ec40	fix(telegram): deny all users when allowed_users has no valid ID (#6230) ... * fix(telegram): deny all users when allowed_users has no valid ID TelegramChannel parsed channels.telegram.allowed_users with int() and dropped every entry that failed without logging, and _check_user treated an empty result as "no allowlist". A list of @usernames therefore opened the bot to everyone. A single string ID was iterated character by character ("123456" allowed users 1-6 and blocked 123456), which is what an allowed_users: $ENV reference resolves to, and null or a bare integer crashed the channel at startup. The allowlist is now parsed into frozenset[int] | None: None (unset, null, empty list, blank string) still allows everyone, a single ID is a one-entry list, entries that are not positive numeric IDs (@usernames, floats, booleans, comma-joined strings) are dropped with a warning, and a configured list with no valid ID denies every user and logs an error. * docs(changelog): reference #6230 for the telegram allowed_users fix * fix(telegram): say "positive" in the allowed_users drop warning 0 and negative integers are numeric but still dropped, so "expected one numeric Telegram user ID" misdescribed them. The warning now asks for a positive numeric ID, matching _parse_telegram_user_id, and the drop test covers 0 and -5 alongside @alice. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-03 21:37:39 +08:00
Hyeonsang Cho	9a97cb9420	fix(channels): send Discord typing indicators through the discord.py 2.x API (#6138) ... * fix(channels): send Discord typing indicators through the discord.py 2.x API _start_typing() called channel.trigger_typing(), which discord.py removed in 2.0; the project requires discord.py>=2.7.0. The typing loop swallowed every exception, so each tick raised AttributeError and no indicator was ever sent, and the tests never noticed because they mocked the removed method on a SimpleNamespace. Await channel.typing() instead (awaiting the returned Typing sends one indicator), and log a failed tick at DEBUG rather than discarding it. The existing typing-lifecycle tests now mock typing(), and new tests drive _start_typing() against a fake shaped like the 2.x Messageable, which has no trigger_typing. * docs(changelog): reference #6138 * fix(channels): warn on a Discord typing loop's first failed indicator Logging every failed typing tick at DEBUG left a permanently dead indicator invisible at the default INFO level, the same silence that hid the trigger_typing AttributeError. A bot that lost Send Messages in a channel (403 on every tick) or is being rate limited could not be told apart from a working one without enabling DEBUG. Log the first failure of each typing loop at WARNING with the traceback, and later ticks of that loop at DEBUG so a retrying loop cannot flood the log. The level is per loop, so the next reply's loop warns again. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-10-02 22:49:28 +08:00
FanouZeng-TT	4006ca072f	fix(channels): keep the delimiter newline when splitting long Discord replies (#6062) ... * fix(channels): preserve Discord split boundaries * test(channels): cover Discord split at leading remainder newline Signed-off-by: Willem Jiang <willem.jiang@gmail.com> --------- Signed-off-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-09-30 10:17:46 +08:00
Bob	4394c23b66	fix(channels): decode Slack entity escapes in inbound message text (#6045) ... * fix(channels): decode Slack entity escapes in inbound message text Slack sends a user-typed &, < and > as &amp;, &lt; and &gt; in event text, so raw <...> always marks a control sequence. The Slack channel passed that text to the agent as-is, so code, comparisons and names like R&D reached the agent as 'a &lt; b', 'R&amp;D'. Decode the three entities after bot-mention stripping (amp last, so an escaped entity is decoded once) and keep mentions and links untouched. * test(channels): cover Slack entities inside link labels Signed-off-by: Willem Jiang <willem.jiang@gmail.com> --------- Signed-off-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-09-30 09:38:05 +08:00
icn5381	f29c56d2b0	test(wecom): cover SDK compatibility fallback in shutdown (#4788) ... References #4762 Co-authored-by: icn5381 <255778606+icn5381@users.noreply.github.com>	2026-09-22 21:53:36 +08:00
Aleksandr Sapronov	c12a3e6fa0	fix(channels): fix Telegram inbound file download and sandbox readability (#5581) ... * fix(channels): fix Telegram inbound file download and sandbox readability - Shut down the download Bot on the Telegram loop (Bot.shutdown(), which closes its HTTPX clients) before the loop stops, instead of the non-existent Bot.session.close(). - Grant group/other read on channel-downloaded uploads so the non-root AIO/Docker sandbox process can read the root-written 0o600 file. - Apply the sandbox permission change with os.fchmod on a descriptor opened with O_NOFOLLOW (validated as a regular file via fstat), bound to the validated upload inode, so a symlink swapped in after lstat cannot redirect the chmod to a target outside the uploads directory. The open also uses O_NONBLOCK so a sandbox-swapped FIFO cannot block the read-only open before the regular-file check (matching the existing open_upload_file_no_symlink convention). Centralized in a shared apply_upload_sandbox_permits helper reused by the channel inbound path and the HTTP upload readable/writable helpers. - Surface the download failure cause chain in logs with the Bot API URL masked: the configured token is redacted and both URL forms are collapsed, covering the file download URL (/file/bot<token>/...) and the method URLs (/bot<token>/getMe, /bot<token>/getFile). - Migrate the existing receive_file tests onto the download Bot and add coverage for _get_download_bot (loop-bound creation + caching, cleanup on init failure), download-bot routing over the application bot, real-Bot shutdown closing both HTTPX clients, receive_file timeout-containment, masked cause-chain logging (file and method URL forms), and inbound-file sandbox perms (including the swap-after-lstat symlink regression). * fix(uploads): surface sandbox permission failures --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-09-22 20:43:14 +08:00
Eilen Shin	bd995a6a26	fix(agent): align unattended prompt with tool policy (#4919) ... * fix(agent): align autonomous interaction guidance * fix(agent): harden interaction policy selection * fix(gateway): protect legacy interaction flags * fix(channels): honor explicit interaction mode * docs(agent): reduce inherited guidance size * fix(agent): honor unattended policy across approval paths --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-09-20 18:57:49 +08:00
Zheng Haoran	4387dce7be	fix(channels): preserve thread-create lock generation (#5480)	2026-09-16 22:20:16 +08:00
Aleksandr Sapronov	a922efe144	fix(channels): send Telegram messages as rich only when content has rich constructs (#5470)	2026-09-16 21:22:54 +08:00
Hyeonsang Cho	8e86729aa0	fix(gateway): confine artifact PUT to /mnt/user-data/outputs after path resolution (#5321) ... * fix(gateway): confine artifact PUT to /mnt/user-data/outputs after path resolution The outputs-only guard on PUT /api/threads/{id}/artifacts/{path} was a string-prefix check on the raw path. A percent-encoded `..` (`outputs/%2e%2e/uploads/x.txt`) survives nginx's variable proxy_pass untouched, is decoded by Starlette, passes the prefix check, and the resolver only confines the result to `user-data/` -- so an owner could overwrite a sibling upload or workspace file in their own thread. Collapse dot segments before the prefix check, and re-check the resolved host path against the resolved outputs root so a symlink planted inside `outputs/` cannot redirect the write either. The normalized virtual path is what the response echoes and what non-mounted sandboxes receive. * refactor(gateway): share the outputs-confinement rule with channel attachments Review follow-up on #5321: the "only under /mnt/user-data/outputs" rule was implemented independently by the artifact editor and by IM-channel attachment delivery, and the two copies had already drifted. Move it into app/gateway/path_utils.py as normalize_outputs_virtual_path (collapse `..` before the prefix check) and resolve_outputs_confined_path (re-check the resolved host path against the resolved outputs root, which also catches a symlink planted inside outputs/). PUT /artifacts and ChannelManager._resolve_attachments both call the helper; artifact_archive keeps its stricter ZIP-member rules layered on top. Tests that previously stubbed resolve_thread_virtual_path for the editor now stub resolve_outputs_confined_path, and the channel attachment tests patch path_utils.get_paths, which the helper binds at import like the other consumers. The confinement itself is pinned by tests/test_gateway_path_utils.py.	2026-09-10 15:01:56 +08:00
hataa	37b03a3811	fix(channels): bound Discord outbound cross-loop awaits and restart dead clients, fixes #5226 (#5227) ... * fix(channels): bound Discord outbound cross-loop awaits and restart dead clients, fixes #5226 * docs: reduce inherited agent guidance size * fix(channels): stop half-started channels before discarding them _start_channel now tears the instance down (stop + untrack) whenever start() raises or the channel never reaches is_running, so an outbound listener subscribed before the transport was confirmed cannot outlive its channel. Addresses the review on #5227. * fix(channels): retain half-started channels until failed-start cleanup completes Ownership in _stop_and_discard_channel now mirrors ChannelService.stop(): the instance is dropped only after its stop() completes. A cancellation arriving mid-cleanup (or a stop() that raises) leaves it tracked, so a retried readiness attempt stops it again before replacing it and service shutdown can still reach it — untracking first orphaned resources nobody could clean up. Addresses the round-3 review on #5227. * fix(channels): defer replacement when a retained channel fails to stop The pre-retry stop in ensure_channel_ready popped unconditionally, so a retained instance whose second stop() raised was untracked with its outbound listener still subscribed — the same orphan one hop later. restart_channel (del after failed stop) and remove_channel (pop before stop) had the same shape. All three now route through _stop_and_discard_channel and decline the operation for that round when the instance is retained, so _start_channel can never overwrite a still-listening channel. Addresses the review on #5227. * fix(channels): enforce the retention guarantee inside the readiness attempt loop A failed attempt whose cleanup retained the instance used to let the next attempt (attempts=2 is the production default) construct a fresh instance and overwrite the retained one via _start_channel's unconditional assignment — orphaning the first instance's subscribed listener one hop earlier than the cross-round guard covers. The guard now lives at the mechanism: _start_channel refuses to install while the name is still tracked, and ensure_channel_ready ends the loop on retention. The shared discard helper's log message is path-neutral. Addresses the review on #5227. * fix(channels): make the retained-instance guard message path-neutral The guard can fire for any still-tracked instance, not only failed cleanup, so the message must not assume the cause. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-09-10 14:36:55 +08:00
PeaceMaker-best	e3df6ea4a8	feat(channels): select custom agents per conversation (#5168) ... * feat(channels): select custom agents per conversation Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> * fix(channels): reserve agent slash command across clients Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> * fix(tui): hide reserved slash commands from skills Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> * fix(channels): preserve selected agent across clients --------- Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>	2026-09-06 16:46:51 +08:00
cui fliter	adfc307677	fix(channels): synchronize ChannelStore reads (#5083) ... Signed-off-by: cuishuang <imcusg@gmail.com>	2026-08-29 08:01:38 +08:00
Eilen Shin	15bbf3a4c1	fix(channels): await real cross-thread tasks on shutdown (#4816)	2026-08-14 23:29:04 +08:00
Eilen Shin	ce4ef1bb2f	fix(channels): bound inbound intake and worker lifecycle (#4800) ... * fix(channels): bound inbound intake and worker lifecycle * fix(channels): harden overload retry and shutdown draining * fix(channels): retain shutdown task ownership	2026-08-14 12:35:24 +08:00
AoHanBei	38ff44778a	fix(wecom): serialize websocket shutdown (#4762) ... * fix(wecom): await connection task shutdown * fix(wecom): serialize websocket shutdown	2026-08-11 22:27:11 +08:00
AoHanBei	df01102dfc	fix(discord): prevent typing tasks after stop (#4752) ... * fix(discord): prevent typing tasks after stop * fix(discord): serialize typing cleanup on event loop * fix(discord): harden typing cleanup on loop exit	2026-08-11 21:54:19 +08:00
ajayr	d732b90dc3	feat(channels): add Buzz (Nostr) channel connector (#4649) ... * feat(channels): add Buzz (Nostr) channel connector Adds a Buzz (https://github.com/block/buzz) channel so DeerFlow can join a Nostr-relay workspace as a member: it answers @mentions in channels, replies to DMs, and streams answers by editing one message in place. * app/channels/buzz_nostr.py — pure NIP-01 helpers: canonical event ids, BIP-340 signing/verification, chat/edit/auth builders, relay frames. * app/channels/buzz.py — BuzzChannel: one NIP-42-authenticated websocket, channel discovery (kind 39000) with one subscription per channel, live membership tracking (44100/44101), per-channel replay watermarks, and replies posted once then edited in place (kind 40003). * app/channels/buzz_run_policy.py — same-thread serialization, mirroring the Feishu precedent. Inbound is gated in order: signature verification, self-drop, /connect bind-and-return, pubkey allowlist, then mention / DM / mention-free / thread-follow. Off by default; needs the new optional `buzz` extra (coincurve, lazily imported), which detect_uv_extras resolves from channels.buzz.enabled the same way it already handles channels.discord. Two relay behaviours drove the design and are worth knowing when reviewing: a global {"kinds":[9]} subscription receives nothing from buzz-relay and a multi-value "#h" filter receives nothing either, so one REQ per channel is required; and a single global `since` cursor skips quiet channels, so watermarks are per channel. Signed-off-by: Ajay R <ajayr@formbuddy.com> * fix(channels): only publish assistant messages from the IM stream `_accumulate_stream_text` decided what streamed `messages-tuple` payloads become displayable text by rejecting ONLY payloads whose `type` contained "tool", so it published everything else. DeerFlow writes hidden model context into the messages channel as ordinary messages -- memory recall and the rewritten user turn as hidden HumanMessages (DynamicContextMiddleware), the `<durable_context_data>` block as another (DurableContextMiddleware) -- and LangGraph fans those state writes out on the messages stream, so they reached every streaming IM channel as the assistant's reply. Proved live on a Buzz relay: the connector published a `<memory>` fact block and, in another run, a verbatim echo of the user's own inbound message. Affects Feishu, Telegram, WeCom and Buzz; worst on Buzz, where each update is an immutable public Nostr event that a corrective edit cannot unpublish. Invert the filter to an allowlist of assistant message types. Two new pure helpers keep it testable: - `_stream_payload_type` resolves the type from both shapes the function already handles: the `model_dump()` shape the gateway emits, and LangChain's `to_json()` constructor shape whose own `type` is the literal "constructor" and whose class name is the tail of the `id` path. - `_is_assistant_stream_type` matches "ai"/"assistant" by PREFIX, not substring -- ordinary words contain "ai" ("chain", "domain"), and a substring test would admit a foreign type name by accident. The bare-`str` branch is removed: an untyped payload cannot be attributed to the assistant, nothing in DeerFlow produces one (serialize_messages_tuple always emits `[message_dict, metadata]`), and a runtime that emitted raw text deltas would emit hidden context the same way. Per-message-id buffering and merging are unchanged. Tests pin both directions, including multi-chunk merging across one message id, so the allowlist cannot silently kill streaming, plus an end-to-end `_handle_streaming_chat` test asserting the live payload never reaches an outbound message. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Ajay R <ajayr@formbuddy.com> * chore(helm): bump config_version to 33 in chart values and README config.example.yaml moved to 33 for the buzz channel block; the chart's embedded config example and its README copy track it (config_version only drives the outdated-config warning, per scripts/check_config_version.sh). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ajay R <ajayr@formbuddy.com> --------- Signed-off-by: Ajay R <ajayr@formbuddy.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-08-05 08:29:22 +08:00
Zhipeng Zheng	838037188e	feat(channels): share inbound webhook dedupe across pods via Postgres (#4210) ... * feat(channels): share inbound webhook dedupe across pods via Postgres (#4120) * ci: run cross-pod inbound dedupe integration tests in CI Expose the job Postgres service via DEDUPE_TEST_POSTGRES_URL so the integration tests (issue #4120) actually execute instead of silently skipping. Normalize the URL for asyncpg (postgresql:// -> +asyncpg, drop libpq-only sslmode) and await the now-async _is_duplicate_inbound in test_github_dispatcher.	2026-07-27 23:07:40 +08:00
Zybnev Sergey	a9a5fc9ced	fix(telegram): render final replies as Rich Messages (#4387) ... * fix: render Telegram replies as rich messages * Исправление fallback Rich Messages в Telegram	2026-07-27 22:51:02 +08:00
March-77	a65eb531ae	fix(telegram): receive inbound attachments (#4392) ... * fix(telegram): receive inbound attachments * refactor(telegram): tighten inbound attachment handoff --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-25 21:55:31 +08:00
Daoyuan Li	d2b5f884e3	fix(channels): buffer GitHub follow-ups during busy runs (#4133) ... Queue comments received while a run is active, then submit one deduplicated follow-up after it finishes. Failed drains are requeued and watcher tasks stop cleanly with the channel manager.	2026-07-24 22:41:07 +08:00
Daoyuan Li	314f84bc8d	fix(feishu): check response.success() on card/reaction SDK calls (#4234) ... * fix(feishu): check response.success() on card/reaction SDK calls _reply_card, _create_card, _update_card, and _add_reaction call the lark-oapi SDK and only used the response on the happy path, never checking response.success(). lark-oapi signals a business-level failure (invalid/expired card, permission error, etc.) by returning a response with success()=False rather than raising, so these calls looked identical to callers whether Feishu accepted them or not. This file's own _upload_image/_upload_file/_receive_single_file already guard against exactly this by checking response.success() before trusting the response; the card/reaction helpers just didn't follow that established pattern. The gap is most exposed on _update_card: Feishu supports streaming, so a single conversation issues many _update_card patches, each one a chance to silently drop an update. _send_card_message already has a try/except around _update_card that retries (via _send_with_retry) on non-final failures and falls back to a brand-new card on final ones - but that logic was unreachable because _update_card could never raise on a business failure. Adds response.success() checks to all four methods, raising for _reply_card/_create_card/_update_card (mirroring the upload helpers, and making the existing retry/fallback logic in _send_card_message reachable) and logging a warning for _add_reaction (mirroring _receive_single_file, since a failed reaction is fire-and-forget and must not trigger a redundant resend of the whole card). Adds regression coverage in TestFeishuCardSuccessChecks: a business-failure mock response for each of the four methods, plus two tests driving _send_card_message end to end to confirm the retry and fallback-to-new-card paths actually engage now. * fix(feishu): include log_id in card SDK failure errors + cover create_card retry path willem-bd's review on this PR suggested two non-blocking follow-ups: - _reply_card/_create_card/_update_card's RuntimeError on a business-level failure omitted the Feishu log_id, unlike _add_reaction and _receive_single_file in this same file, which already include it in their warning logs. Adding it gives a Feishu support-traceable id once retries exhaust and the error reaches the caller. - _create_card's failure on the no-thread_ts path (the tail of _send_card_message) only had direct unit coverage (test_create_card_raises_on_business_failure_response), unlike _update_card's failure path, which also has an end-to-end test through send() confirming _send_with_retry engages (test_send_retries_after_update_card_business_failure_then_succeeds). Adds the mirrored end-to-end test for the _create_card path.	2026-07-22 14:52:42 +08:00
Lee minjing	e225ad57d7	feat(uploads): lazy-load historical files via list_uploaded_files tool (#4174) ... * feat(uploads): lazy-load historical files via list_uploaded_files tool Replace per-turn injection of all historical upload metadata with on-demand discovery via a new `list_uploaded_files` built-in tool, following the same deferred-discovery pattern used by skills. - Rename <uploaded_files> block to <current_uploads> (current-run files only) - Add list_uploaded_files tool with include_outline: bool|list[str] - Extract outline helpers to shared deerflow/utils/file_outline.py - Update system prompt to reflect lazy-loading behaviour - Historical file scan removed from UploadsMiddleware.before_agent() Co-Authored-By: Claude <noreply@anthropic.com> * fix(uploads): clear uploaded_files state when no new files in current turn When before_agent() returns None on empty turns, the LastValue uploaded_files field retains the previous turn's filenames. list_uploaded_files then incorrectly excludes those files as "current-run" files, making them invisible until the next upload. Fix: return {"uploaded_files": []} instead of None to explicitly clear state. Add two-turn regression test covering the exact scenario from review feedback. Co-Authored-By: Claude <noreply@anthropic.com> * fix: resolve CI lint errors and stale test assertion from merge - Split long prompt line to fit 240-char limit - Add missing `Any` import in list_uploaded_files_tool - Remove unused `re` import in file_conversion (outline code moved) - Remove unused `os` import in middleware test - Fix test assertion: <uploaded_files> → <current_uploads> after main merge Co-Authored-By: Claude <noreply@anthropic.com> * fix: resolve CI lint errors and stale test assertion from merge - Split long prompt line to fit 240-char limit - Add missing `Any` import in list_uploaded_files_tool - Remove unused `re` import in file_conversion (outline code moved) - Remove unused `os` import in middleware test - Fix test assertion: <uploaded_files> → <current_uploads> after main merge Co-Authored-By: Claude <noreply@anthropic.com> * fix: add current_uploads to input sanitization exempt tags The lazy-loading PR renamed <uploaded_files> to <current_uploads>. The anti-drift guard scans all framework XML blocks and requires each to be either blocked or explicitly exempted. current_uploads wraps trusted server-generated file metadata, not user input, so it belongs in the exempt set. Co-Authored-By: Claude <noreply@anthropic.com> * test: regenerate replay golden after uploaded_files state change before_agent now returns {"uploaded_files": []} instead of None, adding uploaded_files to SSE values events. Regenerated via DEERFLOW_WRITE_GOLDEN=1. Co-Authored-By: Claude <noreply@anthropic.com> * fix: review feedback — memory pipeline, stale tags, state clearing, nits - Match both tags in memory stripping pipeline (uploaded_files|current_uploads) - Remove stale uploaded_files from _BLOCKED_TAG_NAMES - Clear uploaded_files on all before_agent early-return paths - Fix ponytail: stray word in file_conversion re-export comment - Remove dead total_omitted branch in _format_omitted_summary - ruff format fixes Co-Authored-By: Claude <noreply@anthropic.com> * fix: block current_uploads, sanitize only original user content Per review feedback: instead of exempting <current_uploads> (which allows user forgery), move it to _BLOCKED_TAG_NAMES and change InputSanitizationMiddleware._process_request to scan only the original user content (ORIGINAL_USER_CONTENT_KEY) when available. Server-injected trusted blocks are no longer checked against the blocked-tag denylist. Co-Authored-By: Claude <noreply@anthropic.com> * docs: clarify fallback reason in input sanitization comment Co-Authored-By: Claude <noreply@anthropic.com> * @ fix: third-round review feedback — state visibility, sanitization, regex, nits - list_uploaded_files_tool: logger.warning instead of silent try/except on runtime.state read failure (High) - input_sanitization_middleware: _extract_text_from_content skips empty text blocks to match message_content_to_text behaviour; rfind fallback path logs warning for observability (Medium) - memory pipeline regexes: backreference (?P<tag>)(?P=tag) in message_processing.py and prompt.py (Low) - file_conversion.py: re-export moved to top of file (Low) - Tests: middleware→tool state bridge test; integrated forged-tag + multimodal sanitization tests PR #4174 — Follow-up issues: #4212, #4213, #4214 Co-Authored-By: Claude <noreply@anthropic.com> @ * @ fix: 4th-round review — denylist, sanitization, scandir, nits - Add "uploaded_files" back to _BLOCKED_TAG_NAMES (old tag still processed by deermem; user forgery must be escaped) (consistency) - Fix inaccurate rfind-fallback comment: UploadsMiddleware keeps string as string, fallback is unreachable for strings (doc fix) - Distinguish "empty string key" (upload without text) from "non-string key" (caller forgery) so empty-text uploads never escape the server block (edge) - Merge dual os.scandir(uploads_dir) calls into one list re-use (minor) - Add comment on .md sibling skip known limitation: user-uploaded .md files whose stem collides with a converted doc are hidden (boundary, no code change) Co-Authored-By: Claude <noreply@anthropic.com> @ * @ fix: tighten rfind-failure fallback — distinguish server blocks from user blocks When _extract_text_from_content and message_content_to_text disagree on multimodal list content and rfind fails, use content[0] (server-injected <current_uploads> block) vs content[1:] (user blocks) to sanitize only user blocks. Raw strings and non-standard dict blocks that _extract_text_from_content misses are now also sanitized. Non-distinguishable paths (< 2 text blocks, non-list content) still degrade to full sanitization (safe — server block may be escaped but user forgery never leaks). All fallback paths log via logger.warning. Decision 18 / willem-bd 4th-round comment #3 Co-Authored-By: Claude <noreply@anthropic.com> @ * @ fix: correct comments referencing text_blocks → content in rfind fallback Co-Authored-By: Claude <noreply@anthropic.com> @ * fix: 5th-round review — dead code, subagent gating, integration test, perf, consistency - Delete unreachable ORIGINAL_USER_CONTENT_KEY guard in rfind fallback branch (original_user_content guaranteed non-empty str at that point) - Remove list_uploaded_files from BUILTIN_TOOLS; add include_upload_tool param to get_available_tools(), default True; task_tool.py passes False so subagents no longer receive a tool whose state exclusion is broken - Add integration test exercising real create_agent graph (not mocked runtime.state) to verify LangGraph propagates before_agent state writes into ToolRuntime.state during same-turn tool calls - Cache DirEntry.stat() st_size in candidates tuple to avoid second per-file syscall in the rendering loop - Make the upload-tag pre-check case-insensitive (content_str.lower()) to match _UPLOAD_BLOCK_RE re.IGNORECASE PR #4174 — willem-bd 5th-round review items #1-#5 Co-Authored-By: Claude <noreply@anthropic.com> * fix(channels): pass files metadata through _human_input_message() for IM uploads _human_input_message() was not passing additional_kwargs.files to the downstream message. UploadsMiddleware read no files, wrote uploaded_files=[], and list_uploaded_files reported same-run IM attachments as historical files (fancyboi999 repro). Fix: add files parameter to _human_input_message(), call site passes files=uploaded. Regression test locks the contract. Co-Authored-By: Claude <noreply@anthropic.com> * fix(channels): remove legacy <uploaded_files> manual prepend to fix double-injection regression Commit 8d86dbf6 added files= pass-through to UploadsMiddleware but left the manual _format_uploaded_files_block() prepend in place. Every IM attachment reached the model twice — once via the legacy <uploaded_files> block and once via <current_uploads>. This commit removes the manual prepend and the now-dead _format_uploaded_files_block() function. UploadsMiddleware is the sole upload-context producer for both IM and web paths. Reported-by: fancyboi999 (PR review) Co-Authored-By: Claude <noreply@anthropic.com> * docs: update #4212 issue body to reflect completed fixes and narrowed remaining scope * chore: remove temporary scratch file * fix(middleware): neutralize user-derived values inside <current_uploads> block Upload-derived filenames, paths, outline titles, and preview text are interpolated verbatim inside the trusted <current_uploads> wrapper, which InputSanitizationMiddleware exempts from sanitization. A crafted filename or document heading containing blocked authority tags would bypass the guardrail and enter model context as trusted framework data. Fix: call neutralize_untrusted_tags() on all four user-derived values inside _format_file_entry(), preserving the outer <current_uploads> wrapper untouched. Reported-by: fancyboi999 (P1 security review) Co-Authored-By: Claude <noreply@anthropic.com> * fix(middleware): neutralize extension labels in omitted-file summary Files exceeding the 10-item context cap bypass _format_file_entry(). Their extensions, derived from user-controlled filenames via _extension_label(), were interpolated verbatim into the trusted <current_uploads> wrapper — another path for blocked authority tags to escape the guardrail. Fix: neutralize extension values inside _extension_label(), the single extraction point for all extension labels. Reported-by: fancyboi999 (P1 security review) Co-Authored-By: Claude <noreply@anthropic.com> * fix(tools): neutralize user-derived values in list_uploaded_files tool result Apply neutralize_untrusted_tags() to every model-visible user-derived value returned by list_uploaded_files: filename, virtual path, extension, outline titles, outline preview lines, and omitted-file extension summary. This closes the last remaining injection bypass in the upload lazy-loading path - the <current_uploads> block and its omitted summary were already neutralized (previous commits), but the list_uploaded_files tool produced a second exit for the same attacker-controlled metadata that ToolResultSanitizationMiddleware did not cover. Co-Authored-By: Claude <noreply@anthropic.com> * fix(tests): add missing include_upload_tool=False to task_tool mock assertions PR #4174 added include_upload_tool parameter to get_available_tools(). task_tool.py correctly passes include_upload_tool=False for subagents but 5 existing tests' assert_called_once_with expectations were not updated, causing CI failures. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-07-22 14:02:56 +08:00
Daoyuan Li	2bb22643ad	fix(feishu): check response.success() on send_file's reply/create calls (#4335) ... willem-bd's review on PR #4234 (Feishu card response.success() checks) flagged send_file (around lines 313-318) as having the same unchecked- response pattern: after _upload_image/_upload_file (which already raise on a business-level failure), the resulting file/image message.reply or message.create call's response was never checked, so a failed send logged "file sent" and returned True exactly like a real success. Adds the same response.success() check used by _reply_card/_create_card/ _update_card, raising (caught by the existing try/except, which already logs and returns False) so the caller can no longer mistake a silent business-level failure for a delivered file/image.	2026-07-21 18:37:10 +08:00
Daoyuan Li	8153e68eb8	fix(channels): escape Slack reserved chars before mrkdwn conversion (#4197) ... * fix(channels): escape &/</> in Slack outbound text before mrkdwn conversion Slack requires callers to replace &, <, and > with their HTML entity equivalents before sending message text, since an unescaped <...> triggers Slack's own mention/link syntax (e.g. <@USERID>, <http://url|label>). SlackChannel.send() ran msg.text through the markdown-to-mrkdwn converter and sent the result straight to chat_postMessage with no escaping step, so technical/code content like "if a < b && b > c:" would arrive as literal Slack markup and render broken or misinterpreted. Escaping must happen before the mrkdwn conversion, not after: the converter emits its own <url|label> syntax for real markdown links ([text](url)), and that generated syntax must reach Slack unescaped. Escaping raw input first (via html.escape(text, quote=False), which replaces & before </>) and leaving the converter's output alone satisfies both requirements. * fix(channels): preserve a line-leading > as Slack's blockquote marker _escape_slack_text's html.escape(text, quote=False) replaces every ">" with "&gt;", including a ">" at the start of a line -- Slack's own mrkdwn blockquote marker, which the converter otherwise passes through unchanged. Agent output using markdown blockquotes for quoting/callouts lost its blockquote styling and arrived as a literal "&gt; " prefix instead of a rendered blockquote. Only "&" and "<" neutralize Slack's "<...>" mention/link syntax; ">" is special to Slack only at the start of a line. Restore a line-leading ">" to a literal character after escaping (re.sub with MULTILINE), so the mrkdwn converter still renders it as a blockquote; a "<"/"&" anywhere, and a ">" that is not at a line start, still escape. New tests cover the line-start preservation, that the exemption does not widen into "never escape '>'" (a mid-line/non-leading ">" still escapes), and that restoration applies per line in multiline text; all three revert cleanly against the unconditional html.escape() to reproduce the blockquote-corruption bug. Full test_channels.py (259 tests) plus ruff check/format are clean.	2026-07-21 10:02:49 +08:00
Daoyuan Li	9a5d701355	fix(channels): document GitHub dedupe TTL, tighten redelivery tests (#4274) ... * fix(channels): document GitHub dedupe TTL, tighten redelivery tests Follow-up to PR #4104's review, whose non-blocking notes were left unaddressed at merge time: - Document the previously-undocumented inbound-dedupe TTL (INBOUND_DEDUPE_TTL_SECONDS, 10 min, in-memory only) directly above the constant in ChannelManager: what it is, why the window, and what happens at the boundary (a manual "Redeliver" clicked after the TTL has elapsed, or any redelivery after a Gateway restart, is not deduped, since _recent_inbound_events is never persisted to ChannelStore). Cross-reference it from the GitHub dispatcher's fan-out comment, which previously implied unconditional redelivery coverage. - test_channels.py::test_github_redelivery_is_deduped_like_other_channels: the _gh helper stamped a 2-part delivery:agent id while production stamps a 3-part delivery:user:agent id. Align the helper with the real shape and add a cross-user assertion the old 2-part shape could not even express. - test_github_dispatcher.py::test_missing_delivery_header_leaves_dedupe_open: previously only asserted the dispatcher-layer id was None. Add the manager-level _is_duplicate_inbound assertion across two header-less deliveries so a future regression that treats a missing id as a real key is caught here too, not only at the dispatcher layer. All four affected test files (test_github_dispatcher.py, test_channels.py, test_github_channel.py, test_github_registry.py) pass; ruff check and ruff format --check are clean. Verified the two tightened tests actually discriminate by temporarily reintroducing each bug they target (2-part id shape; missing-id treated as a real key) and confirming the new assertions fail, then reverting. * fix(channels): correct GitHub redelivery claim in dedupe comments fancyboi999's review on this PR flagged that the previous commit's new comments (manager.py, dispatcher.py) justified the 10-minute dedupe TTL by claiming GitHub automatically retries a failed delivery after its 10-second timeout. GitHub's own documentation says the opposite: failed deliveries (non-2xx, timeout, connection error) are recorded as failed and never automatically redelivered, for both repository and GitHub App webhooks alike. Every redelivery is explicit: the "Redeliver" button, the REST API, or an operator's own recovery script. - manager.py: rework the INBOUND_DEDUPE_TTL_SECONDS comment to state GitHub's actual behavior (no auto-retry), cite the authoritative doc (docs.github.com/en/webhooks/using-webhooks/handling-failed-webhook-deliveries), and justify the 10-minute window as a bound on explicit near-term replays rather than an automatic-retry window. No longer asserts unverified retry behavior for the other channels either. - dispatcher.py: same correction for the dedupe_message_id comment, which previously implied "retried after a timeout" as a distinct, automatic case alongside the manual "Redeliver" button. - test_channels.py / test_github_dispatcher.py: reworded the two docstrings and one section comment that repeated the same "retry-on-timeout" framing, so the corrected mental model doesn't sit next to contradicting prose nearby. No assertions changed. Verified directly against GitHub's current documentation (fetched, not paraphrased from memory): "Handling failed webhook deliveries" states plainly that GitHub does not auto-redeliver; "Automatically redelivering failed deliveries for a GitHub App webhook" / "...for a repository webhook" are both guides for a user-written recovery script (GitHub's own example runs on a 6-hour cron), not a native GitHub feature. No difference in this behavior between GitHub Apps and repository webhooks. All four affected test files pass; the two `test_bot_login_whitespace_only` / `test_trigger_mention_login_whitespace_only` failures in test_github_dispatcher.py are pre-existing on this branch's unmodified HEAD (confirmed against a clean checkout of the same commit) and are unrelated to this change. ruff check and ruff format --check are clean on all four touched files.	2026-07-19 22:08:48 +08:00