* fix(channels): keep IM binding codes on screen and report goal state in replies
Connecting Slack, Discord, Feishu, DingTalk, WeCom, Buzz or QQ now opens a
dialog that keeps the /connect command on screen with a copy button, its
expiry time and the binding status, instead of a toast that disappears.
The bots' /connect replies and the IM /goal replies now use the owner's
web UI language (else channel_connections.notification_locale), and /goal
reports the goal's state as the web goal bar does: in progress, why it
stopped, or that it was met.
* docs(changelog): note the IM binding dialog and localized IM replies (#7591)
* test(channels): check IM reply tables against every notification locale
The connect-reply check compared a fixed en-US/zh-CN pair, and the /goal
reply table had none. Both now match NOTIFICATION_LOCALES with the same
keys in each, like the existing _GOAL_REASON_TEXT check, so a new locale
fails here instead of raising KeyError at reply time.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* fix(discord): accept inbound attachments and file-only messages
Empty content no longer drops messages that have attachments. Capture
CDN descriptors on inbound, download in receive_file via Bot auth, and
hand bytes through INBOUND_FILE_CONTENT_KEY like Slack/Telegram.
Closes#6856
* fix(discord): gate bot auth and share inbound file helpers
Address review on #6857: attach Bot Authorization only for Discord CDN
hosts, report Content-Encoding refusals as download failed, and hoist
filename/size helpers into app.channels.inbound_files (Telegram delegates
to the same module).
* fix(discord): sort telegram imports for ruff I001
CI lint-backend failed on unsorted imports in telegram.py after the
shared inbound_files helper was introduced.
* style: ruff-format discord inbound attachments test
* fix(discord): show the enforced attachment download limit
---------
Co-authored-by: Li-john1021 <Li-john1021@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): select the model per conversation via /model
IM channels could list models with /models but had no way to choose one:
the model for a conversation came only from channel/session config or the
server default, so a user who needed a stronger or cheaper model for one
task had to reconfigure the whole channel or move to the web UI.
- /model <name> pins a model to the current conversation, validated
against the caller-visible /api/models list (list membership doubles
as the authorization check), persisted in thread metadata so it
survives a Gateway restart and applies from the next message in
place — unlike /agent use, which must start a new conversation.
- The pin is written to all three runtime carriers (run context,
configurable, context) after the session layers merge, so it wins
over channel/session configured models; like any explicitly
requested model it also takes precedence over a Custom Agent's
configured model.
- /model (no args) reports the effective model and its source through
one resolver that mirrors the run merge order (default -> channel ->
user, context carrier over configurable, both session-config
carriers scanned); /model default clears the pin; /models names the
pinned model in a footer.
- Pin lookups degrade instead of failing messages: the status query
reports "Current model: unknown" on a lookup failure (matching the
/models footer), and the reuse-thread preload leaves the in-process
cache untouched so the next turn retries a transient threads.get.
- Reserve "model" in the slash-skill reserved-name contract (backend,
contract fixture, frontend), register the Telegram handler, and
document the command in /help, README, and README_zh.
* fix(channels): fence model pin cache and mirror runtime model resolution
Review follow-up for /model:
- Serialize cold pin loads with persist+cache publication under a
per-manager lock, so a slow threads.get that captured pre-pin
metadata can never overwrite a concurrently committed selection
(commands are not run-serialized).
- _resolve_configured_model_name now replicates the run path's shallow
session merges and applies the real runtime precedence —
config.context (preserved by build_run_config, unbeatable by the
setdefault-filled body context) > top-level context > configurable —
instead of scanning per-layer carriers; a later layer replacing a
nested carrier wholesale no longer leaks a stale value into the
status reply.
- /api/models resolves the trusted bound owner for internal channel
calls and filters the list with the owner's role, so /model
selection gates on the same principal that run admission admits for
that account.
Tests: pin-write-vs-in-flight-load race, status compared against the
real merged runtime config (parametrized across carriers),
nested-carrier replacement, and owner-role filtering with the real
RBAC provider; each verified red by reverting exactly its guard.
* fix(channels): share model resolvers with the run path and harden the pin lifecycle
The /model status reply re-implemented the session-layer merge and the
Gateway carrier precedence, and it ignored the custom-agent model
fallback — on a thread pinned to a custom agent whose config sets
``model:`` it reported "server default" while the run actually used
the agent's model. Review also found the pin lock spanned a network
call and a stale pin bricked its conversation behind the generic
error fallback.
- Extract ``_select_carried_model_name`` as the single carrier-precedence
rule; the status reply reuses the run path's exact ``_merge_dicts``
layer shapes, so a future layering change has one place to touch.
- Extract ``_resolve_message_assistant_id`` / ``_resolve_assistant_id``
and share the assistant precedence chain between the run path and
the status reply.
- Fall back to the resolved custom agent's configured ``model`` when no
session layer carries one, mirroring the agent factory's
``_resolve_model_name(requested or agent_model_name)``; the reply now
distinguishes "(from channel configuration)" and "(from agent
configuration)".
- Run the cold ``threads.get`` in ``_load_thread_model`` outside
``_model_pin_lock``; only cache publication takes the lock, with a
post-GET re-check so an in-flight /model write still wins over the
older snapshot.
- Detect the run-admission allowlist rejection in ``_handle_message``
and recover: clear the stale pin (durable write plus cache under the
lock) and reply with the reset and the /model recovery instead of
the generic internal-error fallback.
- Document /model and the ``channel_model_name`` pin semantics in
backend/app/channels/AGENTS.md.
* fix(channels): fence stale-pin recovery and cover streaming admission failures
Review round 4 found the stale-pin recovery itself unsafe and
incomplete:
- Recovery cleared whatever pin was current when the delayed
admission rejection arrived, so a rejection for an old pin could
erase a replacement selection committed mid-flight (and tell the
user the new model was unavailable). ``_stale_pin_recovery`` now
takes the failed run's own selection (``expected_pin``, read from
its run context) and clears only while that selection is still
current under the lock; a replaced or already-reset selection gets
a non-destructive reply instead.
- Feishu/Telegram/DingTalk streaming turns never reached the recovery
hook because ``_handle_streaming_chat`` swallows start_run's 400
into its terminal-error reply. The recovery now runs in the
streaming terminal-error path too, delivered as the single final
outbound with the same dedupe cleanup; blocking, streaming, and
fire-and-forget run-creation paths all handle the rejection at the
site that knows the submitted selection.
* docs(channels): fit the /model entry within the agent-guidance size budget
The base file sits one byte under the AGENTS.md hard limit, so
documenting /model requires net-zero growth: keep the command-inventory
addition and a compact pin-semantics sentence, and losslessly tighten
the surrounding prose (the file was already at the limit).
* fix(channels): apply the review suggestions on /model replies
Three non-blocking findings from the latest review:
- The /models pin footer decorated the failure string too, advertising a
pin next to an error that lists nothing. The command now fetches the
listing through ``_fetch_gateway_json`` and appends the footer only on
a successful fetch.
- The /model status reply labeled every session-layer model "from
channel configuration", misreporting user- and default-layer models.
The label is now "session configuration" for every session layer.
- Stale-pin recovery replies read as if the turn went through, but the
run was rejected before starting and the triggering message never
reached the agent. All recovery variants now end with an explicit
"Your message was not processed — please send it again."
* fix(channels): recover stale model pins on the follow-up drain path
The drain is the fourth run-creation path and the only one without
stale-pin recovery: a /model pin that left the allowlist while follow-ups
were buffered made every drained runs.create fail admission, requeueing
the batch into the same 400 on every cycle with no user-facing recovery —
contradicting the documented "stale pins self-clear" invariant.
- Compute expected_pin from the drain's run context (mirroring
_handle_chat_on_thread, initialized None since _resolve_run_params is
itself inside the guarded try) and run _stale_pin_recovery on drain
failure before requeueing; the pin self-clears and the requeued batch
succeeds on the next cycle.
- Pass the caller-authoritative thread_id into _stale_pin_recovery:
every call site already holds it, so recovery never re-derives it from
the message — a store-mapping rewrite between dispatch and recovery
can never clear the wrong conversation's pin, and the lookup's
failure mode (and its guard) disappears entirely.
- Return StalePinRecovery(text, resolved) so requeueing callers can
tell a cleared pin from a repeat failure: the drain logs INFO when
the next cycle is free of this rejection and WARNING when recovery
could not remove it (persist failed, or the selection came from
session configuration, which still carries it) — no silent infinite
requeue loop. Drain logs carry structured facts only: the reply text
ends with a "please resend" note that would double-process buffered
messages retried automatically (GitHub, the only buffering channel,
sends log-only anyway).
- Reword the config-carried recovery reply: with no pin held, the
rejected selection came from session configuration — name that source
and the fix instead of claiming a default takeover that would not
happen.
- Correct the _load_thread_model lock docstring (writes deliberately
hold _model_pin_lock across threads.update) and read the agent config
for /model status off the event loop via asyncio.to_thread, with a
Blockbuster-gated blocking_io regression test.
- Pin the under-lock re-check in _stale_pin_recovery with a concurrency
test (a /model committed during the lock-wait window wins; recovery
persists nothing), mutation-verified — the drain is the only caller
running on a background task without the serialized-run lock, so it is
the path most exposed to that window.
* Revert "fix(channels): recover stale model pins on the follow-up drain path"
This reverts commit 0b4652c689afb56791e5e670901c97a4bf44a156. The drain
recovery work moves to a dedicated PR stacked on this branch, keeping #6534
scoped to the /model feature itself.
* fix(channels): recover stale model pins on the follow-up drain path
The drain is the fourth run-creation path and the only one without
stale-pin recovery: a /model pin that left the allowlist while follow-ups
were buffered made every drained runs.create fail admission, requeueing
the batch into the same 400 on every cycle with no user-facing recovery —
contradicting the documented "stale pins self-clear" invariant.
- Compute expected_pin from the drain's run context (mirroring
_handle_chat_on_thread, initialized None since _resolve_run_params is
itself inside the guarded try) and run _stale_pin_recovery on drain
failure before requeueing; the pin self-clears and the requeued batch
succeeds on the next cycle.
- Pass the caller-authoritative thread_id into _stale_pin_recovery:
every call site already holds it, so recovery never re-derives it from
the message — a store-mapping rewrite between dispatch and recovery
can never clear the wrong conversation's pin, and the lookup's
failure mode (and its guard) disappears entirely.
- Return StalePinRecovery(text, resolved) so requeueing callers can
tell a cleared pin from a repeat failure: the drain logs INFO when
the next cycle is free of this rejection and WARNING when recovery
could not remove it (persist failed, or the selection came from
session configuration, which still carries it) — no silent infinite
requeue loop. Drain logs carry structured facts only: the reply text
ends with a "please resend" note that would double-process buffered
messages retried automatically (GitHub, the only buffering channel,
sends log-only anyway).
- Reword the config-carried recovery reply: with no pin held, the
rejected selection came from session configuration — name that source
and the fix instead of claiming a default takeover that would not
happen.
- Correct the _load_thread_model lock docstring (writes deliberately
hold _model_pin_lock across threads.update) and read the agent config
for /model status off the event loop via asyncio.to_thread, with a
Blockbuster-gated blocking_io regression test.
- Pin the under-lock re-check in _stale_pin_recovery with a concurrency
test (a /model committed during the lock-wait window wins; recovery
persists nothing), mutation-verified — the drain is the only caller
running on a background task without the serialized-run lock, so it is
the path most exposed to that window.
(cherry picked from commit 0b4652c689afb56791e5e670901c97a4bf44a156)
* fix(channels): apply the third review round on /model command replies
- Delete the dead kind == "models" branch in _fetch_gateway: /models has
rendered the listing inline since the pin-footer fix, so the helper's
copy was unreachable and had already diverged (unguarded m["name"], no
pin footer). The internal-auth-header test now drives the surviving
/memory kind.
- Guard the inline /models listing comprehension like the
_handle_model_command sibling (isinstance(m, Mapping) and
isinstance(m.get("name"), str)), so a malformed payload entry can no
longer turn the command into the generic internal-error reply;
regression: test_models_command_skips_malformed_entries.
- Drop the unused user parameter from _resolve_bound_owner_for_model_list:
the resolved owner comes solely from the trusted header, and the
parameter implied the caller's own principal shaped the filter.
* fix(channels): apply the fourth review round on the /model lifecycle
- Preserve fresh model pins across cache eviction: a cold threads.get
racing a successful /model write could still publish its stale
snapshot after clear-on-pressure evicted the fresh entry — the
in-cache re-check cannot see an eviction. The model-pin cache now
carries a write generation; a load whose generation moved during the
GET refetches once instead of publishing over the committed
selection. Regression:
test_fresh_model_pin_survives_cache_eviction_during_cold_load
(blocked GET, write, eviction, then the refetch wins),
mutation-verified.
- Resolve context-carried agents before reporting their model: /model
status now mirrors the run path's effective-agent rule — explicit
message/thread choice overwrites every carrier, then the merged
carriers' agent_name (new _select_carried_agent_name, sharing the
model-name precedence via _select_carried_value), then the
session-layer assistant_id that only setdefaults into context. A
session with context.agent_name=coder now reports coder-model instead
of server default, and assistant_id=writer alongside
context.agent_name=coder reports coder-model instead of writer-model.
Regressions: context-carried, context-beats-assistant_id, and
thread-pin-beats-context status tests.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(wechat): deny all users when allowed_users has no valid ID
A single iLink user ID was split into characters, and a list with no
usable ID collapsed to an empty set that the inbound gate read as
allow-all.
* fix(wechat): fail closed on numeric and mapping allowed_users
Integer-valued floats now match the sender's integer text. Fractions,
non-finite numbers, and mappings no longer admit anyone, and a denied
sender is dropped before inbound media is downloaded.
* docs(channels): keep WeChat guidance within size budget
* fix(wechat): warn on scalar allowlist separators
* refactor(channels): share fail-closed allowlist parsing
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(telegram): deny all users when allowed_users has no valid ID
TelegramChannel parsed channels.telegram.allowed_users with int() and
dropped every entry that failed without logging, and _check_user treated
an empty result as "no allowlist". A list of @usernames therefore opened
the bot to everyone. A single string ID was iterated character by
character ("123456" allowed users 1-6 and blocked 123456), which is what
an allowed_users: $ENV reference resolves to, and null or a bare integer
crashed the channel at startup.
The allowlist is now parsed into frozenset[int] | None: None (unset,
null, empty list, blank string) still allows everyone, a single ID is a
one-entry list, entries that are not positive numeric IDs (@usernames,
floats, booleans, comma-joined strings) are dropped with a warning, and
a configured list with no valid ID denies every user and logs an error.
* docs(changelog): reference #6230 for the telegram allowed_users fix
* fix(telegram): say "positive" in the allowed_users drop warning
0 and negative integers are numeric but still dropped, so "expected one
numeric Telegram user ID" misdescribed them. The warning now asks for a
positive numeric ID, matching _parse_telegram_user_id, and the drop test
covers 0 and -5 alongside @alice.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
`_can_send_rich` checked `len(text)`, a code-point count, against the 32768
rich-message cap, while Telegram measures that cap in UTF-16 code units like
every other limit it enforces. An emoji-heavy reply that fits by code points
is still over the cap by code units, so the rich send was attempted, rejected
by the API, and the reply degraded to chunked plain text.
The check now reuses `_first_utf16_chunk`, the bounded scan the plain-text path
already uses, so eligibility costs the limit rather than the whole reply.
* fix(channels): split Telegram messages by UTF-16 code units, not code points
* perf(telegram): clip stream previews without walking the whole reply
Every cumulative stream update measured the full answer and then split the full
answer to keep only its first chunk, so a long reply cost two complete passes
per update on the async channel path. _first_utf16_chunk stops at the first
character that does not fit, and the preview is built from that bounded prefix.
The tests now measure chunk length with the UTF-16 codec instead of a copy of
the production formula, cover the width helper and the bounded clip directly,
and fail if the preview stops being bounded by the limit.
* fix(channels): fix Telegram inbound file download and sandbox readability
- Shut down the download Bot on the Telegram loop (Bot.shutdown(), which
closes its HTTPX clients) before the loop stops, instead of the
non-existent Bot.session.close().
- Grant group/other read on channel-downloaded uploads so the non-root
AIO/Docker sandbox process can read the root-written 0o600 file.
- Apply the sandbox permission change with os.fchmod on a descriptor
opened with O_NOFOLLOW (validated as a regular file via fstat), bound
to the validated upload inode, so a symlink swapped in after lstat
cannot redirect the chmod to a target outside the uploads directory.
The open also uses O_NONBLOCK so a sandbox-swapped FIFO cannot block the
read-only open before the regular-file check (matching the existing
open_upload_file_no_symlink convention).
Centralized in a shared apply_upload_sandbox_permits helper reused by
the channel inbound path and the HTTP upload readable/writable helpers.
- Surface the download failure cause chain in logs with the Bot API URL
masked: the configured token is redacted and both URL forms are
collapsed, covering the file download URL (/file/bot<token>/...) and
the method URLs (/bot<token>/getMe, /bot<token>/getFile).
- Migrate the existing receive_file tests onto the download Bot and add
coverage for _get_download_bot (loop-bound creation + caching, cleanup
on init failure), download-bot routing over the application bot,
real-Bot shutdown closing both HTTPX clients, receive_file
timeout-containment, masked cause-chain logging (file and method URL
forms), and inbound-file sandbox perms (including the swap-after-lstat
symlink regression).
* fix(uploads): surface sandbox permission failures
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Move SQL-dependent connection identity and /start bind work onto the
Gateway main loop via _submit_threadsafe_coroutine, and send PTB replies
through _run_on_telegram_loop so the Telegram worker never blocks or
touches SQLAlchemy/HTTP across event loops.
When the main loop is not running (e.g. during gateway shutdown), the
bind path logs a warning and returns False instead of running SQLAlchemy
on the wrong loop, matching the Feishu bind pattern.
* implement goal continuations
* fix(goal): address review findings for goal continuations
- goal: key the no-progress breaker on a signature of the latest visible
assistant evidence instead of the evaluator's volatile free-text, so it
actually fires on stalled turns; thread the signature through every
worker persist / no-progress call site
- goal: align _stand_down_reason default caps with should_continue_goal
(8 / 2) so the two gate functions agree on goals missing the fields
- runtime: offload the synchronous checkpointer fallback via
asyncio.to_thread (goal.py + worker.py) to keep blocking IO off the loop
- frontend: i18n the GoalStatus "Goal" label (goalLabel in en/zh/types)
- frontend: extract pure composer helpers into input-box-helpers.ts with
unit tests (parseGoalCommand, readGoalResponseError, skill suggestions)
- tests: cover the evidence-based no-progress and default-cap behavior
- docs: align backend/AGENTS.md goal paragraph with actual behavior
- e2e: prettier-format chat.spec.ts (fixes the lint-frontend CI failure)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(frontend): hide goal continuation counter until the agent continues
The goal status bar rendered a raw "0/8" before any auto-continuation, which
read as a mysterious score. Now the counter is hidden until
continuation_count > 0, then shows "Continuing N/M" with a tooltip explaining
the auto-continuation cap.
- Extract getGoalContinuationDisplay into a pure helper (hides at 0) + unit tests
- Add goalContinuing / goalContinuationTooltip i18n keys (en/zh/types)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(goal): address review findings for goal continuations
Frontend correctness
- Fix the optimistic /goal result permanently shadowing server goal state:
the streamed continuation counter never surfaced for a goal set in-session.
Extract a shared useActiveGoal hook (used by both chat pages) that reconciles
the optimistic copy with server state via a goalReconciliationKey, de-duping
the copy-pasted goal block across the two pages.
- Stop /goal status|clear failures from escaping handleSubmit as unhandled
rejections (handleGoalCommand now returns success; the run only starts when a
goal was actually saved).
- Use a function replacer for the goal-status toast so an objective containing
$&/$1 isn't treated as a replacement pattern.
Backend cleanliness / correctness
- De-duplicate four byte-identical helpers (_call_checkpointer_method,
_message_type, _additional_kwargs, _is_visible_message) by importing them
from runtime.goal instead of re-defining them in the run worker.
- Remove the dead `checkpoint_tuple.tasks` durability guard (CheckpointTuple has
no tasks field) and document that pending_writes is the durability signal.
- Decompose the 176-line _prepare_goal_continuation_input: extract
_reread_goal_and_checkpoint and a _persist closure so the thread-unchanged
guard and stand-down persistence aren't open-coded three times. Document the
last-writer-wins write-window limitation as a follow-up.
- Add a shared parse_goal_command helper and use it from the TUI and IM-channel
/goal handlers (one place for the status/clear/set semantics).
Tests
- Restore the 11 command-registry tests dropped by the previous goal change
(filter_commands ranking/description, build_registry builtins/skills, resolve
cases) alongside the new goal tests.
- Add coverage for the IM-channel _handle_goal_command, the TUI _handle_goal
handler, parse_goal_command, and goalReconciliationKey.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix goal review feedback
* fix goal continuation checkpoint races
* prioritize goal commands while streaming
Route composer submits through a shared helper so /goal commands can be handled before the streaming stop shortcut, while ordinary streaming submits still stop the active run.
Testing: cd frontend && pnpm exec rstest run tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; cd frontend && pnpm check
* preserve goal status during clarification
Keep omitted stream goal fields distinct from explicit null clears so clarification interrupts do not hide an active thread goal that is still present in the checkpoint.
Testing: pnpm exec rstest run tests/unit/components/workspace/use-active-goal.test.ts tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; pnpm check; git diff --check
* style: format active goal hook
Run Prettier on use-active-goal.ts to satisfy the frontend lint workflow formatting gate.
Testing: pnpm format; pnpm exec rstest run tests/unit/components/workspace/use-active-goal.test.ts tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; pnpm check; git diff --check
* fix goal review followups
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(channels): make channel connect flow deterministic
* make format
* fix(channels): apply connect-code before allowed_users on telegram and wechat
The bind-bootstrap reorder shipped for slack/dingtalk only. Telegram and
WeChat still gate _check_user/allowed_users before connect-code dispatch, so
a newly allowlisted-but-unbound user is silently rejected when binding via the
browser deep-link / connect-code flow — the same deadlock the PR fixes.
- telegram: consume the /start deep-link token before the allowed_users gate.
- wechat: handle the /connect code before the allowed_users gate, and defer
inbound file extraction + context-token tracking past the gate so blocked
senders no longer trigger CDN downloads or token bookkeeping.
Adds regression tests for both adapters mirroring the slack/dingtalk coverage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(channels): enforce single-active-owner invariant at the DB layer
_revoke_other_active_owners did a SELECT-then-UPDATE in app code with no row
lock or constraint covering active rows. Under READ COMMITTED, two concurrent
connect-code consumes for the same (provider, external_account_id, workspace_id)
from different owners could each observe "no other active owner" and both commit
a connected row, leaving find_connection_by_external_identity nondeterministic.
- Add a partial unique index on (provider, external_account_id, workspace_id)
WHERE status != 'revoked' (portable to SQLite >= 3.8.0 and PostgreSQL) so the
database guarantees at most one non-revoked row per external identity.
- Reorder upsert_connection to revoke other owners' active rows before the new
connected row is flushed (so the index is satisfied at commit), wrapped in a
bounded rollback-and-retry loop. A losing concurrent writer now retries
against the now-visible state instead of committing a duplicate.
Adds DB-constraint, revoked-slot-reuse, and concurrent-upsert regression tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(channels): harden connect-status polling primitive
pollChannelConnectionUntilResolved was a free-floating recursive setTimeout
started from onSuccess with no cancellation, no per-provider dedup, a redundant
second endpoint per tick, and an unbounded loop on a non-finite expires_in.
- Extract a framework-agnostic, cancellable poller (connect-poll.ts) that polls
only listChannelConnections() and invalidates the providers query once when the
bind resolves, instead of fetching both endpoints every tick.
- Guard expires_in with a finite check + default window so undefined/NaN can no
longer produce a poll loop that runs until the page closes.
- Track one active poll handle per provider in useConnectChannelProvider via a
ref Map: a new connect cancels the prior poll for that provider, and a useEffect
cleanup cancels all polls on unmount.
Adds unit tests for resolve-and-stop, cancellation, and non-finite-expiry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(channels): stop leaking blocked-sender content in DingTalk INFO log; document bind semantics
Moving the allowed_users gate past _extract_text meant the parsed-message INFO
log (text=%r, first 100 chars) fired for senders that allowed_users would have
rejected, defeating the filter's noise/privacy role. Move that log to after the
allowed_users gate so blocked senders' message text never reaches INFO logs.
Also document the two operator-relevant semantic changes in backend/CLAUDE.md:
connect-code dispatch runs before allowed_users (so allowed_users is no longer a
bind-time defense; the model relies on code confidentiality + 600s TTL + one-time
consumption), and the single-active-owner-per-external-identity transfer semantics
now backed by the partial unique index.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(channels): note connect-code-vs-allowlist and ownership transfer in operator guide
Mirror the backend/CLAUDE.md notes in the operator-facing IM_CHANNEL_CONNECTIONS.md:
connect codes are consumed before allowed_users (so a not-yet-allowlisted user can
still complete a first bind, and allowed_users is not a bind-time defense), and an
external identity has at most one active owner with last-bind-wins transfer enforced
at the DB layer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(channels): lift connect-code dispatch into Channel base class
Each adapter duplicated the ordering-sensitive boilerplate of extracting a
/connect code and guarding on the connection repo before its allowed_users gate.
The duplication is what let telegram/wechat drift and keep the gate ahead of the
bind. Centralize it:
- Move `_connection_repo` onto Channel.__init__ (removing 7 duplicate assignments).
- Add Channel._pending_connect_code(text), which guards on the repo and extracts
the code, documenting that adapters MUST consult it before authorization so a
browser-initiated bind can bootstrap a not-yet-authorized identity.
- Route slack, discord, feishu, dingtalk, wechat, and wecom through the helper.
This also fixes a latent inconsistency where slack dispatched a bind even when
no connection repo was configured.
Pure refactor — the full channel suite stays green; adds a direct unit test for
the base helper's contract.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* make format
* fix(channels): redact DingTalk parsed-message INFO log content
Log text_len instead of the first 100 chars of message text, so message
content never reaches INFO logs (the after-gate move already keeps blocked
senders out entirely). This takes over the redaction from #3584 so only this
PR touches dingtalk.py, letting the two PRs merge in any order conflict-free.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(channels): add operational guardrails
* make format
* fix(channels): converge with #3582 to avoid merge-order conflicts
Drop this PR's DingTalk INFO-log redaction and hand it to #3582, which
already restructures that handler and will redact the same log there. This
PR no longer touches dingtalk.py, so the two PRs can merge to main in any
order without a conflict.
For WeChat, drop the contested thread_ts priority reorder (review #3) and
keep only what inbound dedupe needs: a server-stable message_id in the
inbound metadata (message_id/msg_id, no client_id per review #6). This is a
single added line inside the metadata dict, a region #3582 never touches, so
it auto-merges regardless of order.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(channels): address three correctness review findings
1. Connect-code cap was racy (willem #1): _create_state ran delete-expired,
count, and insert as three separate transactions, so concurrent connect
POSTs from one owner could each see count < cap and all insert past it. Add
ChannelConnectionRepository.create_oauth_state_within_cap which does
delete+count+insert in a single transaction serialized per (owner,
provider) — Postgres via pg_advisory_xact_lock, SQLite via the write lock
the leading DELETE takes — and have the router use it.
2. Inbound dedupe key fell back to "" workspace (willem #3): two workspaces
delivering without team/guild/aibotid would collapse to the same key and
dedupe each other's messages. _inbound_dedupe_key now fails closed
(returns None) when no workspace identifier is present.
3. Dedupe key was recorded on receipt and never released on failure
(ShenAC #1): a transient error (DB blip, Gateway 503) left the key in place
for the full TTL, so a provider redelivery of the same message_id — exactly
the retry dedupe should absorb — was silently dropped. _handle_message now
releases the key in the unexpected-exception branch so redelivery can
recover, while keeping record-on-receipt so retries during handling are
still deduped.
Tests: repo cap enforcement incl. concurrent-issuance non-leak; dedupe
fail-closed; dedupe key release-on-failure redelivery recovery.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(channels): address cleanup/efficiency and test review findings
Efficiency / cleanup:
- Dedupe key set drops client-generated ids (client_msg_id, client_id);
keep only server-stable event_id/message_id/msg_id, which a provider's own
redelivery preserves (ShenAC #6). Every provider already emits message_id.
- TTL/overflow pruning of _recent_inbound_events is now O(k): switch to an
OrderedDict and popitem(last=False) from the front instead of scanning all
4096 entries on every inbound (willem #4).
- Log "received inbound" only after the dedupe check so a provider retrying N
times no longer logs N accepts; document that manager dedupe covers the
agent run/final answer, not provider ack side-effects (willem #5, ShenAC #2).
- Slack drops the redundant `team_id or event.get("team")` fallback the caller
already resolved (willem #6).
- create_oauth_state_within_cap prunes only this owner/provider's expired codes
instead of a global DELETE on every connect POST; global cleanup still runs
on consume_oauth_state (willem #7).
Tests:
- Dedupe test uses tmp_path instead of a leaked mkdtemp, uses distinct objects
per publish, and adds a negative control: a different message_id is still
processed, catching over-dedupe regressions (willem #8, ShenAC #4).
- Slack HTTP-mode rejection test supplies app_token so the missing-token early
return can't mask the guard, giving the state assertions teeth (ShenAC #3).
- count_oauth_states test pins that the active row survives, not just the count
(ShenAC #5).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* make format
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(spec): telegram streaming output design
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(plan): telegram streaming implementation plan
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(telegram): report streaming support for telegram channel
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(channels): use slack as the non-streaming sample channel in manager tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(telegram): register running-reply placeholder as stream target
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(telegram): pin last_edit_at sentinel in placeholder registration test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(telegram): extract _send_new_message from send()
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(telegram): edit streamed message in place for non-final updates
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(telegram): finalize streamed message with overflow splitting
When is_final=True arrives and stream state exists, pop the state, edit
the streamed placeholder with the final text, split overflow into follow-up
send_message calls, update _last_bot_message, and clear stream state.
Falls back to _send_new_message when no stream state is registered.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(telegram): exercise the not-modified handler in final edit path
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: telegram channel now streams replies via message editing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(telegram): harden final-delivery path with guarded retry and chunk retries
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(channels): accept runtime 'messages' SSE event for streaming text accumulation
The embedded runtime (matching LangGraph Platform semantics) emits SSE
event name 'messages' for the requested 'messages-tuple' stream mode,
so the manager never accumulated token deltas and streaming channels
only updated from end-of-step 'values' snapshots — on Telegram this
looked like 'Working on it...' followed by the full answer in one block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(telegram): widen stream-edit throttle to 3s in group chats
Telegram caps bots at 20 messages/minute per group, stricter than the
1 msg/s per-chat guideline. Groups have negative chat ids, so pick the
interval by sign.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(telegram): address review findings — thread fallback messages, bound stream registry, share stream-event constants
- Fallback/new stream messages now carry reply_to_message_id parsed from
thread_ts so they stay nested under the user's message (finding 1)
- STREAM_MODES / MESSAGE_STREAM_EVENTS constants link the requested
stream modes to the SSE event names they arrive under (finding 2)
- _register_stream_message bounds the in-flight registry at 256 entries,
evicting oldest, guarding against leaks when a final never arrives (finding 4)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add user-owned IM channel connections
* Fix dev startup and channel connect popup
* Use async channel connect flow
* Harden dev service daemon startup
* Support local IM channel connections
* Align IM connections with local channels
* Fix safe user id digest algorithm
* Address Copilot IM channel feedback
* Address IM channel review comments
* Support all integrated IM channel connections
* Format additional channel connection tests
* Keep unavailable channel connect buttons clickable
* Fix IM channel provider icons
* Add runtime setup for enabled IM channels
* Guard global shortcut key handling
* Keep configured IM channels editable
* Avoid password autofill for channel secrets
* Make channel threads visible to connection owners
* Persist IM runtime config locally
* Allow disconnecting runtime IM channels
* Route no-auth channel sessions to local user
* Use default user for auth-disabled local mode
* Show IM channel source on threads
* Prefill IM channel runtime config
* Reflect IM channel runtime health
* Ignore Feishu message read events
* Ignore Feishu non-content message events
* Let setup wizard enable IM channels
* Fix frontend formatting after merge
* Stabilize backend tests without local config
* Isolate channel runtime config tests
* Address channel connection review comments
* Use sha256 user buckets with legacy migration
* Ensure runtime IM channels are ready after restart
* Persist disconnected IM channel state
* Address channel connection review comments
* Address channel connection review findings
Frontend connect flow:
- Open the runtime-config dialog only when a provider still needs
credentials; configured providers go straight to the connect flow, so
the binding-code/deep-link path is reachable from the UI again.
- After saving credentials, continue into the connect flow when a user
binding is still required (multi-user mode) instead of stopping at a
"Connected" toast.
- Extract shared provider-state helpers to core/channels/provider-state
and add unit + e2e coverage for the direct-connect and
configure-then-connect paths.
Provider status semantics:
- Report connection_status from the user's newest connection row;
with no binding it is not_connected, except in auth-disabled local
mode where a configured running channel is effectively connected.
Concurrency and event-loop correctness:
- Offload ChannelRuntimeConfigStore construction and writes, channel
service construction, and Slack connection replies to threads; add a
tests/blocking_io/ anchor for the runtime-config handlers.
- Consume binding codes with a conditional UPDATE so a code can only be
used once under concurrent workers; retry upsert_connection as an
update when a concurrent insert wins the unique constraint.
- Serialize ensure_channel_ready per channel so concurrent provider
polls cannot double-start a channel worker.
Config and migration hardening:
- Stop mutating the get_app_config()-cached Telegram provider config;
the runtime store now owns the UI-entered bot username.
- Register channel_connections in STARTUP_ONLY_FIELDS with the
standardized startup-only Field description.
- Match the legacy unsafe-id bucket by recomputing its exact SHA-1 name
so another user's same-prefix bucket can never be migrated.
- Remove the unused Telegram process_webhook_update path and document
src/core/channels in the frontend docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address PR review comments on authz scoping and channel runtime
Security (review feedback from ShenAC-SAC):
- Scope internal-token callers to the connection owner carried in
X-DeerFlow-Owner-User-Id instead of bypassing owner checks outright,
in both require_permission(owner_check=True) and the stateless run
endpoints. Internal callers keep access to their own and
shared/legacy threads, and may claim a default-owned channel thread
for its real owner, but a leaked internal token no longer grants
cross-user thread access.
- Require admin privileges for POST/DELETE /api/channels/{provider}/
runtime-config: runtime credentials and channel workers are
instance-wide shared state (same model as the MCP config API).
Read-only provider listing stays available to all users.
Performance (review feedback from willem-bd):
- Skip the redundant thread channel-metadata PATCH after the first
successful backfill per thread.
- Reuse the per-connection Slack WebClient until its token changes
instead of constructing one per outbound message.
- Reconcile channel readiness for all providers concurrently in
GET /api/channels/providers.
Also resolve the code-quality unused-import flag in the blocking-io
anchor by pre-importing the channel service via importlib.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix prettier formatting in provider-state test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Reconcile UI runtime channel config with config reload on restart
Main now reloads a channel's config.yaml entry on restart_channel()
(#3514, issue #3497). Adapt the user-owned connection flow to coexist:
- configure_channel() restarts with reload_config=False — the caller
just supplied the authoritative config (browser-entered credentials
that are never written to config.yaml), so a file reload must not
clobber it with the stale on-disk entry.
- _load_channel_config() re-applies the UI runtime-store overlay used
at startup, so an operator-triggered restart keeps browser-entered
credentials for channels without a config.yaml entry and does not
resurrect a channel disconnected from the UI.
- Offload the reload's disk IO (config.yaml + runtime store) with
asyncio.to_thread, matching the blocking-IO policy on this branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* refactor: extract shared utils to break harness→app cross-layer imports
Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: split backend/src into harness (deerflow.*) and app (app.*)
Physically split the monolithic backend/src/ package into two layers:
- **Harness** (`packages/harness/deerflow/`): publishable agent framework
package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
models, MCP, skills, config, and all core infrastructure.
- **App** (`app/`): unpublished application code with import prefix `app.*`.
Contains gateway (FastAPI REST API) and channels (IM integrations).
Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure
Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: add harness→app boundary check test and update docs
Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.
Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add config versioning with auto-upgrade on startup
When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.
- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix comments
* fix: update src.* import in test_sandbox_tools_security to deerflow.*
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: handle empty config and search parent dirs for config.example.yaml
Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
looking next to config.yaml, fixing detection in common setups
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: correct skills root path depth and config_version type coercion
- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
after harness split, file lives at packages/harness/deerflow/skills/
so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
_check_config_version() to prevent TypeError when YAML stores value
as string (e.g. config_version: "1")
- tests: add regression tests for both fixes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update test imports from src.* to deerflow.*/app.* after harness refactor
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
