mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-11-20 11:20:35 +00:00
* feat(goal): add /goal from-chat for IM channels First slice of #7050: add an explicit IM command that composes the active goal from the recent conversation instead of taking the text literally. Natural-language intent detection remains out of scope. Gateway: - POST /api/threads/{thread_id}/goal/from-chat synthesizes the objective from the thread summary and recent agreement turns (visible user/assistant text, clarification questions, and card answers), preserving agreed constraints and the stopping condition. - Reject a busy thread before the model call; the save reuses the existing checkpoint-write reservation so a run that starts mid-synthesis still wins. - Refuse to synthesize while a human-input request is unanswered, and return structured no_context/unclear/synthesis_failed outcomes without echoing model-generated reasons. IM: - /goal from-chat [hint] shows the saved objective, then starts exactly one run through the ordinary chat path; a longer read timeout covers synthesis. - Map busy/pending/unclear/no-context/timeout outcomes to localized replies; a saved goal whose run cannot start is reported honestly. - The TUI explains that from-chat is not supported yet; the web input box still treats it literally (documented follow-up). Docs: README, CHANGELOG, IM channel guide, TUI guide, AGENTS, and the acceptance plan/execution record. Tests: new route, synthesis and manager suites, plus parser and TUI coverage. * fix(goal): keep the from-chat confirmation visible on every channel The saved-objective reply was published as a non-final progress update. QQ and DingTalk card mode drop non-final outbounds before a stream exists, and Feishu/Telegram fold the update into the run's stream, so the run output overwrote the objective. Publish it as the turn's final message. Redact PII on whole lines and the summary before the context budget can split an identifier; the composed prompt is still redacted once more as defense in depth, matching the goal evaluator's direct call. Also correct the gateway/IM docs (from-chat never creates a checkpoint, the busy pre-check is worker-local, and synthesis uses the deployment default model), fix the route test clock patch, and add regression tests for the final-outbound and truncation-boundary cases. * fix(goal): align from-chat with the goal-route admission gates Reject a from-chat save while a tool approval is pending, matching PUT/DELETE /goal, and suppress subagent-batch continuations queued under the previous goal after the save. Both invariants landed upstream while this branch was in review; the new route now shares them. Add route tests for both behaviors. * docs(changelog): remove the from-chat entry to avoid conflicts The maintainer asked to drop the changelog entry because the section conflicts with upstream changes. The feature remains documented in the goals guide and README. * fix(goal): close three review gaps in /goal from-chat - Enforce model-use authorization for the deployment default model before the synthesis call, matching the suggestions route, so a denied role never sends conversation context to the model. - Save with expected_checkpoint_id pinned to the head the synthesis read; a write that landed during the model call is a 409 goal_changed instead of silently clobbering the newer goal. - Re-check the conversation binding before starting the run: a /new during synthesis keeps the goal in the original thread and reports that the task did not start instead of running it in the switched conversation. * fix(goal): preserve from-chat handoff and run status Pin the saved thread through run preparation and reject a changed binding. Read configuration and model authorization off the event loop, and report rejected starts separately from requests whose response was lost. Cover the handoff races and streaming/wait admission outcomes through the real channel manager and SDK transport, plus file-backed config under the strict blocking-I/O detector. Co-authored-by: Loveca <44801706+Loveca@users.noreply.github.com> * fix(goal): preserve reasoning tags in synthesis JSON Use the existing JSON-aware reasoning stripper so literal tag text in objective and reason values survives parsing. Cover complete and unclosed tags while still removing model reasoning before the JSON payload. Co-authored-by: Loveca <44801706+Loveca@users.noreply.github.com> --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Totoro <msy626836554@gmail.com>
96 lines
5.3 KiB
Markdown
96 lines
5.3 KiB
Markdown
# IM channels (`app/channels/`)
|
|
|
|
Each provider module adapts one chat platform to the shared `MessageBus` and
|
|
`ChannelManager`. Channels are Gateway clients: they create threads and runs through
|
|
the LangGraph-compatible HTTP API with `langgraph-sdk`, and the Gateway owns the
|
|
thread lifecycle. Provider setup, flow diagrams and operator notes are in
|
|
[IM_CHANNEL_CONNECTIONS.md](../../docs/IM_CHANNEL_CONNECTIONS.md) and
|
|
[GITHUB_AGENTS.md](../../docs/GITHUB_AGENTS.md).
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
cd backend
|
|
python -m pytest tests -q -k "channel or <provider>" # e.g. telegram, feishu, wecom, qq
|
|
make test-blocking-io # channels share the Gateway event loop
|
|
```
|
|
|
|
## Where the code misleads
|
|
|
|
- **Shared event loop.** All channels run on the Gateway event loop, which also
|
|
serves HTTP requests. Run blocking SDK calls, file I/O and SDK teardown through
|
|
`asyncio.to_thread`. When a provider SDK calls back from its own thread, submit
|
|
work with `Channel._submit_threadsafe_coroutine()` so that the real task is
|
|
retained and drained on `stop()`.
|
|
- **Stream text is an allowlist.** Middlewares inject hidden `HumanMessage`s
|
|
(recalled memory, rewritten user turns) that also arrive on the `messages-tuple`
|
|
stream. Publish only assistant message types to the platform. A denylist leaks
|
|
hidden model context to chat users.
|
|
- **Several Gateway workers.** A leader lease decides which worker starts each
|
|
transport, and thread bindings live in SQL so that replicas agree. State that
|
|
you keep in process memory (caches, buffers, watchers) is invisible to the other
|
|
workers; say so in the code when you add it. GitHub has no lease because it is
|
|
webhook-driven.
|
|
- **Stop can time out.** `ChannelService` keeps a channel instance whose `stop()`
|
|
timed out and retries it later. A new provider must make `stop()` safe to call
|
|
again and must release SDK resources only after its own tasks have drained.
|
|
- **Final reply before dedupe release.** Publish the final outbound message before
|
|
you release the inbound dedupe key. A provider redelivery can otherwise overtake
|
|
the reply.
|
|
- **Goal handoff pins the saved thread.** Recheck the conversation binding after
|
|
confirmation delivery and run preparation. Use run-admission evidence when
|
|
reporting a rejected start or a response lost after submission.
|
|
- **Telegram photo fallback is narrow.** Only a photo `BadRequest` with
|
|
`PHOTO_INVALID_DIMENSIONS` or `IMAGE_PROCESS_FAILED` is retried, once, as a
|
|
document with the same bytes, file name and reply target. Other errors and
|
|
cancellation keep their behavior. Count only the delivery that succeeds.
|
|
- **Docker networking.** Channels run inside the `gateway` container, so
|
|
`localhost` in `channels.langgraph_url` / `channels.gateway_url` points at that
|
|
container. Compose deployments use `http://gateway:8001`.
|
|
|
|
## Identity and security
|
|
|
|
- **Owner versus sender.** For a user-owned connection, the DeerFlow owner becomes
|
|
the run `user_id` and the platform sender stays `channel_user_id`. Put
|
|
`channel_user_id` in the runtime context only. `configurable` is checkpointed
|
|
and would persist one sender's identity for the whole group thread.
|
|
- **One storage owner per message.** Resolve the storage owner once in the manager
|
|
and pass it as `user_id=` through every file helper. Uploads and outbound
|
|
artifacts then use the same bucket as the agent run. Resolve outbound artifacts
|
|
through `app.gateway.path_utils.resolve_outputs_confined_path`, which rejects
|
|
paths and symlinks that leave `outputs/`.
|
|
- **Connect codes run before allowlists.** `/connect <code>` (Telegram
|
|
`/start <code>`) is handled before `allowed_users` on purpose, so that a new user
|
|
can bind. The code is the secret: show it only in the browser that requested it
|
|
and never echo it to the chat.
|
|
- **Feishu post commands use the caption.** Add placeholders for top-level post
|
|
attachments after command detection. For commands that enter the chat path,
|
|
`receive_file` adds their placeholders after the manager parses the arguments.
|
|
- **One active owner per external identity.** A partial unique index in the
|
|
database enforces this. Application code alone cannot, because two workers can
|
|
bind at the same time.
|
|
- **Inbound media.** Fetch provider media only from the URL and hosts that the
|
|
channel validated, enforce the size limit while streaming, and pass the bytes to
|
|
the shared upload pipeline. Bot API file URLs contain the bot token; keep them
|
|
out of messages, metadata and logs.
|
|
- **Logs.** Log identifiers and error names. Keep credentials, message content and
|
|
provider response bodies out of the logs.
|
|
|
|
## GitHub channel
|
|
|
|
- The webhook route skips session auth and CSRF because the HMAC signature
|
|
authenticates it. It is mounted only when `GITHUB_WEBHOOK_SECRET` is set.
|
|
- Outbound is log-only by design. Each agent posts its own comment with `gh` from
|
|
its sandbox, so the manager starts the run and does not wait for it.
|
|
- Follow-up buffering while a thread is busy is process-local. Read
|
|
[the contract](../../docs/GITHUB_AGENTS.md#follow-up-buffering-while-busy-4121)
|
|
before you change it.
|
|
|
|
## Adding a provider
|
|
|
|
Keep the service, provider, credential and config registrations aligned in the
|
|
same change. Buzz
|
|
needs the `buzz` dependency extra, and its relay delivers chat events only to
|
|
channel-scoped subscriptions
|
|
([details](../../docs/IM_CHANNEL_CONNECTIONS.md#buzz-subscription-model)).
|