mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-11-20 19:50:35 +00:00
* feat(goal): add /goal from-chat for IM channels First slice of #7050: add an explicit IM command that composes the active goal from the recent conversation instead of taking the text literally. Natural-language intent detection remains out of scope. Gateway: - POST /api/threads/{thread_id}/goal/from-chat synthesizes the objective from the thread summary and recent agreement turns (visible user/assistant text, clarification questions, and card answers), preserving agreed constraints and the stopping condition. - Reject a busy thread before the model call; the save reuses the existing checkpoint-write reservation so a run that starts mid-synthesis still wins. - Refuse to synthesize while a human-input request is unanswered, and return structured no_context/unclear/synthesis_failed outcomes without echoing model-generated reasons. IM: - /goal from-chat [hint] shows the saved objective, then starts exactly one run through the ordinary chat path; a longer read timeout covers synthesis. - Map busy/pending/unclear/no-context/timeout outcomes to localized replies; a saved goal whose run cannot start is reported honestly. - The TUI explains that from-chat is not supported yet; the web input box still treats it literally (documented follow-up). Docs: README, CHANGELOG, IM channel guide, TUI guide, AGENTS, and the acceptance plan/execution record. Tests: new route, synthesis and manager suites, plus parser and TUI coverage. * fix(goal): keep the from-chat confirmation visible on every channel The saved-objective reply was published as a non-final progress update. QQ and DingTalk card mode drop non-final outbounds before a stream exists, and Feishu/Telegram fold the update into the run's stream, so the run output overwrote the objective. Publish it as the turn's final message. Redact PII on whole lines and the summary before the context budget can split an identifier; the composed prompt is still redacted once more as defense in depth, matching the goal evaluator's direct call. Also correct the gateway/IM docs (from-chat never creates a checkpoint, the busy pre-check is worker-local, and synthesis uses the deployment default model), fix the route test clock patch, and add regression tests for the final-outbound and truncation-boundary cases. * fix(goal): align from-chat with the goal-route admission gates Reject a from-chat save while a tool approval is pending, matching PUT/DELETE /goal, and suppress subagent-batch continuations queued under the previous goal after the save. Both invariants landed upstream while this branch was in review; the new route now shares them. Add route tests for both behaviors. * docs(changelog): remove the from-chat entry to avoid conflicts The maintainer asked to drop the changelog entry because the section conflicts with upstream changes. The feature remains documented in the goals guide and README. * fix(goal): close three review gaps in /goal from-chat - Enforce model-use authorization for the deployment default model before the synthesis call, matching the suggestions route, so a denied role never sends conversation context to the model. - Save with expected_checkpoint_id pinned to the head the synthesis read; a write that landed during the model call is a 409 goal_changed instead of silently clobbering the newer goal. - Re-check the conversation binding before starting the run: a /new during synthesis keeps the goal in the original thread and reports that the task did not start instead of running it in the switched conversation. * fix(goal): preserve from-chat handoff and run status Pin the saved thread through run preparation and reject a changed binding. Read configuration and model authorization off the event loop, and report rejected starts separately from requests whose response was lost. Cover the handoff races and streaming/wait admission outcomes through the real channel manager and SDK transport, plus file-backed config under the strict blocking-I/O detector. Co-authored-by: Loveca <44801706+Loveca@users.noreply.github.com> * fix(goal): preserve reasoning tags in synthesis JSON Use the existing JSON-aware reasoning stripper so literal tag text in objective and reason values survives parsing. Cover complete and unclosed tags while still removing model reasoning before the JSON payload. Co-authored-by: Loveca <44801706+Loveca@users.noreply.github.com> --------- Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com> Co-authored-by: Totoro <msy626836554@gmail.com>
4.9 KiB
4.9 KiB
Runtime (deerflow/runtime/)
Executes runs: admission, the run worker, the stream bridge (memory and Redis), checkpoint access, the run event store and journal, and the server-installed capabilities of a run. Background: runtime details, run event stream, streaming, checkpoint retention.
Verify
cd backend
python -m pytest tests -q -k "run or stream or checkpoint or event_store or journal"
make test-blocking-io
Streams and reconnect
- A replay gap is not the end of a run. When the retained window no longer covers a reconnect cursor, the bridge reports a gap and the reader reloads state.
- Reconnect cursors are client input. Parse them strictly and answer a malformed one as bad input.
- Redis END publication is one Lua script. The producer-token check, the END detection and the append happen atomically. Restart all Gateways together when the stream mode changes, because old and new producers cannot share a stream.
- Subscribers get their own copy. The memory bridge copies events outside its condition lock, and caches copy on ingest and on read, so one consumer cannot change what another sees.
- Keep registry locks short. Heartbeats take the same locks and must not queue behind slow work.
- Subgraph frames keep their namespace. A bare
valuesframe from a subgraph would overwrite the thread view of an SDK client. Consumers that only handle the root run ignore namespaced frames.
Checkpoints
- The channel mode is fixed for the life of the process. A reader in
fullmode rejects adeltacheckpoint; it does not guess. - Read and change state through
CheckpointStateAccessorwith the thread's effective schema. It preserves mode gates, parentage and pending writes. - Delta resume and rollback rewrite the current head. They do not fork. Reject a missing tuple or pending sends before the first write.
Admission and terminal commit
- Only the worker that owns the run commits its terminal state. A lost or expired lease fences later writes, so a recovered run is not overwritten.
- Register completion before cleanup awaits. Cancellation during cleanup is then safe, and an idempotent request gets the stored result replayed.
- Terminal cleanup always finishes. Streams, journals and graph references are closed even when the run is cancelled.
- A provider tool-call ID is not an owner. Providers reuse IDs across parent runs. The server-generated execution ID owns the registry entry, polling, cancellation and cleanup; the provider ID only correlates what the user sees.
Run events and journal
RunJournalis changed on its owning event loop only. A recorder failure fails open, and a slow recorder must not delay tool state.- Usage replay may enrich a message, never add or remove one. A provider can report the end of the same model call twice.
- Event-store mutations of one thread share one fence (an asyncio lock plus a PostgreSQL lock). The fence serializes writers. It does not tell an old run incarnation from a new one.
- The JSONL store is single-process. It splits on LF only, because Unicode line separators inside a field are data. A started mutation drains through cancellation. The sequence watermark is saved before a run is deleted, and a corrupt watermark fails closed.
- The in-memory store copies caller-owned content at admission.
- Event reads pass the trusted user identity, and the owner on a
RunRecordmatches the durable row. - An evidence receipt needs the worker's own final acknowledgement. An unknown or cancelled write outcome yields no receipt.
Goals
- Goal synthesis is read-only.
runtime/goal_synthesis.pycomposes the/goal from-chatobjective from the recent agreement; the Gateway route owns the save. It treats the conversation as data, applies the evaluator's PII redaction and oneshot-LLM path, excludes evidence tool results, keeps the newest turn within its own 12,000-character budget even when a single paste exceeds it, and caps the compaction summary at half so recent constraints retain room.
Server-installed capabilities
- Capabilities exist for one request. The scheduler capability, the conversation reader and client capability hints are installed by the server. A copy that a caller sends is rejected in both context carriers, and none is restored from a checkpoint.
- Run metadata and hidden messages are display data. They grant nothing.
- A scheduled goal is installed in a fresh thread only and is removed before the terminal status commits.
- Retry the goal evaluator only for a malformed verdict from a normal model completion. Other failures are final.
- A truncated workspace snapshot cannot prove that a file is absent.