DanielWalnut ef5f54c5bf
feat(tui): Hermes-like terminal workbench (deerflow) backed by DeerFlowClient (#3760)
* feat(tui): add Hermes-like terminal workbench backed by DeerFlowClient

Implements the `deerflow` TUI from RFC #3540: a terminal-native, embedded
workbench over the existing harness (no Gateway/frontend/nginx/Docker), built
Python-native with Textual and learning UX patterns from tao-pi.

Architecture — every layer except the Textual app is pure and unit-tested:
- view_state.py: ViewState + reduce(state, action), the testable heart
- runtime.py: StreamEvent -> reducer actions (pure translate + threaded driver)
- message_format / command_registry / input_history / render / theme: pure
- app.py: Textual App; runs the sync DeerFlowClient.stream() on a worker thread
  and marshals actions back to the UI thread. Slash command palette, model and
  thread modal pickers, ↑/↓ history, Ctrl+C interrupt, TTY-aware fallback.
- cli.py: pure launch-mode planning + headless --print/--json + `deerflow`
  console script (textual is an optional [tui] extra; degrades to headless help)

Web UI visibility (the RFC's key decision): persistence.py writes a threads_meta
row under the local default user into the same DB the Gateway reads, so terminal
sessions appear in the Web UI sidebar without running the Gateway. Best-effort,
no-op on the memory backend; all DB work on one long-lived background loop.

Tests: 95 TUI tests — pure layers via pytest, app/palette/overlays via Textual's
pilot harness with a fake session, and a threads_meta read/write round-trip.
ruff clean; respects the harness->app import boundary. Docs: backend/docs/TUI.md
plus CLAUDE.md/README updates and preview screenshots.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tui): de-duplicate streamed assistant text and tool cards; keep Tab in composer

Self-test surfaced three issues, all root-caused to consuming non-strict
streaming from DeerFlowClient (proven by the client's own
test_dedup_requires_messages_before_values_invariant, which shows the client can
re-emit a message id's full content twice):

- Assistant text was doubled (e.g. "answer answer") because the reducer blindly
  concatenated same-id deltas. Now merges by content: a re-send or cumulative
  snapshot replaces; only genuine increments append.
- Tool activity showed duplicate and empty "gear" cards from partial/re-emitted
  tool-call chunks. ToolStarted now de-dupes by tool_call_id, drops id-less
  noise chunks, and fills the name on a later chunk; a tool result with no prior
  card still surfaces as a completed card.
- Tab moved focus off the composer to the scroll region (felt like broken cursor
  logic). Tab is now consumed by the composer (completes a command when the
  palette is open, no-op otherwise).

Adds reducer tests for each case plus a Tab-focus test; 102 TUI tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tui): make Esc interrupt an active run (matches the status hint)

The status line advertised "esc interrupt" but Esc was only wired to close the
slash palette, so it did nothing during a run. Esc now: closes the palette when
open, interrupts the active run when streaming, and is a no-op when idle. The
interrupt logic is shared with Ctrl+C via _interrupt_run(). Adds a regression
test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tui): stop prior answers duplicating on threads with history

On a thread with history, DeerFlowClient re-emits every prior message on each
new turn (its streamed_ids dedup is per-stream-call), and a re-emitted older
message can arrive after a newer message has already started. The reducer only
matched the *most recent* assistant row by id and otherwise appended, so each
re-emitted older answer was duplicated verbatim at the end of the transcript.

Match an assistant row by id anywhere in the transcript and merge in place.
Tool cards already de-dupe by call id globally, so they were unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tui): correct CJK cursor drift in the composer

Confirmed a Textual Input bug (latest 8.2.7): Input._cursor_offset adds an
unconditional +1 at the end of the value, overshooting by one cell after
double-width (CJK) characters. That misplaces the hardware/IME cursor — the
drift seen when typing Chinese in iTerm2 (the on-screen block cursor, drawn
separately in render_line, is fine; English doesn't use an IME so it looks
correct). Reproduced with a bare Input, so it's upstream, not our layout.

Add ComposerInput(Input) overriding _cursor_offset to the true cell position and
use it for the composer. Numeric tests pin the CJK end/mid and ASCII cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(tui): render finalized assistant messages as Markdown

The transcript showed raw Markdown (literal **bold**, ## headings, - lists,
links). Finalized assistant messages now render as Rich Markdown — headings,
bold/italic, lists, inline code + code blocks, blockquotes, horizontal rules and
links — with the ● speaker marker aligned to the top of the body.

The actively-streaming message stays plain text so partial Markdown doesn't
reflow/jump, then snaps to its rendered form when the run ends. Transcript
re-renders are coalesced on a ~60ms timer (dirty flag) so per-token Markdown
re-parsing stays smooth on long threads. Tests cover both the rendered and the
streaming-plain paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* style(tui): apply ruff format

CI lint runs `ruff format --check` via uvx (latest ruff); apply the formatter so
the lint-backend job passes. No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(tui): address code-quality review comments

From github-code-quality[bot] on #3760:
- runtime.py: give the `_ClientLike` Protocol method a docstring body instead of
  a bare `...` (flagged as a no-effect statement), matching the harness
  convention for Protocol stubs (e.g. SafetyTerminationDetector).
- test_tui_cli_main.py: drop the unnecessary `lambda: _FakeSession()` wrappers in
  monkeypatch.setattr; pass `_FakeSession` directly (same behavior).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tui): keep history Markdown-rendered when a follow-up run starts

Previously the transcript rendered "the last assistant row" as plain text while
streaming. But when a follow-up turn starts, the last assistant row is the
*previous, finalized* answer until the new message begins — and the client
re-emits prior messages early in the turn — so sending a follow-up reverted the
previous answer from rendered Markdown back to raw text.

Track the actively-streaming message id in ViewState instead: it's reset on
RunStarted, set only when an AssistantDelta actually adds new content (history
re-emits are no-ops and don't mark it), and cleared on RunEnded. The renderer
keeps only that one message plain; all history stays Markdown.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(readme): add Terminal Workbench (TUI) section to root README

Mention the new `deerflow` TUI alongside the Embedded Python Client in the root
README.md and README_zh.md (install, launch/headless commands, feature summary,
Web UI visibility), with a ToC entry and a preview screenshot. Links to
backend/docs/TUI.md for the full guide.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tui): address review feedback (willem-bd)

Ten findings from the TUI code review:

1. /resume was dead-ended — registered + in /help + tested as a builtin, but no
   dispatch branch. Wired it to thread resolution / the switcher.
2. --resume <title> was forwarded raw into the checkpointer (blank thread).
   Added Session.resolve_ref() to resolve id-or-title via list_threads; used by
   --resume and /resume.
3. str(get("id","")) returned "None" for an explicit id:None (truthy), defeating
   the empty-id guard so unrelated null-id tool calls collapsed into one card.
   Coerce via a None-safe helper.
4. Headless --print/--json no longer spin up the persistence loop/engine/pool
   (open_session(persistence=False)).
5. _LoopThread + engine are now closed: Session.close() (dispose engine + stop
   loop) called from a try/finally around app.run().
6. --cli --continue (and piped --cli) now run headless instead of erroring.
7. Cancelled runs no longer persist a truncated title (guard on _cancelled).
8. Palette highlight resets to the top when the filter set changes.
9. Dropped the never-populated tools count from the header.
10. Documented the `not row.error` merge guard.

Adds regression tests for each; 126 TUI tests pass, ruff check + format clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-06-25 20:10:49 +08:00

307 lines
9.6 KiB
Python

"""Pure view-state reducer for the DeerFlow TUI.
This module has **no** Textual / rendering dependency. It models the visible
conversation as an immutable list of typed rows and a small set of actions,
and exposes a single pure ``reduce(state, action) -> state`` function.
Keeping this layer pure makes the interesting behaviour (streaming deltas,
tool cards, error rows) testable with plain ``pytest`` and a handful of
synthetic actions, independent of any terminal.
The runtime bridge (``deerflow.tui.runtime``) is responsible for translating
``DeerFlowClient`` ``StreamEvent`` objects into these actions; the Textual app
renders ``ViewState`` into widgets. Both sides depend on this module, not on
each other.
"""
from __future__ import annotations
from dataclasses import dataclass, field, replace
from typing import Literal
from .message_format import format_tool_detail, format_tool_result, summarize_tool_title
# --------------------------------------------------------------------------- #
# Rows — the immutable units the transcript is built from.
# --------------------------------------------------------------------------- #
@dataclass(frozen=True)
class UserRow:
text: str
kind: Literal["user"] = "user"
@dataclass(frozen=True)
class AssistantRow:
text: str
id: str | None = None
error: bool = False
kind: Literal["assistant"] = "assistant"
@dataclass(frozen=True)
class ToolRow:
tool_call_id: str
tool_name: str
title: str
detail: str = ""
result: str = ""
status: Literal["running", "ok", "error"] = "running"
kind: Literal["tool"] = "tool"
@dataclass(frozen=True)
class SystemRow:
text: str
tone: Literal["info", "error"] = "info"
kind: Literal["system"] = "system"
Row = UserRow | AssistantRow | ToolRow | SystemRow
# --------------------------------------------------------------------------- #
# Actions — the only ways the state can change.
# --------------------------------------------------------------------------- #
@dataclass(frozen=True)
class UserSubmitted:
text: str
@dataclass(frozen=True)
class RunStarted:
pass
@dataclass(frozen=True)
class RunEnded:
usage: dict | None = None
@dataclass(frozen=True)
class AssistantDelta:
id: str
text: str
@dataclass(frozen=True)
class AssistantError:
text: str
@dataclass(frozen=True)
class ToolStarted:
tool_call_id: str
tool_name: str
args: dict = field(default_factory=dict)
@dataclass(frozen=True)
class ToolResult:
tool_call_id: str
content: str
is_error: bool = False
tool_name: str = ""
@dataclass(frozen=True)
class SystemMessage:
text: str
tone: Literal["info", "error"] = "info"
@dataclass(frozen=True)
class ThreadTitle:
title: str
@dataclass(frozen=True)
class ClearRows:
pass
Action = UserSubmitted | RunStarted | RunEnded | AssistantDelta | AssistantError | ToolStarted | ToolResult | SystemMessage | ThreadTitle | ClearRows
# --------------------------------------------------------------------------- #
# State.
# --------------------------------------------------------------------------- #
@dataclass(frozen=True)
class ViewState:
rows: tuple[Row, ...] = ()
streaming: bool = False
usage: dict | None = None
title: str | None = None
# Id of the message currently being generated this turn. Only this row renders
# as plain text while streaming; everything else (history) stays Markdown.
streaming_id: str | None = None
def initial_state(rows: tuple[Row, ...] = ()) -> ViewState:
return ViewState(rows=tuple(rows))
# --------------------------------------------------------------------------- #
# Reducer.
# --------------------------------------------------------------------------- #
def _append(state: ViewState, row: Row) -> ViewState:
return replace(state, rows=state.rows + (row,))
def reduce(state: ViewState, action: Action) -> ViewState:
"""Return a new ``ViewState`` after applying ``action``. Pure."""
if isinstance(action, UserSubmitted):
return _append(state, UserRow(text=action.text))
if isinstance(action, RunStarted):
# New turn: no message is actively streaming yet (the client re-emits
# prior messages first; those must not be treated as the active one).
return replace(state, streaming=True, streaming_id=None)
if isinstance(action, RunEnded):
return replace(
state,
streaming=False,
streaming_id=None,
usage=action.usage if action.usage is not None else state.usage,
)
if isinstance(action, AssistantDelta):
return _apply_assistant_delta(state, action)
if isinstance(action, AssistantError):
return _append(state, AssistantRow(text=action.text, error=True))
if isinstance(action, ToolStarted):
return _apply_tool_started(state, action)
if isinstance(action, ToolResult):
return _apply_tool_result(state, action)
if isinstance(action, SystemMessage):
return _append(state, SystemRow(text=action.text, tone=action.tone))
if isinstance(action, ThreadTitle):
return replace(state, title=action.title)
if isinstance(action, ClearRows):
return replace(state, rows=(), title=None, streaming_id=None)
return state
def _apply_assistant_delta(state: ViewState, action: AssistantDelta) -> ViewState:
"""Update the assistant row with this id (anywhere in the transcript), or
start a new one.
On a thread with history, the client re-emits every prior message on each
new turn (its dedup is per-turn), and a re-emitted *older* message can arrive
after a newer one has started — so we must match by id across the whole
transcript, not just the most recent assistant row, or prior answers get
duplicated.
Updates also merge by content rather than blindly concatenating, to absorb
full re-sends / cumulative snapshots vs. genuine incremental deltas:
* new text == accumulated, or starts with it -> cumulative/re-send: replace
* accumulated starts with new text -> stale/shorter re-send: keep
* otherwise -> a real delta: append
"""
rows = list(state.rows)
for i, row in enumerate(rows):
# ``not row.error``: error rows are appended without an id, so they never
# match here anyway — the guard is belt-and-suspenders to keep an error
# row from being merged into if a future change ever gives it an id.
if isinstance(row, AssistantRow) and row.id == action.id and not row.error:
merged = _merge_stream_text(row.text, action.text)
if merged == row.text:
# No-op re-send (e.g. a values snapshot re-emitting history) —
# don't mark this as the actively-streaming message.
return state
rows[i] = replace(row, text=merged)
return _mark_streaming(replace(state, rows=tuple(rows)), action.id)
return _mark_streaming(_append(state, AssistantRow(text=action.text, id=action.id)), action.id)
def _mark_streaming(state: ViewState, message_id: str) -> ViewState:
"""Record the actively-streaming message id (only while a run is active)."""
if state.streaming:
return replace(state, streaming_id=message_id)
return state
def _merge_stream_text(existing: str, incoming: str) -> str:
if not existing:
return incoming
if incoming.startswith(existing):
return incoming # cumulative snapshot or exact full re-send
if existing.startswith(incoming):
return existing # shorter/stale re-send
return existing + incoming # genuine incremental delta
def _apply_tool_started(state: ViewState, action: ToolStarted) -> ViewState:
"""Create or update a tool card, de-duplicated by ``tool_call_id``.
Streaming tool calls arrive as multiple chunks for one call id (name first,
then growing arguments), and the client may re-emit the call via a values
snapshot. Chunks with no id are argument-fragment noise and are dropped.
"""
if not action.tool_call_id:
return state
rows = list(state.rows)
for i, row in enumerate(rows):
if isinstance(row, ToolRow) and row.tool_call_id == action.tool_call_id:
name = action.tool_name or row.tool_name
detail = format_tool_detail(name, action.args) or row.detail
rows[i] = replace(row, tool_name=name, title=summarize_tool_title(name), detail=detail)
return replace(state, rows=tuple(rows))
return _append(
state,
ToolRow(
tool_call_id=action.tool_call_id,
tool_name=action.tool_name,
title=summarize_tool_title(action.tool_name),
detail=format_tool_detail(action.tool_name, action.args),
status="running",
),
)
def _apply_tool_result(state: ViewState, action: ToolResult) -> ViewState:
if not action.tool_call_id:
return state
rows = list(state.rows)
for i, row in enumerate(rows):
if isinstance(row, ToolRow) and row.tool_call_id == action.tool_call_id:
rows[i] = replace(
row,
status="error" if action.is_error else "ok",
result=format_tool_result(action.content),
)
return replace(state, rows=tuple(rows))
# No matching tool card (started chunks missed) -> surface the result anyway.
return _append(
state,
ToolRow(
tool_call_id=action.tool_call_id,
tool_name=action.tool_name,
title=summarize_tool_title(action.tool_name),
status="error" if action.is_error else "ok",
result=format_tool_result(action.content),
),
)