mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-28 08:56:13 +00:00
2e15e3fe0d
* fix: generate title for interrupted first turn * test(title): cover partial-exchange + dict-form messages Harden the interrupted-run fallback path added in 19fc34fd: - TitleMiddleware._should_generate_title now accepts a lone first-turn user message when allow_partial_exchange=True, so the worker can still derive a title if cancellation lands before any AI chunk is checkpointed. - runtime/runs/worker._ensure_interrupted_title computes the next checkpoint step defensively (treat missing/non-int step as 0) and renames a shadowed ckpt_config local for readability. - Add four unit tests in tests/test_title_middleware_core_logic.py: partial-exchange allows user-only, partial-exchange still respects an existing title, dict-form messages are recognized, and the sync fallback path derives a title from dict-form messages — matching what channel_values stores in the checkpoint. Refs #3859. * fix: persist interrupted-title via channel_versions bump Address PR #3874 review feedback: ``_ensure_interrupted_title`` previously called ``aput(..., new_versions={})``. LangGraph's DB-backed savers (``PostgresSaver`` and the v4 ``SqliteSaver`` blob layout) strip inline ``channel_values`` from ``put`` and only persist blobs for channels named in ``new_versions`` — so the fallback ``title`` channel was dropped on read-back and ``threads_meta.display_name`` stayed ``"Untitled"`` after refresh on those backends. The original in-memory e2e passed because ``InMemorySaver`` keeps the inline snapshot verbatim. Fix mirrors ``_rollback_to_pre_run_checkpoint`` in the same file: bump ``channel_versions["title"]`` (via ``checkpointer.get_next_version`` when available, else int/string fallbacks), persist the new version on the checkpoint, and declare it in ``new_versions`` so the DB savers actually write the blob. Regression coverage in ``tests/test_run_worker_rollback.py``: - ``test_ensure_interrupted_title_bumps_channel_version_and_declares_it_in_new_versions`` — exact ``aput`` invariants: ``new_versions == {"title": 1}``, the written checkpoint's ``channel_versions["title"]`` is bumped, and the pre-existing ``messages`` version is preserved. - ``test_ensure_interrupted_title_bumps_existing_string_version`` — string-shaped prior version (some savers use UUID-style versions); bumped value must differ from the prior, no overwrite-in-place. - ``test_ensure_interrupted_title_skips_when_title_already_set`` — title short-circuit; no extra ``aput``. - ``test_ensure_interrupted_title_returns_none_when_no_checkpoint`` — no checkpoint yet; returns ``None`` without writing. - ``test_ensure_interrupted_title_round_trip_with_real_sqlite_checkpointer`` — full round-trip against a real ``AsyncSqliteSaver`` on a disk-backed DB, then closes and re-opens the saver to simulate a fresh connection. The fallback title must still be present on the second ``aget_tuple``. This is the exact scenario the review flagged. Validated locally with the full backend suite: 5195 passed, 18 skipped. Refs #3859. Addresses review on #3874. * test(worker, title): harden interrupted-title fallback for every saver Defensive coverage on top of the channel_versions fix (commit 05253957), addressing edge cases surfaced during a second-pass review of #3874. Worker: - Extract version bump into ``_bump_channel_version(checkpointer, current)`` with explicit fallbacks for int / float / numeric-string / UUID-shaped string / None / bool, AND a wrap-around defense when the saver's ``get_next_version`` raises or returns an unchanged value. The invariant is: returned version MUST differ from the prior. Without this, a saver bug (or a custom backend) could leave ``new_versions={"title": v}`` no-op on DB savers — the very class of bug the original review pointed out. Title middleware: - Coerce ``state.get("messages")`` from ``None`` to ``[]`` on both ``_should_generate_title`` and ``_build_title_prompt``. A partially-initialized checkpoint can carry ``messages=None`` on the channel_values channel (the worker reads raw channel_values, not BaseMessages), and the default kwarg only protects against a missing key. Repro: ``TypeError: 'NoneType' object is not iterable`` from the next() generator — confirmed by reverting the fix and watching ``test_*_handles_none_messages_channel`` go red. Tests (TDD-verified red→green for the new asserts): - ``test_run_worker_rollback.py``: * ``_bump_channel_version`` — 8 tests covering every version type (int, float, numeric string, UUID-style string, None, bool) and every saver-side fault mode (no ``get_next_version`` / raising / stuck on identity). * ``test_ensure_interrupted_title_*`` — 5 additional helper boundary tests: title.enabled=false short-circuit; empty messages list; messages=None; aput-error propagation (helper contract: caller swallows, not the helper); idempotency on a real InMemorySaver across two invocations. * ``test_ensure_interrupted_title_preserves_non_title_channel_versions`` — pins that ``new_versions`` only contains ``"title"`` and that other channels' versions are untouched (regression anchor for a sloppier draft that bumped every channel). * ``test_worker_finally_block_swallows_helper_exceptions`` — pins the integration contract: even if the helper raises, the worker's threads_meta status sync still runs and ``publish_end`` is still awaited so the SSE stream closes cleanly. - ``test_title_middleware_core_logic.py``: * 4 additional tests: ``messages=None`` on both ``_should_generate_title`` and ``_build_title_prompt``; the ``role: user`` / ``role: assistant`` (OpenAI-style) dict normalization; partial-exchange path with a dict-form message. Verification: - ``PYTHONPATH=. uv run pytest tests/ -x --ignore=tests/blocking_io -q`` → 5215 passed, 18 skipped. - ``ruff check`` + ``ruff format --check`` clean on every touched file. - Red/green TDD verification: temporarily reverted the ``new_versions={}`` fix → 4 new tests went red as expected; restored and the suite is green again. Same red/green dance for the ``messages=None`` coercion. Refs #3859. Addresses second-pass review on #3874. * fix(title): ignore dict context reminders in fallback * fix(worker): link interrupted-title checkpoint to its parent The title-bump checkpoint written by ``_ensure_interrupted_title`` was landing without a ``parent_checkpoint_id`` — a real orphan in the LangGraph history graph. Reproduction (disk-backed AsyncSqliteSaver): [seed] checkpoint_id = 1f173dbc... [helper] wrote title = "Why is the sky blue?" [issue 1] new checkpoint = 1f173dbc..., parent = None [issue 1] is new checkpoint orphaned? True Root cause: ``_ensure_interrupted_title`` built ``write_config`` as ``{"thread_id": ..., "checkpoint_ns": ...}`` only. ``BaseCheckpointSaver`` implementations read ``configurable.checkpoint_id`` from that config as the *parent* id when inserting (see ``langgraph/checkpoint/sqlite/aio.py`` ``aput``: ``config["configurable"].get("checkpoint_id")`` becomes the ``parent_checkpoint_id`` column). With no value, the saver writes NULL — the new checkpoint is a tree root. Consequences: - Any future LangGraph ``runs.resume_from`` / time-travel feature has no backward edge to walk past the title-bump. - History-visualization UIs built on ``alist()`` render the title-bump as a sibling of the prior checkpoint, not its descendant. Fix: read ``checkpoint_id`` off the tuple's own config and thread it into ``write_config["configurable"]["checkpoint_id"]`` before calling ``aput``, the same pattern every middleware-driven write uses. Three new regression tests against real ``AsyncSqliteSaver`` (disk-backed, fresh connections so we exercise the on-disk read path): - ``test_ensure_interrupted_title_links_new_checkpoint_to_its_parent`` — asserts ``latest.parent_config["configurable"]["checkpoint_id"]`` equals the seeded checkpoint id. TDD red-green verified: reverting the fix flips this test red with ``AssertionError: title-bump checkpoint must have a parent_config``. - ``test_ensure_interrupted_title_appears_in_history_with_audit_marker`` — pins the audit contract: the title-bump entry in ``alist()`` carries ``metadata.source == "update"`` and ``metadata.writes`` contains ``runtime_interrupt_title``. This is a deliberate design choice — we do NOT hide the entry from history (audit trail belongs in the saver), but its source and writes marker MUST be unambiguous so UIs/tools can identify it. - ``test_ensure_interrupted_title_survives_immediate_next_turn`` — cancel → immediate user follow-up scenario. Simulates the agent's next turn appending a (user, ai) pair without touching the title channel, then opens a fresh saver and verifies the title is still present after the next-turn checkpoint write. Pins the channel-version-blob invariant established by commit 05253957 — without the ``new_versions={"title": v}`` declaration there, the title blob would vanish from the DB and this test would read back ``None``. Verification: - ``PYTHONPATH=. uv run pytest tests/ -x --ignore=tests/blocking_io -q`` → 5222 passed, 15 skipped. - ``ruff check`` + ``ruff format --check`` clean on every touched file. - Reproduction script confirms ``parent_checkpoint_id`` is now non-null and the next-turn read-back preserves the fallback title. Refs #3859. * Revert "fix(worker): link interrupted-title checkpoint to its parent" This reverts commit c763ed9334781db1acdce0f5f33d663d8d5f80ff. * test: trim over-engineered test coverage Reduce review surface area on PR #3874 by dropping defensive tests that don't pin a real invariant. After self-review: - ``_bump_channel_version``: 8 tests → 2 (happy path + saver-error fallback). Dropped float / bool / numeric-string / UUID-string / missing-get-next-version / stuck-get-next-version branches — those are speculative scaffolding for savers we don't ship. - ``_ensure_interrupted_title``: dropped ``returns_none_when_title_disabled``, ``returns_none_with_no_user_message``, ``returns_none_when_no_checkpoint`` — boundary guards already exercised by the e2e test and the ``handles_none_messages_channel`` regression anchor. Net: -107 lines of test code. Remaining coverage still pins every red-green-verified invariant (channel_versions bump, string-version bump, idempotency, sqlite round-trip, non-title channel preservation, aput-error contract, worker finally swallowing, partial-exchange). Verification: 5209 passed, 15 skipped. * fix: harden interrupted title finalization * fix: serialize interrupted title finalization * fix: preserve interrupt semantics during title finalization * fix: preserve delayed interrupted title recovery --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
231 lines
6.9 KiB
Markdown
231 lines
6.9 KiB
Markdown
# 自动 Title 生成功能实现总结
|
||
|
||
## ✅ 已完成的工作
|
||
|
||
### 1. 核心实现文件
|
||
|
||
#### [`packages/harness/deerflow/agents/thread_state.py`](../packages/harness/deerflow/agents/thread_state.py)
|
||
- ✅ 添加 `title: str | None = None` 字段到 `ThreadState`
|
||
|
||
#### [`packages/harness/deerflow/config/title_config.py`](../packages/harness/deerflow/config/title_config.py) (新建)
|
||
- ✅ 创建 `TitleConfig` 配置类
|
||
- ✅ 支持配置：enabled, max_words, max_chars, model_name, prompt_template
|
||
- ✅ 提供 `get_title_config()` 和 `set_title_config()` 函数
|
||
- ✅ 提供 `load_title_config_from_dict()` 从配置文件加载
|
||
|
||
#### [`packages/harness/deerflow/agents/middlewares/title_middleware.py`](../packages/harness/deerflow/agents/middlewares/title_middleware.py) (新建)
|
||
- ✅ 创建 `TitleMiddleware` 类
|
||
- ✅ 实现 `_should_generate_title()` 检查是否需要生成
|
||
- ✅ 默认使用本地 fallback 生成标题，避免流式回复结束前等待额外 LLM 调用；显式配置 `model_name` 时可使用 LLM 标题
|
||
- ✅ 实现 `after_model()` / `aafter_model()` 钩子，在首次对话后自动触发
|
||
- ✅ 包含 fallback 策略（LLM 未配置或失败时使用用户消息前几个字符）
|
||
|
||
#### [`packages/harness/deerflow/config/app_config.py`](../packages/harness/deerflow/config/app_config.py)
|
||
- ✅ 导入 `load_title_config_from_dict`
|
||
- ✅ 在 `from_file()` 中加载 title 配置
|
||
|
||
#### [`packages/harness/deerflow/agents/lead_agent/agent.py`](../packages/harness/deerflow/agents/lead_agent/agent.py)
|
||
- ✅ 导入 `TitleMiddleware`
|
||
- ✅ 注册到 `middleware` 列表：`[SandboxMiddleware(), TitleMiddleware()]`
|
||
|
||
### 2. 配置文件
|
||
|
||
#### [`config.yaml`](../../config.example.yaml)
|
||
- ✅ 添加 title 配置段：
|
||
```yaml
|
||
title:
|
||
enabled: true
|
||
max_words: 6
|
||
max_chars: 60
|
||
model_name: null # null = 快速本地 fallback；填模型名才启用 LLM 标题
|
||
```
|
||
|
||
### 3. 文档
|
||
|
||
#### [`docs/AUTO_TITLE_GENERATION.md`](../docs/AUTO_TITLE_GENERATION.md) (新建)
|
||
- ✅ 完整的功能说明文档
|
||
- ✅ 实现方式和架构设计
|
||
- ✅ 配置说明
|
||
- ✅ 客户端使用示例（TypeScript）
|
||
- ✅ 工作流程图（Mermaid）
|
||
- ✅ 故障排查指南
|
||
- ✅ State vs Metadata 对比
|
||
|
||
#### [`TODO.md`](TODO.md)
|
||
- ✅ 添加功能完成记录
|
||
|
||
### 4. 测试
|
||
|
||
#### [`tests/test_title_generation.py`](../tests/test_title_generation.py) (新建)
|
||
- ✅ 配置类测试
|
||
- ✅ Middleware 初始化测试
|
||
- ✅ TODO: 集成测试（需要 mock Runtime）
|
||
|
||
---
|
||
|
||
## 🎯 核心设计决策
|
||
|
||
### 为什么使用 State 而非 Metadata？
|
||
|
||
| 方面 | State (✅ 采用) | Metadata (❌ 未采用) |
|
||
|------|----------------|---------------------|
|
||
| **持久化** | 自动（通过 checkpointer） | 取决于实现，不可靠 |
|
||
| **版本控制** | 支持时间旅行 | 不支持 |
|
||
| **类型安全** | TypedDict 定义 | 任意字典 |
|
||
| **标准化** | LangGraph 核心机制 | 扩展功能 |
|
||
|
||
### 工作流程
|
||
|
||
```
|
||
用户发送首条消息
|
||
↓
|
||
Agent 处理并返回回复
|
||
↓
|
||
TitleMiddleware.after_model()/aafter_model() 触发
|
||
↓
|
||
检查：是否首次对话？是否已有 title？
|
||
↓
|
||
默认从首条用户消息生成本地 fallback title
|
||
↓
|
||
如果显式配置 title.model_name，才调用 LLM 生成更精炼的 title
|
||
↓
|
||
返回 {"title": "..."} 更新 state
|
||
↓
|
||
Checkpointer 自动持久化（如果配置了）
|
||
↓
|
||
客户端从 state.values.title 读取
|
||
```
|
||
|
||
---
|
||
|
||
## 📋 使用指南
|
||
|
||
### 后端配置
|
||
|
||
1. **启用/禁用功能**
|
||
```yaml
|
||
# config.yaml
|
||
title:
|
||
enabled: true # 设为 false 禁用
|
||
```
|
||
|
||
2. **自定义配置**
|
||
```yaml
|
||
title:
|
||
enabled: true
|
||
max_words: 8 # 标题最多 8 个词
|
||
max_chars: 80 # 标题最多 80 个字符
|
||
model_name: null # null = 快速本地 fallback；填模型名才启用 LLM 标题
|
||
```
|
||
|
||
3. **配置持久化（可选）**
|
||
|
||
如果需要在本地开发时持久化 title：
|
||
|
||
```python
|
||
# checkpointer.py
|
||
from langgraph.checkpoint.sqlite import SqliteSaver
|
||
|
||
checkpointer = SqliteSaver.from_conn_string("deerflow.db")
|
||
```
|
||
|
||
```json
|
||
// langgraph.json
|
||
{
|
||
"graphs": {
|
||
"lead_agent": "deerflow.agents:lead_agent"
|
||
},
|
||
"checkpointer": "checkpointer:checkpointer"
|
||
}
|
||
```
|
||
|
||
### 客户端使用
|
||
|
||
```typescript
|
||
// 获取 thread title
|
||
const state = await client.threads.getState(threadId);
|
||
const title = state.values.title || "New Conversation";
|
||
|
||
// 显示在对话列表
|
||
<li>{title}</li>
|
||
```
|
||
|
||
**⚠️ 注意**：Title 在 `state.values.title`，而非 `thread.metadata.title`
|
||
|
||
---
|
||
|
||
## 🧪 测试
|
||
|
||
```bash
|
||
# 运行测试
|
||
pytest tests/test_title_generation.py -v
|
||
|
||
# 运行所有测试
|
||
pytest
|
||
```
|
||
|
||
---
|
||
|
||
## 🔍 故障排查
|
||
|
||
### Title 没有生成？
|
||
|
||
1. 检查配置：`title.enabled = true`
|
||
2. 确认是首次对话（1 个用户消息 + 1 个助手回复）
|
||
3. 如果显式配置了 `title.model_name`，检查标题模型是否可用；未配置时会走本地 fallback
|
||
|
||
### Title 生成但看不到？
|
||
|
||
1. 确认读取位置：`state.values.title`（不是 `thread.metadata.title`）
|
||
2. 检查 API 响应是否包含 title
|
||
3. 重新获取 state
|
||
|
||
### Title 重启后丢失？
|
||
|
||
1. 本地开发需要配置 checkpointer
|
||
2. LangGraph Platform 会自动持久化
|
||
3. 检查数据库确认 checkpointer 工作正常
|
||
|
||
### 中断首轮后仍显示默认标题？
|
||
|
||
1. `runtime/runs/worker.py` 会在 interrupted-run cleanup 中保持 run 处于 finalizing 状态，避免同线程新 run 在 fallback title 写入期间覆盖 checkpoint
|
||
2. 如果取消发生在可用 checkpoint 写入前，worker 会使用本次 `graph_input` 中的首条用户消息生成本地 fallback title
|
||
3. fallback title 写入前会重新读取 latest checkpoint；如果同线程状态已经前进，只对最新 snapshot 做 title-only 更新，避免旧消息重新成为 latest
|
||
|
||
---
|
||
|
||
## 📊 性能影响
|
||
|
||
- **默认延迟**：默认 `title.model_name: null` 不会发起额外 LLM 调用，仅从首条用户消息生成本地 fallback title
|
||
- **显式 LLM 标题延迟**：只有配置 `title.model_name` 时，首轮回复后才会等待一次标题模型调用
|
||
- **并发安全**：在 `after_model()` / `aafter_model()` 中更新 state，不需要客户端额外请求
|
||
- **资源消耗**：每个 thread 只生成一次
|
||
|
||
### 优化建议
|
||
|
||
1. 默认保持 `model_name: null`，避免流式回复结束前的额外 LLM 等待
|
||
2. 如需更精炼标题，再显式配置较快的标题模型
|
||
3. 减少 `max_words` 和 `max_chars`，并让 prompt 保持简洁
|
||
|
||
---
|
||
|
||
## 🚀 下一步
|
||
|
||
- [ ] 补充 prompt template 的集成测试
|
||
- [ ] 支持多语言 title 生成
|
||
- [ ] 添加 title 重新生成功能
|
||
- [ ] 监控 title 生成成功率和延迟
|
||
|
||
---
|
||
|
||
## 📚 相关资源
|
||
|
||
- [完整文档](../docs/AUTO_TITLE_GENERATION.md)
|
||
- [LangGraph Middleware](https://langchain-ai.github.io/langgraph/concepts/middleware/)
|
||
- [LangGraph State 管理](https://langchain-ai.github.io/langgraph/concepts/low_level/#state)
|
||
- [LangGraph Checkpointer](https://langchain-ai.github.io/langgraph/concepts/persistence/)
|
||
|
||
---
|
||
|
||
*实现完成时间: 2026-01-14*
|