mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 21:26:39 +00:00
c943d953cd
* docs(checkpoints): add a bilingual checkpoint storage manual Add the missing docs-site coverage for dual-mode checkpoint storage (`database.checkpoint_channel_mode: full | delta`) as a ten-chapter manual under `harness/checkpoints/` in both languages: concepts, quick start, channel modes, snapshot cadence, history cache, resume and rollback, operating checkpoints, observability, troubleshooting by symptom, and a reference of every configuration key, error message, HTTP status mapping, public symbol, load-bearing test, and pinned upstream defect. The index page is `asIndexPage`, so `/docs/harness/checkpoints` is the section itself and its `_meta.ts` lists only the nine child chapters. Discovery: `harness/_meta.ts` registers the section, the harness configuration module table gains a `database` row, and the thread persistence section of `agents-and-threads` links to the manual. Content is grounded in the code. Only the three measured values recorded in `backend/docs/runtime-guidance-details.md` are quoted; the upstream-derived 41x storage figure in the untracked delta-channel RFC is deliberately not cited. Pinned upstream defects are documented with their issue and PR numbers and the versions they are pinned at (`langgraph 1.2.9`, `langgraph-checkpoint 4.2.0`, `langgraph-checkpoint-postgres 3.1.2`). * docs(checkpoints): correct the history-cache restart requirement The history cache is built with the checkpointer at Gateway startup (`langgraph_runtime()` -> `make_checkpointer()` -> `make_checkpoint_cache()`), so editing `database.checkpoint_cache.*` does not change a running process. The manual previously marked the whole section "not restart-required", which conflated "not frozen for correctness" with "hot-reloadable" and could mislead an operator changing capacity, credentials, or retention. Distinguish the two properties everywhere the section is described: history-cache.mdx marks every field as needing a restart and adds a callout explaining the difference, reference.mdx redefines the Restart column and fixes the five rows and the effect list (adding the `asyncprov` evidence alias), troubleshooting.mdx replaces the "two keys need no restart" claim and gains a dedicated symptom entry, and observability.mdx notes that changed counters need a restart. The Chinese text is corrected in parallel. Verified: the docs link test and the docs unit suite pass, frontmatter parses for all 20 pages, and every in-page anchor resolves on the rendered English and Chinese history-cache, troubleshooting, reference, and observability pages. * docs(checkpoints): account for periodic snapshots in the storage-growth claim Every periodic snapshot stores the entire message list as of that step (`_DeltaSnapshot(ch.get())`, persisted as a checkpoint blob), so with a fixed snapshot frequency K the retained total after N turns is O(N²/K + N), not O(N). The manual claimed unconditional linear growth, which understated capacity planning — the more so because nothing in DeerFlow prunes those snapshots yet. Distinguish per-step delta writes from total retained storage in index.mdx (prose and table), modes.mdx (step-cost row), and operations.mdx; note in cadence.mdx that K is also the capacity knob that scales the snapshot bytes. The English and Chinese chapters are corrected in parallel. The reference chapter keeps the released #4292 wording as a verbatim quotation and now names the retained-storage bound beside it. Verified: `_DeltaSnapshot(ch.get())` and its blob serialization in the pinned langgraph (channels/delta.py:196-198, checkpoint/serde/types.py:19-25) confirm the premise; the docs link test and docs unit suite pass; all ten changed pages render with resolving anchors and no stale linear-growth wording. * docs(checkpoints): correct the run-error surfaces and repair two evidence pointers Review follow-ups on the checkpoint manual, both languages. Error-surface corrections: - Run creation is not always asynchronous: when a thread's run-event feed is empty but a checkpoint head exists, ensure_checkpoint_history_seeded reads that head inside start_run, where only ConflictError (409) and UnsupportedStrategyError (501) are mapped, so a mode mismatch returns a plain 500 before admission. Troubleshooting, observability and the reference route tables now scope the "does not pre-gate" claim. - The 503 symptom is retargeted. Only Gateway startup, make_lead_agent and DeerFlowClient.__init__ call the freeze helpers, so a config.yaml edit cannot make a state/history/thread route answer 503. A mode edit is injected over by the run worker and ignored until a restart; a cadence edit fails the next run with the restart-required message. The router's 503 mapping is marked defensive in modes and the reference. - The regenerate section title no longer groups the state endpoints, which map a mismatch to 409. - operations: the Textual TUI catches the mismatch through stream_actions(); only the headless --print/--json runners print a traceback. Evidence-pointer repairs: - reference: this branch's CHANGELOG insertion shifted the #5734 entry from 3032-3039 to 3040-3048; the citation is re-pointed. - reference: the #8548 row cited an untracked worktree plan file; replaced by the in-repo test anchor, and "mirror issue #8551" reworded to "related". - reference: new `agent` alias for the agent-factory freeze sites. - history-cache: the bare `deps:`/`asyncprov:` aliases are expanded to full paths; the alias legend lives only on the reference page. * docs(checkpoints): say a delta fork is accepted, not refused "Delta mode cannot fork" overstated the constraint. Nothing in the stack rejects the fork: LangGraph stores the second child, no saver checks for an existing sibling, and ensure_checkpoint_mode_compatible returns immediately when the process mode is delta. What breaks is the read — the walk replays the abandoned sibling's writes stored under the shared parent. Reproduced on the pinned stack (langgraph 1.2.9 / langgraph-checkpoint 4.2.0, AsyncSqliteSaver) by running this repo's own #4458 fixture without the linearization call: the forked resume returns ["h1", "a1", "h2", "a2", "a2-new"] instead of ["h1", "a1", "h2", "a2-new"], i.e. the superseded answer comes back. - resume-and-rollback: title/description/heading now say "cannot fork correctly"; new "Forks are accepted, not refused" subsection covering the unguarded write side. - troubleshooting: the "old answer reappears" cause states the same split. - reference: the POST /threads/{id}/state row notes that a client-supplied checkpoint_id older than the head is applied as a delta fork with no linearization — the write shape upstream tracks as #8551 (open), which the run-path linearization does not cover. * docs(checkpoints): scope the unguarded state-update fork to what the probe showed The previous commit said a mutation against an older checkpoint stores its writes under the shared parent and implied the sibling branch suffers for it. Probing the pinned stack (AsyncSqliteSaver, delta messages channel) shows: - the write does land under the shared parent and joins the write set the parent's other children walk (raw walk for a sibling includes the abandoned head's own write); - but reducer channels are written with Overwrite, which replaces the accumulated value, so the forked branch reads clean and the abandoned branch is unchanged in that shape — #8551 did not reproduce; state the masking mechanism and the upstream tracking issue, not a corruption claim. * docs(checkpoints): fix a frontmatter description that broke the docs build `resume-and-rollback.mdx` shipped `description: Delta mode cannot fork a thread correctly: a forked delta read ...` in the previous commit. The unquoted ": " turns the value into a nested mapping, so Nextra's YAML frontmatter parse failed and the e2e lane could not start its web server (YAMLParseError at line 2, column 14) — the exact failure class this manual warns docs authors about, introduced by me two commits ago. - Replace the colon with an em dash in the English description (the Chinese one uses a full-width colon and was unaffected). - Add tests/unit/content/docs-frontmatter.test.ts: every `src/content/**/*.mdx` frontmatter block must be YAML-parseable, implemented as the one rule that actually breaks it (an unquoted scalar containing ": "), because the unit suite never compiles MDX and only a build or a dev render catches it. Verified the guard fails on the broken line and passes on the fix. Verified with a clean `BETTER_AUTH_SECRET=local-dev-secret pnpm build` on this tree: all pages prerender, resume-and-rollback EN/ZH included.