deer-flow/frontend/tests/unit/content/docs-frontmatter.test.ts
Zeren Wang c943d953cd
docs(checkpoints): add a ten-chapter checkpoint storage manual in both languages (#5845)
* docs(checkpoints): add a bilingual checkpoint storage manual

Add the missing docs-site coverage for dual-mode checkpoint storage (`database.checkpoint_channel_mode: full | delta`) as a ten-chapter manual under `harness/checkpoints/` in both languages: concepts, quick start, channel modes, snapshot cadence, history cache, resume and rollback, operating checkpoints, observability, troubleshooting by symptom, and a reference of every configuration key, error message, HTTP status mapping, public symbol, load-bearing test, and pinned upstream defect.

The index page is `asIndexPage`, so `/docs/harness/checkpoints` is the section itself and its `_meta.ts` lists only the nine child chapters. Discovery: `harness/_meta.ts` registers the section, the harness configuration module table gains a `database` row, and the thread persistence section of `agents-and-threads` links to the manual.

Content is grounded in the code. Only the three measured values recorded in `backend/docs/runtime-guidance-details.md` are quoted; the upstream-derived 41x storage figure in the untracked delta-channel RFC is deliberately not cited. Pinned upstream defects are documented with their issue and PR numbers and the versions they are pinned at (`langgraph 1.2.9`, `langgraph-checkpoint 4.2.0`, `langgraph-checkpoint-postgres 3.1.2`).

* docs(checkpoints): correct the history-cache restart requirement

The history cache is built with the checkpointer at Gateway startup (`langgraph_runtime()` -> `make_checkpointer()` -> `make_checkpoint_cache()`), so editing `database.checkpoint_cache.*` does not change a running process. The manual previously marked the whole section "not restart-required", which conflated "not frozen for correctness" with "hot-reloadable" and could mislead an operator changing capacity, credentials, or retention.

Distinguish the two properties everywhere the section is described: history-cache.mdx marks every field as needing a restart and adds a callout explaining the difference, reference.mdx redefines the Restart column and fixes the five rows and the effect list (adding the `asyncprov` evidence alias), troubleshooting.mdx replaces the "two keys need no restart" claim and gains a dedicated symptom entry, and observability.mdx notes that changed counters need a restart. The Chinese text is corrected in parallel.

Verified: the docs link test and the docs unit suite pass, frontmatter parses for all 20 pages, and every in-page anchor resolves on the rendered English and Chinese history-cache, troubleshooting, reference, and observability pages.

* docs(checkpoints): account for periodic snapshots in the storage-growth claim

Every periodic snapshot stores the entire message list as of that step (`_DeltaSnapshot(ch.get())`, persisted as a checkpoint blob), so with a fixed snapshot frequency K the retained total after N turns is O(N²/K + N), not O(N). The manual claimed unconditional linear growth, which understated capacity planning — the more so because nothing in DeerFlow prunes those snapshots yet.

Distinguish per-step delta writes from total retained storage in index.mdx (prose and table), modes.mdx (step-cost row), and operations.mdx; note in cadence.mdx that K is also the capacity knob that scales the snapshot bytes. The English and Chinese chapters are corrected in parallel. The reference chapter keeps the released #4292 wording as a verbatim quotation and now names the retained-storage bound beside it.

Verified: `_DeltaSnapshot(ch.get())` and its blob serialization in the pinned langgraph (channels/delta.py:196-198, checkpoint/serde/types.py:19-25) confirm the premise; the docs link test and docs unit suite pass; all ten changed pages render with resolving anchors and no stale linear-growth wording.

* docs(checkpoints): correct the run-error surfaces and repair two evidence pointers

Review follow-ups on the checkpoint manual, both languages.

Error-surface corrections:
- Run creation is not always asynchronous: when a thread's run-event feed is
  empty but a checkpoint head exists, ensure_checkpoint_history_seeded reads
  that head inside start_run, where only ConflictError (409) and
  UnsupportedStrategyError (501) are mapped, so a mode mismatch returns a
  plain 500 before admission. Troubleshooting, observability and the
  reference route tables now scope the "does not pre-gate" claim.
- The 503 symptom is retargeted. Only Gateway startup, make_lead_agent and
  DeerFlowClient.__init__ call the freeze helpers, so a config.yaml edit
  cannot make a state/history/thread route answer 503. A mode edit is
  injected over by the run worker and ignored until a restart; a cadence edit
  fails the next run with the restart-required message. The router's 503
  mapping is marked defensive in modes and the reference.
- The regenerate section title no longer groups the state endpoints, which
  map a mismatch to 409.
- operations: the Textual TUI catches the mismatch through stream_actions();
  only the headless --print/--json runners print a traceback.

Evidence-pointer repairs:
- reference: this branch's CHANGELOG insertion shifted the #5734 entry from
  3032-3039 to 3040-3048; the citation is re-pointed.
- reference: the #8548 row cited an untracked worktree plan file; replaced by
  the in-repo test anchor, and "mirror issue #8551" reworded to "related".
- reference: new `agent` alias for the agent-factory freeze sites.
- history-cache: the bare `deps:`/`asyncprov:` aliases are expanded to full
  paths; the alias legend lives only on the reference page.

* docs(checkpoints): say a delta fork is accepted, not refused

"Delta mode cannot fork" overstated the constraint. Nothing in the stack
rejects the fork: LangGraph stores the second child, no saver checks for an
existing sibling, and ensure_checkpoint_mode_compatible returns immediately
when the process mode is delta. What breaks is the read — the walk replays
the abandoned sibling's writes stored under the shared parent.

Reproduced on the pinned stack (langgraph 1.2.9 / langgraph-checkpoint
4.2.0, AsyncSqliteSaver) by running this repo's own #4458 fixture without
the linearization call: the forked resume returns
["h1", "a1", "h2", "a2", "a2-new"] instead of ["h1", "a1", "h2", "a2-new"],
i.e. the superseded answer comes back.

- resume-and-rollback: title/description/heading now say "cannot fork
  correctly"; new "Forks are accepted, not refused" subsection covering the
  unguarded write side.
- troubleshooting: the "old answer reappears" cause states the same split.
- reference: the POST /threads/{id}/state row notes that a client-supplied
  checkpoint_id older than the head is applied as a delta fork with no
  linearization — the write shape upstream tracks as #8551 (open), which the
  run-path linearization does not cover.

* docs(checkpoints): scope the unguarded state-update fork to what the probe showed

The previous commit said a mutation against an older checkpoint stores its
writes under the shared parent and implied the sibling branch suffers for it.
Probing the pinned stack (AsyncSqliteSaver, delta messages channel) shows:

- the write does land under the shared parent and joins the write set the
  parent's other children walk (raw walk for a sibling includes the abandoned
  head's own write);
- but reducer channels are written with Overwrite, which replaces the
  accumulated value, so the forked branch reads clean and the abandoned
  branch is unchanged in that shape — #8551 did not reproduce;

state the masking mechanism and the upstream tracking issue, not a corruption
claim.

* docs(checkpoints): fix a frontmatter description that broke the docs build

`resume-and-rollback.mdx` shipped `description: Delta mode cannot fork a
thread correctly: a forked delta read ...` in the previous commit. The
unquoted ": " turns the value into a nested mapping, so Nextra's YAML
frontmatter parse failed and the e2e lane could not start its web server
(YAMLParseError at line 2, column 14) — the exact failure class this manual
warns docs authors about, introduced by me two commits ago.

- Replace the colon with an em dash in the English description (the Chinese
  one uses a full-width colon and was unaffected).
- Add tests/unit/content/docs-frontmatter.test.ts: every `src/content/**/*.mdx`
  frontmatter block must be YAML-parseable, implemented as the one rule that
  actually breaks it (an unquoted scalar containing ": "), because the unit
  suite never compiles MDX and only a build or a dev render catches it.
  Verified the guard fails on the broken line and passes on the fix.

Verified with a clean `BETTER_AUTH_SECRET=local-dev-secret pnpm build` on this
tree: all pages prerender, resume-and-rollback EN/ZH included.
2026-09-25 10:23:36 +08:00

53 lines
1.8 KiB
TypeScript

import { readFileSync, readdirSync } from "node:fs";
import { join } from "node:path";
import { describe, expect, it } from "@rstest/core";
const CONTENT_ROOT = join(process.cwd(), "src/content");
// An unquoted frontmatter scalar containing ": " parses as a nested mapping and
// makes Nextra's YAML frontmatter fail, which 500s every page in the section.
// Only `next build` (or a dev-server render) compiles MDX, so the unit suite has
// to keep this guard on its own.
const UNQUOTED_MAPPING_VALUE = /^([A-Za-z_][\w-]*):[ \t]*(?![|>])/;
const SAFE_VALUE = /^(".*"|'.*'|\[.*\]|\{.*\}|[|>]|$)/;
describe("docs frontmatter", () => {
const files = readdirSync(CONTENT_ROOT, { recursive: true })
.map((entry) => join(CONTENT_ROOT, String(entry)))
.filter((path) => path.endsWith(".mdx"));
it("finds the content tree it is meant to guard", () => {
expect(files.length).toBeGreaterThan(0);
});
it("keeps every frontmatter block YAML-parseable", () => {
const offenders = files.flatMap((path) => {
const source = readFileSync(path, "utf8");
if (!source.startsWith("---\n")) {
return [];
}
const displayPath = path
.slice(CONTENT_ROOT.length + 1)
.split("\\")
.join("/");
return source
.slice(4, source.indexOf("\n---", 4))
.split("\n")
.flatMap((line, index) => {
const key = UNQUOTED_MAPPING_VALUE.exec(line);
if (key === null) {
return [];
}
const value = line.slice(key[0].length);
return !SAFE_VALUE.test(value) && value.includes(": ")
? [
`${displayPath}:${index + 2}: unquoted "${key[1]}" value contains ": "`,
]
: [];
});
});
expect(offenders).toEqual([]);
});
});