3 Commits

Author	SHA1	Message	Date
Amorend	85c3909c2e	feat: show real-time context window usage (#3125) (#3183) ... * feat: show real-time context window usage in chat UI (#3125) Adds a `context_usage` block to `GET /api/threads/{id}/token-usage` (token count from the live checkpoint, the thread model's `context_window`, and a percentage), introduces a new `ModelConfig.context_window` distinct from the per-call `max_tokens` output cap, and surfaces the percentage in the chat header — inside `TokenUsageIndicator` when token-usage tracking is on, or as a standalone badge when it's off so context capacity stays visible independent of cost tracking. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: per-category breakdown for context window usage Replace the single-number context_usage payload with a Claude-Code-style breakdown — messages, system prompt, skills, system/MCP tools (active + deferred), custom agents, memory injection, autocompact buffer, and free space — and surface it in the chat UI with a segmented progress bar and per-row table. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(config): document context_window across model examples Add `context_window` to every example model in config.example.yaml so the new chat-UI "% context used" indicator works out of the box for whichever example a user adopts. Each value is the published default at the time of writing; users are pointed at the official model spec to verify. Bumps config_version to 11 so `make config-upgrade` flags outdated user configs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style: ruff format (line-length 240) No behavior change — collapses two multi-line expressions that fit on one line under the project's 240-char limit. Picked up by `make format`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * review: address Copilot bot comments on #3183 - token-usage-indicator: switch `{contextPercentage && (...)}` to an explicit `!= null` check. (The string `"0"` is actually truthy in JS so the original code wasn't buggy, but the explicit check is clearer.) - context-usage-breakdown: drop the `useMemo` around segments/totals — the computation is O(n) over a handful of rows and the previous memo deps omitted `t.contextUsage.categories`, so the bar's tooltips/aria-labels could stay in the old language after a locale switch. - context_usage._split_tools: snapshot MCP names from `get_cached_mcp_tools()` directly instead of re-reading `extensions_config.json` after `get_available_tools()` already loaded it. Removes redundant file I/O on every `/token-usage` poll. (`get_available_tools()` still emits its own INFO logs — silencing those is out of scope here.) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style(frontend): prettier --write context-usage-breakdown CI's `pnpm format` (prettier --check) caught two lines previously formatted by hand. Collapses one comma to fit on one line; no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(gateway): correct context-usage breakdown + add exact token counting The context-usage indicator shipped two bugs that silently zeroed whole breakdown rows (both caught by try/except, so the feature looked alive but produced wrong numbers): 1. _count_system_prompt passed app_config= to get_deferred_tools_prompt_section, which only accepts deferred_names -> TypeError swallowed -> system_prompt row always 0, and used_tokens/percentage undercounted by the full prompt. Also subtracted the deferred section twice (the rendered prompt already excluded it). Fix: derive deferred names deterministically and pass them to apply_prompt_template; drop the redundant subtraction. 2. _split_tools imported a non-existent get_deferred_registry -> ImportError swallowed -> all four tool-category rows always 0. Fix: classify via the public is_mcp_tool predicate + tool_search.enabled (mirrors build_deferred_tool_setup); the MCP tag is set by get_available_tools. Added token_usage.counting (approximate|exact). 'exact' routes text/schema/ message counting through the model tokenizer (tiktoken cl100k_base) via the existing memory-module machinery (lazy load + cache + cooldown + CJK-aware fallback), so CJK-heavy threads stop being undercounted by chars//4. Regression + e2e tests added; 6621 backend tests pass. * fix(gateway): harden context usage accounting * fix(gateway): count promoted MCP tools as active in context usage Promoted tools (deferred MCP tools the thread has fetched via tool_search) have their full schema bound on every subsequent turn by DeferredToolFilterMiddleware, so they consume context like any active tool. The breakdown previously left them in the reserved *_deferred rows, under- counting the thread's used_tokens. Classification now treats a tool as deferred only when tool_search is enabled, it is MCP-sourced, AND it has not been promoted. The promoted set is read from the checkpoint's channel_values and scoped by catalog hash — matching the runtime middleware, so a stale promotion from MCP-config drift cannot inflate the active count. The static system prompt still lists all deferred tool names (promotions only affect schema binding, not the prompt), so _count_system_prompt's deferred rendering is intentionally left unchanged. 8 new tests cover classification, catalog-hash scoping (match / drift / compute-failure / malformed), and checkpoint extraction. * fix(context): address review feedback * fix(context): count structured message payloads * fix(context): harden usage accounting * fix(config): bump schema for context usage fields * refactor: narrow context usage to core indicator --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-31 21:57:22 +08:00
Vanzeren	3c8b82c594	fix(runtime): serialize checkpoint writes with active runs (#4437) ... * fix(runtime): serialize checkpoint writes with active runs * fix(runtime): address checkpoint reservation reviews * fix(runtime): address reservation race reviews * fix(runtime): refine reservation conflict semantics	2026-07-25 23:18:34 +08:00
Vanzeren	42baed8c8c	feat(checkpoint): dual-mode checkpoint storage with LangGraph DeltaChannel (#4292) ... * feat(checkpoint): dual-mode checkpoint storage with LangGraph DeltaChannel Add a restart-required database.checkpoint_channel_mode ("full" default, "delta") that stores the messages channel via LangGraph 1.2 DeltaChannel, cutting checkpoint storage from O(n^2) to O(n) for append-only history. Existing full checkpoints seed delta state transparently; no data migration. - config: mode schema + freeze-on-first-use with CheckpointModeReconfigurationError; mode marker persisted in checkpoint metadata; unsafe delta->full downgrade rejected fail-closed with CheckpointModeMismatchError (run-level error, failed state read) - state: delta message state schema; CheckpointStateAccessor centralizes materialized reads for all consumers (threads API, branches, regeneration, compaction, state updates, memory, goal workers) - runtime: raw writers (run durations, interrupted title, thread goal) parent their checkpoints to the checkpoint they derive from, preserving delta ancestry; rollback forks the pre-run lineage through a state mutation graph with Overwrite restores; InMemorySaver delta-history override delegates to the base walk (fixes dropped first write after migration, also present upstream) - tests: conformance suite over {memory, sqlite, postgres} covering migration replay, stable message IDs, storage shape and writer preservation; conftest fixture isolates the frozen mode between tests; stale config fakes refreshed - ci: backend unit tests gain a postgres service * fix(checkpoint): close materialization gaps in goal flow, guard public factory - Route goal-continuation message reads through CheckpointStateAccessor: raw channel_values reads see the delta sentinel in delta mode, which disabled goal continuation (stand_down=no_durable_end_of_turn) after durable assistant turns. Raw tuples remain for tuple-only metadata (checkpoint id, pending_writes). - Reject checkpoint_channel_mode='delta' + checkpointer in create_deerflow_agent at construction: factory-built persisted graphs bypass mode-marker injection and the fail-closed gate, reproducing silent mixed-mode state loss. Delta without persistence stays allowed. - Import the postgres saver lazily (pytest.importorskip in the fixture) so the documented default install collects the suite; add a CI job running pytest --collect-only on uv sync --group dev without extras. - Fix test_checkpointer fallback test to patch get_app_config at its use site (provider module), making it deterministic when a local config.yaml selects a persistent backend. * fix(gateway): preserve extension-owned channels in state mutations, bump config version - build_state_mutation_graph / build_checkpoint_state_mutation_accessor accept an explicit state_schema; branch and POST /state now compile the mutation graph from the thread's effective schema (graph_state_schema on the assistant graph). The base-ThreadState fallback silently discarded channels contributed by custom AgentMiddleware.state_schema on branch (data loss) and returned a false-success 200 on POST /state. - POST /state validates values keys against the mutation graph's channels and rejects unknown fields with 422 instead of ignoring them; reducer detection covers extension channels (BinaryOperatorAggregate or DeltaChannel) so Overwrite replace semantics work for middleware reducers in both modes. - Endpoint regression: custom AgentMiddleware.state_schema value survives branch, updates through POST /state, and an unknown field receives 422. - config_version 26 -> 27 for the new database.checkpoint_channel_mode (example, Helm chart values + README, support-bundle fixture), so existing installs get the outdated-config warning and make config-upgrade merges the field; covered by a test driving the real example file and the real config-upgrade script. * fix(gateway): resolve assistant schema via one boundary, copy branch reducer values with Overwrite GET /threads/{id}/state now resolves the thread's assistant_id through a single reusable boundary (thread metadata -> assistant_id -> effective graph), so channels contributed by a custom AgentMiddleware.state_schema are materialized instead of dropped by the default lead schema. POST /state uses the same boundary instead of resolving the schema ad hoc. Branch writes wrap every copied reducer channel in Overwrite (derived from the effective mutation graph: BinaryOperatorAggregate + DeltaChannel), not just messages, so already-aggregated values are never re-merged. Regression tests use a real AgentMiddleware.state_schema with a non-identity reducer in both full and delta modes: GET /state returns the extension value, POST /state replaces it, branch preserves it byte-for-byte; the unknown-field 422 is a separate assertion. * refactor(checkpoint): collapse read-path round-trips and ship dual-mode parity tests Address review round 4 on PR #4292: - Push ahistory/history limit through Pregel into checkpointer.alist (SQL LIMIT) instead of materializing all rows and breaking in Python - Fold the read-side mode-compat gate onto the returned snapshot's metadata; only writes keep the pre-write tuple fetch (fail-closed) - Cache factory-built accessor graphs per (assistant_id, mode) with factory-identity revalidation; state reads no longer build a lead agent per request - get_thread: one snapshot fetch + one raw pending_writes fetch on the resolved checkpoint (post-checkpoint __error__ writes never surface in snapshot.tasks; verified empirically) - DeerFlowClient.get_thread: single checkpointer.list walk collects pending_writes per checkpoint instead of N get_tuple calls - InMemorySaver delta-history patch: stand-down when the upstream override disappears, try/except guard, validated-version warning, guard tests - make_lead_agent mode precedence: first freeze is owned by app_config (client-supplied configurable key ignored); once frozen, injected key/app_config must match or fail closed - Rollback: lock in non-message channel restoration via fork inheritance with a dedicated reducer-channel test - Add tests/test_threads_checkpoint_mode.py and tests/test_gateway_checkpoint_mode.py referenced by AGENTS.md and the PR validation section: lifecycle parity (memory + sqlite), per-step blob-count storage guard, gateway endpoint parity Counted-saver tests pin checkpoint round-trips for aget/ahistory so these regressions cannot silently return. * fix(checkpoint): precise mode-mismatch HTTP mapping, gate E2E, and accessor resilience - threads router: map CheckpointModeMismatchError to 409 (with cause and thread id) and CheckpointModeReconfigurationError to 503 across all state endpoints instead of swallowing both into a generic 500 - gate coverage: seed a real delta checkpoint into AsyncSqliteSaver and assert aget/aupdate/ahistory fail closed; assert 409 at the HTTP boundary through the real route stack - rollback: compile the restore mutation graph with the thread's effective state schema per the build_state_mutation_graph contract - inheritance contract locks: rollback and manual compaction preserve middleware-contributed channels via checkpoint fork cloning - services: revalidate the accessor-graph cache against app_config identity so config.yaml hot-reloads never serve a stale compiled graph - services: degrade full-mode state reads to raw checkpointer reads when the agent factory is unavailable (delta gate still applies; delta mode has no fallback) - deps: override websockets==16.0 (langgraph-sdk 0.4.2's <16 pin silently downgraded 16.0 -> 15.0.1; pin is not grounded in any API incompatibility) and bump the langchain lower bound to what the lockfile actually resolves * fix(checkpoint): include anchor checkpoint in degraded history walk + cover get_thread - _RawCheckpointReadAccessor.ahistory: alist(before=...) is exclusive while pregel's get_state_history treats config.checkpoint_id as the inclusive start; fetch the anchor explicitly so both read paths paginate identically - extend the degraded-path gateway test: GET /thread returns raw values, and POST /history with before starts at the anchor checkpoint * fix(gateway): preserve degraded checkpoint timestamps * fix(gateway): harden degraded checkpoint access * fix(gateway): resolve assistants for checkpoint reads --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-22 08:33:29 +08:00