* feat(gateway): cache-aware cost accounting + /api/console observability endpoints
- Capture prompt-cache hits (usage_metadata.input_token_details.cache_read)
in RunJournal and SubagentTokenCollector as a sparse cache_read_tokens key
in token_usage_by_model (JSON field — no schema migration; legacy bucket
shapes unchanged)
- New read-only /api/console router: GET /stats (headline counters),
GET /runs (cross-thread paginated history joined with thread titles),
GET /usage (zero-filled daily token series + per-model breakdown);
user-scoped, 503 on the memory database backend
- Optional models[*].pricing (currency, input_per_million,
output_per_million, input_cache_hit_per_million) powers real spend
estimation; cache-hit input tokens are billed at the hit price (omitted
hit price falls back to the miss price as a conservative upper bound);
unpriced models yield cost: null
- create_chat_model strips the presentation-only pricing block so it never
reaches the provider client (unknown kwargs are forwarded into the
completion payload and break live calls)
- Tests: console router SQLite round-trips, journal/collector cache capture
incl. a DeepSeek raw-usage pin test, factory strip regression
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: address review feedback on cost sum and sparse cache_read_tokens
- console.py: replace the walrus-in-generator total-cost sum with an
explicit loop (review noted the multi-line form reads ambiguously)
- token_collector.py: omit cache_read_tokens from usage records when the
provider reported no cache hits, matching the journal's sparse
per-model bucket shape; absent is treated as 0 downstream
- add a regression test pinning the sparse record shape
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: coffeeFish <codeingforcoffee@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(gateway): attribute token usage to actual models
Capture per-call model names from LLM response metadata for lead, middleware, and subagent calls.
Persist a per-run token_usage_by_model breakdown and aggregate by that map in both SQL and memory stores, with legacy fallback to the run-level model_name for older rows.
Add regression coverage for by_model totals, caller consistency, active progress snapshots, store parity, and SubagentTokenCollector model propagation.
* fix(gateway): harden by-model token aggregation
Use usage.get("total_tokens", 0) when reducing per-model token usage maps so aggregation tolerates partially written or manually edited JSON blobs without changing behavior for journal-written rows.
* docs(gateway): clarify by-model run count semantics
Document that by_model[*].runs counts the number of runs in which a model appeared, so multi-model runs can increment multiple model buckets.
* fix: bucket subagent token usage into RunRow.subagent_tokens
Add caller-bucketed token tracking to RunJournal so subagent and
middleware LLM calls are written to the correct RunRow columns instead
of all falling into lead_agent_tokens (default 0).
- RunJournal: accumulate _lead_agent_tokens / _subagent_tokens /
_middleware_tokens in on_llm_end, deduped by langchain run_id.
Add record_external_llm_usage_records() for external sources
(respects track_token_usage flag). Return caller buckets from
get_completion_data().
- SubagentTokenCollector: new lightweight callback handler that
collects LLM usage within subagent execution.
- SubagentExecutor: wire collector into subagent run_config and sync
records to SubagentResult on every chunk (timeout/cancel safe).
- SubagentResult: add token_usage_records and usage_reported fields.
- task_tool: report subagent usage to parent RunJournal on every
terminal status (COMPLETED/FAILED/CANCELLED/TIMED_OUT), including
the CancelledError path, guarded against double-reporting.
No DB migration needed — RunRow columns already exist.
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: address token usage review feedback
* Address review follow-ups
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
