1 Commits

Author SHA1 Message Date
subhash7476
20bd989fc1
feat(runtime): add per-LLM-call telemetry and summarization no-op event (#6236)
* feat(runtime): add P0 per-LLM-call telemetry and summarization no-op counter

Measurement only; no runtime behavior changes.

- RunJournal enriches llm.ai.response / llm.error metadata with
  langchain_run_id, caller_category, provider, model, stop_reason, status,
  token counts and rendered request size (chars). Unavailable values are None.
- Summarization middleware emits a middleware:summarize event per summarizer
  call and counts outputs identical to the prior summary_text. The call is
  never skipped.
- All telemetry fails soft.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(runtime): register summarize middleware tag, document telemetry contract

- Add MIDDLEWARE_SUMMARIZE_TAG to the middleware event catalog and the
  contract's known_tags; emit the middleware:summarize event with it.
- Document the optional per-call telemetry metadata on llm.ai.response and
  llm.error in contracts/run_event_stream_contract.json.
- Measure request size by walking string leaves instead of json.dumps on the
  event-loop thread.
- Initialise summarization call/no-op counters in __init__.
- Move RunJournal telemetry docs to the runtime guide.
- Add tests: llm.error without a start, replayed on_llm_end, and a DB-store
  metadata round trip.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(runtime): refresh token counters on usage replay

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-04 14:52:04 +08:00