1 Commits

Author SHA1 Message Date
subhash7476
a558884c3a
perf(summarization): skip the summarizer LLM call for an input already proven no-op (#6318)
* perf(summarization): skip the summarizer LLM call for an input already proven no-op

When the exact rendered summarizer prompt (plus candidate models) previously
produced output byte-identical to the current summary_text, reuse that summary
instead of calling the LLM. Compaction proceeds through the normal path, so
messages, hooks and state transitions are unchanged.

- Only proven no-op inputs are cached (bounded LRU, thread-safe); failed
  attempts are never cached and any cache error falls back to the LLM call.
- The middleware:summarize event gains changes.llm_call_skipped.
- Applies to both compact_state and acompact_state.

Builds on the per-call telemetry in the P0 branch.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(summarization): keep fallback no-ops retryable

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 22:04:36 +08:00