fix(agents): isolate loop detection state by run (#5344)

* fix(agents): scope loop detection state per run

* fix(agents): harden loop scope fallback

* docs: move loop lifecycle detail out of inherited guidance

---------

Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
PeaceMaker-best 2026-09-11 13:45:52 +08:00 committed by GitHub
parent c35022e18b
commit 36ce7590b7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 581 additions and 143 deletions

View File

@ -0,0 +1,38 @@
# Loop detection lifecycle
`LoopDetectionMiddleware` owns call-pattern detection. Its place in the chain
and audit contract are documented in the
[middleware guide](../packages/harness/deerflow/agents/middlewares/AGENTS.md).
## Run-scoped state
Hash histories, frequency windows/counters, and warning-suppression sets
share a `(thread_id, run_id)` key. This gives a fresh budget to each user
run when a compiled graph is cached and reused, while keeping one budget
across repeated graph entries belonging to the same Gateway run (including
hidden goal continuations). `after_agent` clears only transient pending
warnings for its own scope, not those histories; the sync/async
`before_agent` hooks remain topology-preserving no-ops and must not delete a
sibling run's pending warning. Direct LangGraph embedders may omit
`context.run_id`; that fallback is anchored to the invocation's shared
`Runtime.control` object and mapped to an opaque generated ID, because
LangGraph replaces `Runtime` per node and CPython can reuse freed object
addresses. `after_agent` releases the anchor mapping, while the bounded map
covers abnormal exits. The compatibility-named
`max_tracked_threads` limit bounds run scopes, and `reset(thread_id)` clears
every retained run scope for that thread.
## Decision ordering
Loop decisions are severity-first across both detection layers: a warning
candidate never short-circuits frequency accounting for the remaining calls
in an admitted batch. A hard limit can stop scanning immediately because it
rejects the entire batch. Only the selected warning is marked and logged;
hash warnings still precede frequency warnings when neither layer stops the
run. Among simultaneous frequency-warning candidates, the first crossing in
model tool-call order remains selected for compatibility; later calls are
still counted and can warn in a later batch. A frequency warning whose burst
decays within the batch must not leave a stale suppression mark.
`backend/tests/test_loop_detection_middleware.py` covers mixed-tool batches,
window decay, overrides, and sync/async compiled-graph execution.

View File

@ -97,17 +97,11 @@ Before changing a later authorization phase, read the [authorization RFC](../../
27. **SystemMessageCoalescingMiddleware** - Merges every SystemMessage into a single leading SystemMessage per request; provider-agnostic fix for strict backends (vLLM/SGLang/Qwen/Anthropic) that reject non-leading system messages. Touches the per-request payload only (checkpoint state unchanged); on midnight crossings only the latest `dynamic_context_reminder` SystemMessage survives. The subagent builder places its date-only context middleware immediately before this coalescer, so the built-in subagent prompt and hidden date reminder still reach providers as one leading system block 27. **SystemMessageCoalescingMiddleware** - Merges every SystemMessage into a single leading SystemMessage per request; provider-agnostic fix for strict backends (vLLM/SGLang/Qwen/Anthropic) that reject non-leading system messages. Touches the per-request payload only (checkpoint state unchanged); on midnight crossings only the latest `dynamic_context_reminder` SystemMessage survives. The subagent builder places its date-only context middleware immediately before this coalescer, so the built-in subagent prompt and hidden date reminder still reach providers as one leading system block
28. **SubagentLimitMiddleware** - *(optional, if `subagent_enabled`)* Truncates excess ordinary `task` tool calls to enforce both the per-response concurrency limit (`max_concurrent_subagents`, resolved against startup `subagent_runtime.max_running` and the 1-64 safety range before construction) and the per-run total delegation cap (`max_total_subagents` runtime override or `subagents.max_total_per_run`, default 6, clamped to 1-50). The total cap counts current-run entries in the durable delegation ledger (entries are tagged with `run_id` when captured), so repeated planning checkpoints in one run cannot keep launching legal-sized batches indefinitely, while later user turns in the same thread get a fresh run budget. Explicit durable `batch_task` calls are a separate mode with persisted total/live/running limits and are not rewritten into ordinary ledger entries. If the ordinary cap is exhausted, the middleware strips remaining `task` calls, forces `finish_reason="stop"`, and appends a visible limit note so the run can synthesize existing results instead of ending with an empty tool-call response. 28. **SubagentLimitMiddleware** - *(optional, if `subagent_enabled`)* Truncates excess ordinary `task` tool calls to enforce both the per-response concurrency limit (`max_concurrent_subagents`, resolved against startup `subagent_runtime.max_running` and the 1-64 safety range before construction) and the per-run total delegation cap (`max_total_subagents` runtime override or `subagents.max_total_per_run`, default 6, clamped to 1-50). The total cap counts current-run entries in the durable delegation ledger (entries are tagged with `run_id` when captured), so repeated planning checkpoints in one run cannot keep launching legal-sized batches indefinitely, while later user turns in the same thread get a fresh run budget. Explicit durable `batch_task` calls are a separate mode with persisted total/live/running limits and are not rewritten into ordinary ledger entries. If the ordinary cap is exhausted, the middleware strips remaining `task` calls, forces `finish_reason="stop"`, and appends a visible limit note so the run can synthesize existing results instead of ending with an empty tool-call response.
29. **LoopDetectionMiddleware** - *(optional, if `loop_detection.enabled`)* Detects repeated tool-call loops; hard-stop clears both structured `tool_calls` and raw provider tool-call metadata before forcing a final text answer; stamps `loop_capped` via `consume_stop_reason` (#3875 Phase 2), symmetric to `TokenBudgetMiddleware`; persists warned-state transitions (first per call hash or per tool-frequency burst) and hard stops as `middleware:loop_detection`, attributed with `is_subagent` and the optional `agent_id`, without tool arguments, message content, tool results, or argument-derived hashes. Ordinary task subagents get dedicated recorder keys through a parent-loop proxy; never pass `RunJournal` into their isolated loop. Durable batch subagents have no parent run journal and do not persist these transitions 29. **LoopDetectionMiddleware** - *(optional, if `loop_detection.enabled`)* Detects repeated tool-call loops; hard-stop clears both structured `tool_calls` and raw provider tool-call metadata before forcing a final text answer; stamps `loop_capped` via `consume_stop_reason` (#3875 Phase 2), symmetric to `TokenBudgetMiddleware`; persists warned-state transitions (first per call hash or per tool-frequency burst) and hard stops as `middleware:loop_detection`, attributed with `is_subagent` and the optional `agent_id`, without tool arguments, message content, tool results, or argument-derived hashes. Ordinary task subagents get dedicated recorder keys through a parent-loop proxy; never pass `RunJournal` into their isolated loop. Durable batch subagents have no parent run journal and do not persist these transitions
Loop decisions are severity-first across both detection layers: a warning State is run-scoped: new user runs get fresh budgets; same-run goal
candidate never short-circuits frequency accounting for the remaining calls continuations share history. Keep sibling warnings isolated and lifecycle
in an admitted batch. A hard limit can stop scanning immediately because it hooks topology-stable. Before changing this guard, read
rejects the entire batch. Only the selected warning is marked and logged; [Loop detection lifecycle](../../../../../docs/LOOP_DETECTION.md) for
hash warnings still precede frequency warnings when neither layer stops the fallback identity, cleanup/LRU/reset, severity ordering, and test invariants.
run. Among simultaneous frequency-warning candidates, the first crossing in
model tool-call order remains selected for compatibility; later calls are
still counted and can warn in a later batch. A frequency warning whose burst
decays within the batch must not leave a stale suppression mark.
`tests/test_loop_detection_middleware.py` covers mixed-tool batches, window
decay, overrides, and sync/async compiled-graph execution.
30. **TokenBudgetMiddleware** - *(optional, if `token_budget.enabled`)* Enforces per-run token limits 30. **TokenBudgetMiddleware** - *(optional, if `token_budget.enabled`)* Enforces per-run token limits
31. **Custom middlewares** - *(optional)* Any `custom_middlewares` passed to `build_middlewares` are injected here, before config-declared extensions and the terminal-response/safety/clarification tail 31. **Custom middlewares** - *(optional)* Any `custom_middlewares` passed to `build_middlewares` are injected here, before config-declared extensions and the terminal-response/safety/clarification tail
32. **Configured extension middlewares** - *(optional, if `extensions.middlewares` is set in `config.yaml` or `extensions_config.json`)* Zero-argument `AgentMiddleware` classes loaded from `module.path:ClassName` entries via `deerflow.reflection.resolve_class`. Missing packages, invalid classes, and broken modules fail loudly at agent creation. These run after built-ins/programmatic custom middleware and after the lead/subagent loop/token guards, but before the terminal-response/safety/clarification tail; subagents receive the same configured extension middleware class list before their safety tail. Treat these files as trusted operator config because middleware paths instantiate arbitrary code. Gateway skill/MCP toggle endpoints preserve this field through `to_file_dict()` but must not add a write path for `extensions.middlewares` without an explicit trust-boundary review. Lead-only vs subagent-only middleware lists and per-context constructor parameters are not expressible in this MVP. 32. **Configured extension middlewares** - *(optional, if `extensions.middlewares` is set in `config.yaml` or `extensions_config.json`)* Zero-argument `AgentMiddleware` classes loaded from `module.path:ClassName` entries via `deerflow.reflection.resolve_class`. Missing packages, invalid classes, and broken modules fail loudly at agent creation. These run after built-ins/programmatic custom middleware and after the lead/subagent loop/token guards, but before the terminal-response/safety/clarification tail; subagents receive the same configured extension middleware class list before their safety tail. Treat these files as trusted operator config because middleware paths instantiate arbitrary code. Gateway skill/MCP toggle endpoints preserve this field through `to_file_dict()` but must not add a write path for `extensions.middlewares` without an explicit trust-boundary review. Lead-only vs subagent-only middleware lists and per-context constructor parameters are not expressible in this MVP.

View File

@ -37,6 +37,16 @@ instead of carrying it into a later invocation for the same thread. The
hard-stop path still forces termination when the configured safety limit hard-stop path still forces termination when the configured safety limit
is reached. is reached.
Detection histories and warning-suppression state are scoped by
``(thread_id, run_id)`` because one compiled graph can serve many runs for
the same conversation. They deliberately survive ``after_agent``: a
single Gateway run may re-enter that graph for hidden goal continuations,
and those continuations share one loop budget. A later user run receives a
fresh budget even when it reuses the graph. Standalone library invocations
that omit ``run_id`` receive an opaque fallback ID anchored to LangGraph's
run-scoped ``Runtime.control`` object, so replacement ``Runtime`` wrappers
share one budget within an invocation while a later invocation starts fresh.
Stop-reason surfacing (#3875 Phase 2): Stop-reason surfacing (#3875 Phase 2):
Like the token-budget guard, the loop hard stop does NOT raise it Like the token-budget guard, the loop hard stop does NOT raise it
strips ``tool_calls`` so the agent loop terminates naturally with a strips ``tool_calls`` so the agent loop terminates naturally with a
@ -55,6 +65,7 @@ import hashlib
import json import json
import logging import logging
import threading import threading
import uuid
from collections import Counter, OrderedDict, defaultdict, deque from collections import Counter, OrderedDict, defaultdict, deque
from collections.abc import Awaitable, Callable from collections.abc import Awaitable, Callable
from copy import deepcopy from copy import deepcopy
@ -80,11 +91,13 @@ logger = logging.getLogger(__name__)
_DEFAULT_WARN_THRESHOLD = 3 # inject warning after 3 identical calls _DEFAULT_WARN_THRESHOLD = 3 # inject warning after 3 identical calls
_DEFAULT_HARD_LIMIT = 5 # force-stop after 5 identical calls _DEFAULT_HARD_LIMIT = 5 # force-stop after 5 identical calls
_DEFAULT_WINDOW_SIZE = 20 # track last N tool calls _DEFAULT_WINDOW_SIZE = 20 # track last N tool calls
_DEFAULT_MAX_TRACKED_THREADS = 100 # LRU eviction limit _DEFAULT_MAX_TRACKED_THREADS = 100 # LRU limit for tracked thread/run scopes
_DEFAULT_TOOL_FREQ_WARN = 30 # warn after 30 calls to the same tool type _DEFAULT_TOOL_FREQ_WARN = 30 # warn after 30 calls to the same tool type
_DEFAULT_TOOL_FREQ_HARD_LIMIT = 50 # force-stop after 50 calls to the same tool type _DEFAULT_TOOL_FREQ_HARD_LIMIT = 50 # force-stop after 50 calls to the same tool type
_MAX_PENDING_WARNINGS_PER_RUN = 4 _MAX_PENDING_WARNINGS_PER_RUN = 4
type _RunScopeKey = tuple[str, str | None]
def _normalize_tool_call_args(raw_args: object) -> tuple[dict, str | None]: def _normalize_tool_call_args(raw_args: object) -> tuple[dict, str | None]:
"""Normalize tool call args to a dict plus an optional fallback key. """Normalize tool call args to a dict plus an optional fallback key.
@ -216,8 +229,9 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
tool_calls entirely. Default: 5. tool_calls entirely. Default: 5.
window_size: Size of the sliding window for tracking calls. window_size: Size of the sliding window for tracking calls.
Default: 20. Default: 20.
max_tracked_threads: Maximum number of threads to track before max_tracked_threads: Maximum number of thread/run scopes to track before
evicting the least recently used. Default: 100. evicting the least recently used. The configuration name is retained
for compatibility. Default: 100.
tool_freq_warn: Maximum number of same-tool-type calls within a tool_freq_warn: Maximum number of same-tool-type calls within a
sliding window of ``_tool_freq_window`` before injecting a sliding window of ``_tool_freq_window`` before injecting a
frequency warning. Catches cross-file read loops that frequency warning. Catches cross-file read loops that
@ -272,28 +286,35 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
*(hard for _, hard in self._tool_freq_overrides.values()), *(hard for _, hard in self._tool_freq_overrides.values()),
) )
self._lock = threading.Lock() self._lock = threading.Lock()
self._history: OrderedDict[str, list[str]] = OrderedDict() # LangGraph replaces Runtime per graph node but retains one RunControl
self._warned: dict[str, set[str]] = defaultdict(set) # object for the whole invocation. Map that stable anchor to an opaque
# Windowed per-tool-type frequency: recent tool names per thread, # token when embedders omit run_id. Keeping the anchor strongly referenced
# also prevents CPython from reusing its address while the mapping is live;
# after_agent releases normal invocations and the cap bounds abnormal ones.
self._fallback_run_ids: OrderedDict[int, tuple[object, str]] = OrderedDict()
self._max_fallback_run_ids = max(1, self.max_tracked_threads * 2)
self._history: OrderedDict[_RunScopeKey, list[str]] = OrderedDict()
self._warned: dict[_RunScopeKey, set[str]] = defaultdict(set)
# Windowed per-tool-type frequency: recent tool names per run scope,
# trimmed to ``window_size`` so the count decays instead of growing # trimmed to ``window_size`` so the count decays instead of growing
# monotonically (replaces the old monotonic ``_tool_freq`` integer). # monotonically (replaces the old monotonic ``_tool_freq`` integer).
self._tool_name_history: defaultdict[str, deque[str]] = defaultdict(deque) self._tool_name_history: defaultdict[_RunScopeKey, deque[str]] = defaultdict(deque)
# Per-thread Counter mirroring the deque so freq_count is O(1) instead # Per-run Counter mirroring the deque so freq_count is O(1) instead
# of scanning the whole window on every tool call. A single high # of scanning the whole window on every tool call. A single high
# per-tool override (e.g. bash: {hard_limit: 1000}) inflates the window # per-tool override (e.g. bash: {hard_limit: 1000}) inflates the window
# globally, so the scan would cost 1000 per call for every tool; Counter # globally, so the scan would cost 1000 per call for every tool; Counter
# increments on append and decrements on popleft. # increments on append and decrements on popleft.
self._tool_name_counter: defaultdict[str, Counter[str]] = defaultdict(Counter) self._tool_name_counter: defaultdict[_RunScopeKey, Counter[str]] = defaultdict(Counter)
# Per-thread set of tool names already warned about in Layer 2, so a # Per-run set of tool names already warned about in Layer 2, so a
# frequency warning is enqueued once rather than on every subsequent # frequency warning is enqueued once rather than on every subsequent
# call. Cleared per name when the windowed count decays back below the # call. Cleared per name when the windowed count decays back below the
# warn threshold, mirroring the hash-layer ``_warned`` pruning. # warn threshold, mirroring the hash-layer ``_warned`` pruning.
self._tool_freq_warned: dict[str, set[str]] = defaultdict(set) self._tool_freq_warned: dict[_RunScopeKey, set[str]] = defaultdict(set)
# Per-thread/run queue of warnings to inject at the next model call. # Per-thread/run queue of warnings to inject at the next model call.
# Populated by ``after_model`` (detection) and drained by # Populated by ``after_model`` (detection) and drained by
# ``wrap_model_call`` (injection); see module docstring. # ``wrap_model_call`` (injection); see module docstring.
self._pending_warnings: dict[tuple[str, str], list[str]] = defaultdict(list) self._pending_warnings: dict[_RunScopeKey, list[str]] = defaultdict(list)
self._pending_warning_touch_order: OrderedDict[tuple[str, str], None] = OrderedDict() self._pending_warning_touch_order: OrderedDict[_RunScopeKey, None] = OrderedDict()
self._max_pending_warning_keys = max(1, self.max_tracked_threads * 2) self._max_pending_warning_keys = max(1, self.max_tracked_threads * 2)
# Stop reason set when a hard-stop fires (#3875 Phase 2). Keyed by run_id # Stop reason set when a hard-stop fires (#3875 Phase 2). Keyed by run_id
# (matching ``TokenBudgetMiddleware``) and bounded — the lead agent's # (matching ``TokenBudgetMiddleware``) and bounded — the lead agent's
@ -301,8 +322,11 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
# an entry would accumulate for every looped lead run. Intentionally NOT # an entry would accumulate for every looped lead run. Intentionally NOT
# cleared by ``after_agent``/``_clear_current_run_pending_warnings`` so # cleared by ``after_agent``/``_clear_current_run_pending_warnings`` so
# the subagent executor can consume it after the run returns; ``reset()`` # the subagent executor can consume it after the run returns; ``reset()``
# still drops it. # still drops it. The parallel bounded owner map lets
self._stop_reason: BoundedDict[str, str] = BoundedDict(1000) # ``reset(thread_id)`` remove only that thread's unconsumed reasons;
# both maps are written, evicted, and popped together under ``_lock``.
self._stop_reason: BoundedDict[str | None, str] = BoundedDict(1000)
self._stop_reason_thread_id: BoundedDict[str | None, str] = BoundedDict(1000)
def release_policy_parameters(self) -> dict[str, object]: def release_policy_parameters(self) -> dict[str, object]:
return { return {
@ -335,31 +359,65 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
return str(thread_id) return str(thread_id)
return "default" return "default"
def _get_run_id(self, runtime: Runtime) -> str: def _get_run_id(self, runtime: Runtime) -> str | None:
"""Extract run_id from runtime context for per-run warning scoping. """Extract run_id from runtime context for per-run warning scoping.
Keyed by presence, not truthiness: ``SubagentExecutor`` sets Context presence is authoritative, including an explicit ``None``:
``context["run_id"] = self.run_id`` unconditionally (no truthiness ``SubagentExecutor`` later consumes the stop reason with its raw,
guard), so an embedded/TUI-dispatched subagent whose ``run_id`` is possibly-None run_id, so normalizing that value would lose the signal.
never assigned per ``AGENTS.md``'s description of the embedded
``DeerFlowClient`` runs with a context that legitimately carries A RunnableConfig run_id exposed through ``Runtime.execution_info`` is
``run_id=None`` (the key is *present*, not absent). The executor the next-best stable identifier. If neither source provides one, use
later reads the stop reason back with the raw attribute, LangGraph's run-scoped ``Runtime.control`` object as the invocation
``consume_stop_reason(self.run_id)``, so this must return exactly anchor. LangGraph creates replacement Runtime wrappers per graph node,
that value (``None`` included) when the key is present, rather than but preserves that control object across the invocation. The anchor is
collapsing it to a shared fallback indistinguishable from an absent mapped to an opaque generated token instead of embedding ``id(anchor)``
key. A truthiness check (``if run_id:``) previously conflated in the key, because CPython may reuse an address after garbage
"present but None/falsy" with "absent", both mapping to the same collection. The bounded map keeps a strong reference while active and
literal ``"default"`` so a genuine ``run_id=None`` hard-stop was is released by ``after_agent`` on the normal completion path.
recorded under ``"default"`` here but looked up under ``None`` by
the executor, silently losing the ``loop_capped`` stop reason.
Mirrors ``TokenBudgetMiddleware._get_run_id``.
""" """
ctx = getattr(runtime, "context", None) ctx = getattr(runtime, "context", None)
if isinstance(ctx, dict) and "run_id" in ctx: if isinstance(ctx, dict) and "run_id" in ctx:
return ctx["run_id"] return ctx["run_id"]
# Fallback to runtime object ID to prevent collisions across embedded client runs
return str(id(runtime)) execution_info = getattr(runtime, "execution_info", None)
execution_run_id = getattr(execution_info, "run_id", None)
if execution_run_id is not None:
return str(execution_run_id)
control = getattr(runtime, "control", None)
anchor = control if control is not None else runtime
anchor_id = id(anchor)
with self._lock:
existing = self._fallback_run_ids.get(anchor_id)
if existing is not None and existing[0] is anchor:
self._fallback_run_ids.move_to_end(anchor_id)
return existing[1]
fallback_run_id = f"__invocation__:{uuid.uuid4().hex}"
self._fallback_run_ids[anchor_id] = (anchor, fallback_run_id)
self._fallback_run_ids.move_to_end(anchor_id)
while len(self._fallback_run_ids) > self._max_fallback_run_ids:
self._fallback_run_ids.popitem(last=False)
return fallback_run_id
def _release_fallback_run_id(self, runtime: Runtime) -> None:
"""Release a completed invocation's fallback anchor, if it used one."""
ctx = getattr(runtime, "context", None)
if isinstance(ctx, dict) and "run_id" in ctx:
return
execution_info = getattr(runtime, "execution_info", None)
if getattr(execution_info, "run_id", None) is not None:
return
control = getattr(runtime, "control", None)
anchor = control if control is not None else runtime
anchor_id = id(anchor)
with self._lock:
existing = self._fallback_run_ids.get(anchor_id)
if existing is not None and existing[0] is anchor:
self._fallback_run_ids.pop(anchor_id, None)
def consume_stop_reason(self, run_id: str | None) -> str | None: def consume_stop_reason(self, run_id: str | None) -> str | None:
"""Pop and return the stop reason the hard-stop set for this run. """Pop and return the stop reason the hard-stop set for this run.
@ -373,29 +431,36 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
popping keeps the dict from accumulating on a reused instance. popping keeps the dict from accumulating on a reused instance.
""" """
with self._lock: with self._lock:
return self._stop_reason.pop(run_id, None) reason = self._stop_reason.pop(run_id, None)
self._stop_reason_thread_id.pop(run_id, None)
return reason
def _pending_key(self, runtime: Runtime) -> tuple[str, str]: def _run_scope_key(self, runtime: Runtime) -> _RunScopeKey:
"""Return the pending-warning key for the current thread/run.""" """Return the shared tracking key for the current thread/run."""
return self._get_thread_id(runtime), self._get_run_id(runtime) return self._get_thread_id(runtime), self._get_run_id(runtime)
def _pending_key(self, runtime: Runtime) -> _RunScopeKey:
"""Return the pending-warning key for the current thread/run."""
return self._run_scope_key(runtime)
def _evict_if_needed(self) -> None: def _evict_if_needed(self) -> None:
"""Evict least recently used threads if over the limit. """Evict least recently used thread/run scopes if over the limit.
Must be called while holding self._lock. Must be called while holding self._lock.
""" """
while len(self._history) > self.max_tracked_threads: while len(self._history) > self.max_tracked_threads:
evicted_id, _ = self._history.popitem(last=False) evicted_key, _ = self._history.popitem(last=False)
self._warned.pop(evicted_id, None) self._warned.pop(evicted_key, None)
self._tool_name_history.pop(evicted_id, None) self._tool_name_history.pop(evicted_key, None)
self._tool_name_counter.pop(evicted_id, None) self._tool_name_counter.pop(evicted_key, None)
self._tool_freq_warned.pop(evicted_id, None) self._tool_freq_warned.pop(evicted_key, None)
for key in list(self._pending_warnings): self._drop_pending_warning_key_locked(evicted_key)
if key[0] == evicted_id: logger.debug(
self._drop_pending_warning_key_locked(key) "Evicted loop tracking for thread/run scope (LRU)",
logger.debug("Evicted loop tracking for thread %s (LRU)", evicted_id) extra={"thread_id": evicted_key[0], "run_id": evicted_key[1]},
)
def _drop_pending_warning_key_locked(self, key: tuple[str, str]) -> None: def _drop_pending_warning_key_locked(self, key: _RunScopeKey) -> None:
"""Drop all pending-warning bookkeeping for one thread/run key. """Drop all pending-warning bookkeeping for one thread/run key.
Must be called while holding self._lock. Must be called while holding self._lock.
@ -403,7 +468,7 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
self._pending_warnings.pop(key, None) self._pending_warnings.pop(key, None)
self._pending_warning_touch_order.pop(key, None) self._pending_warning_touch_order.pop(key, None)
def _touch_pending_warning_key_locked(self, key: tuple[str, str]) -> None: def _touch_pending_warning_key_locked(self, key: _RunScopeKey) -> None:
"""Mark a pending-warning key as recently used. """Mark a pending-warning key as recently used.
Must be called while holding self._lock. Must be called while holding self._lock.
@ -411,7 +476,7 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
self._pending_warning_touch_order[key] = None self._pending_warning_touch_order[key] = None
self._pending_warning_touch_order.move_to_end(key) self._pending_warning_touch_order.move_to_end(key)
def _prune_pending_warning_state_locked(self, protected_key: tuple[str, str]) -> None: def _prune_pending_warning_state_locked(self, protected_key: _RunScopeKey) -> None:
"""Cap pending-warning state across abnormal or concurrent runs. """Cap pending-warning state across abnormal or concurrent runs.
Must be called while holding self._lock. Must be called while holding self._lock.
@ -464,27 +529,28 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
if not tool_calls: if not tool_calls:
return None return None
thread_id = self._get_thread_id(runtime) scope_key = self._run_scope_key(runtime)
thread_id, run_id = scope_key
call_hash = _hash_tool_calls(tool_calls) call_hash = _hash_tool_calls(tool_calls)
with self._lock: with self._lock:
# Touch / create entry (move to end for LRU) # Touch / create entry (move to end for LRU)
if thread_id in self._history: if scope_key in self._history:
self._history.move_to_end(thread_id) self._history.move_to_end(scope_key)
else: else:
self._history[thread_id] = [] self._history[scope_key] = []
self._evict_if_needed() self._evict_if_needed()
history = self._history[thread_id] history = self._history[scope_key]
history.append(call_hash) history.append(call_hash)
if len(history) > self.window_size: if len(history) > self.window_size:
history[:] = history[-self.window_size :] history[:] = history[-self.window_size :]
warned_hashes = self._warned.get(thread_id) warned_hashes = self._warned.get(scope_key)
if warned_hashes is not None: if warned_hashes is not None:
warned_hashes.intersection_update(history) warned_hashes.intersection_update(history)
if not warned_hashes: if not warned_hashes:
self._warned.pop(thread_id, None) self._warned.pop(scope_key, None)
count = history.count(call_hash) count = history.count(call_hash)
tool_names = [str(tc.get("name") or "?") for tc in tool_calls] tool_names = [str(tc.get("name") or "?") for tc in tool_calls]
@ -495,6 +561,7 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
"Loop hard limit reached — forcing stop", "Loop hard limit reached — forcing stop",
extra={ extra={
"thread_id": thread_id, "thread_id": thread_id,
"run_id": run_id,
"call_hash": call_hash, "call_hash": call_hash,
"count": count, "count": count,
"tools": tool_names, "tools": tool_names,
@ -515,7 +582,7 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
# admitted call has been checked. Only the selected warning is # admitted call has been checked. Only the selected warning is
# marked/logged; a hard stop may supersede it below. # marked/logged; a hard stop may supersede it below.
warning: _LoopDecision | None = None warning: _LoopDecision | None = None
if count >= self.warn_threshold and call_hash not in self._warned.get(thread_id, set()): if count >= self.warn_threshold and call_hash not in self._warned.get(scope_key, set()):
warning = _LoopDecision( warning = _LoopDecision(
message=_WARNING_MSG, message=_WARNING_MSG,
action="warn", action="warn",
@ -526,8 +593,8 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
) )
# --- Layer 2: per-tool-type frequency (windowed) --- # --- Layer 2: per-tool-type frequency (windowed) ---
tool_name_history = self._tool_name_history[thread_id] tool_name_history = self._tool_name_history[scope_key]
name_counter = self._tool_name_counter[thread_id] name_counter = self._tool_name_counter[scope_key]
for tc in tool_calls: for tc in tool_calls:
name = tc.get("name", "") name = tc.get("name", "")
if not name: if not name:
@ -551,7 +618,7 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
# Any tool can evict an older name from the shared # Any tool can evict an older name from the shared
# window. Rearm that name as soon as its burst decays, # window. Rearm that name as soon as its burst decays,
# even when the current call belongs to another tool. # even when the current call belongs to another tool.
self._tool_freq_warned[thread_id].discard(old) self._tool_freq_warned[scope_key].discard(old)
freq_count = name_counter.get(name, 0) freq_count = name_counter.get(name, 0)
eff_warn, eff_hard = self._tool_freq_overrides.get(name, self._default_tool_freq_thresholds) eff_warn, eff_hard = self._tool_freq_overrides.get(name, self._default_tool_freq_thresholds)
@ -561,6 +628,7 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
"Tool frequency hard limit reached — forcing stop", "Tool frequency hard limit reached — forcing stop",
extra={ extra={
"thread_id": thread_id, "thread_id": thread_id,
"run_id": run_id,
"tool_name": name, "tool_name": name,
"count": freq_count, "count": freq_count,
}, },
@ -575,7 +643,7 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
) )
if freq_count >= eff_warn: if freq_count >= eff_warn:
freq_warned = self._tool_freq_warned[thread_id] freq_warned = self._tool_freq_warned[scope_key]
if warning is None and name not in freq_warned: if warning is None and name not in freq_warned:
warning = _LoopDecision( warning = _LoopDecision(
message=_TOOL_FREQ_WARNING_MSG.format(tool_name=name, count=freq_count), message=_TOOL_FREQ_WARNING_MSG.format(tool_name=name, count=freq_count),
@ -588,15 +656,16 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
else: else:
# Windowed count decayed below the warn threshold; allow a # Windowed count decayed below the warn threshold; allow a
# future burst of this tool to warn again. # future burst of this tool to warn again.
self._tool_freq_warned[thread_id].discard(name) self._tool_freq_warned[scope_key].discard(name)
if warning is not None: if warning is not None:
if warning.detection_layer == "identical_call_set": if warning.detection_layer == "identical_call_set":
self._warned[thread_id].add(call_hash) self._warned[scope_key].add(call_hash)
logger.warning( logger.warning(
"Repetitive tool calls detected — injecting warning", "Repetitive tool calls detected — injecting warning",
extra={ extra={
"thread_id": thread_id, "thread_id": thread_id,
"run_id": run_id,
"call_hash": call_hash, "call_hash": call_hash,
"count": warning.count, "count": warning.count,
"tools": list(warning.tool_names), "tools": list(warning.tool_names),
@ -607,11 +676,12 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
# Later calls in this batch may already have decayed this # Later calls in this batch may already have decayed this
# burst. Do not suppress the next burst with a stale mark. # burst. Do not suppress the next burst with a stale mark.
if name_counter.get(warned_name, 0) >= warning.threshold: if name_counter.get(warned_name, 0) >= warning.threshold:
self._tool_freq_warned[thread_id].add(warned_name) self._tool_freq_warned[scope_key].add(warned_name)
logger.warning( logger.warning(
"Tool frequency warning — too many calls to same tool type", "Tool frequency warning — too many calls to same tool type",
extra={ extra={
"thread_id": thread_id, "thread_id": thread_id,
"run_id": run_id,
"tool_name": warned_name, "tool_name": warned_name,
"count": warning.count, "count": warning.count,
}, },
@ -712,9 +782,10 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
# Written under the lock to match ``TokenBudgetMiddleware``: the lead # Written under the lock to match ``TokenBudgetMiddleware``: the lead
# agent's middleware instance is shared across concurrent Gateway # agent's middleware instance is shared across concurrent Gateway
# threads, so the bounded-dict write needs the same guard. # threads, so the bounded-dict write needs the same guard.
run_id = self._get_run_id(runtime) thread_id, run_id = self._run_scope_key(runtime)
with self._lock: with self._lock:
self._stop_reason[run_id] = "loop_capped" self._stop_reason[run_id] = "loop_capped"
self._stop_reason_thread_id[run_id] = thread_id
# Also write to runtime.context so the lead worker can read it # Also write to runtime.context so the lead worker can read it
# without needing a reference to this middleware instance (#4176). # without needing a reference to this middleware instance (#4176).
ctx = getattr(runtime, "context", None) ctx = getattr(runtime, "context", None)
@ -743,14 +814,6 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
return None return None
def _clear_other_run_pending_warnings(self, runtime: Runtime) -> None:
"""Drop stale pending warnings for previous runs in this thread."""
thread_id, current_run_id = self._pending_key(runtime)
with self._lock:
for key in list(self._pending_warnings):
if key[0] == thread_id and key[1] != current_run_id:
self._drop_pending_warning_key_locked(key)
def _clear_current_run_pending_warnings(self, runtime: Runtime) -> None: def _clear_current_run_pending_warnings(self, runtime: Runtime) -> None:
"""Drop pending warnings owned by the current thread/run.""" """Drop pending warnings owned by the current thread/run."""
pending_key = self._pending_key(runtime) pending_key = self._pending_key(runtime)
@ -765,12 +828,14 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
@override @override
def before_agent(self, state: AgentState, runtime: Runtime) -> dict | None: def before_agent(self, state: AgentState, runtime: Runtime) -> dict | None:
self._clear_other_run_pending_warnings(runtime) # Keep this hook in the compiled graph topology. Pending warnings are
# already run-scoped; touching sibling runs here would break overlapping
# invocations, while after_agent and the bounded queue own cleanup.
return None return None
@override @override
async def abefore_agent(self, state: AgentState, runtime: Runtime) -> dict | None: async def abefore_agent(self, state: AgentState, runtime: Runtime) -> dict | None:
self._clear_other_run_pending_warnings(runtime) # Async topology must mirror the sync hook above.
return None return None
@override @override
@ -784,11 +849,13 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
@override @override
def after_agent(self, state: AgentState, runtime: Runtime) -> dict | None: def after_agent(self, state: AgentState, runtime: Runtime) -> dict | None:
self._clear_current_run_pending_warnings(runtime) self._clear_current_run_pending_warnings(runtime)
self._release_fallback_run_id(runtime)
return None return None
@override @override
async def aafter_agent(self, state: AgentState, runtime: Runtime) -> dict | None: async def aafter_agent(self, state: AgentState, runtime: Runtime) -> dict | None:
self._clear_current_run_pending_warnings(runtime) self._clear_current_run_pending_warnings(runtime)
self._release_fallback_run_id(runtime)
return None return None
def _drain_pending_warnings(self, runtime: Runtime) -> list[str]: def _drain_pending_warnings(self, runtime: Runtime) -> list[str]:
@ -838,14 +905,24 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
"""Clear tracking state. If thread_id given, clear only that thread.""" """Clear tracking state. If thread_id given, clear only that thread."""
with self._lock: with self._lock:
if thread_id: if thread_id:
self._history.pop(thread_id, None) for mapping in (
self._warned.pop(thread_id, None) self._history,
self._tool_name_history.pop(thread_id, None) self._warned,
self._tool_name_counter.pop(thread_id, None) self._tool_name_history,
self._tool_freq_warned.pop(thread_id, None) self._tool_name_counter,
for key in list(self._pending_warnings): self._tool_freq_warned,
):
for key in list(mapping):
if key[0] == thread_id:
mapping.pop(key, None)
pending_keys = set(self._pending_warnings) | set(self._pending_warning_touch_order)
for key in pending_keys:
if key[0] == thread_id: if key[0] == thread_id:
self._drop_pending_warning_key_locked(key) self._drop_pending_warning_key_locked(key)
for run_id, owner_thread_id in list(self._stop_reason_thread_id.items()):
if owner_thread_id == thread_id:
self._stop_reason.pop(run_id, None)
self._stop_reason_thread_id.pop(run_id, None)
else: else:
self._history.clear() self._history.clear()
self._warned.clear() self._warned.clear()
@ -855,3 +932,5 @@ class LoopDetectionMiddleware(AgentMiddleware[AgentState]):
self._pending_warnings.clear() self._pending_warnings.clear()
self._pending_warning_touch_order.clear() self._pending_warning_touch_order.clear()
self._stop_reason.clear() self._stop_reason.clear()
self._stop_reason_thread_id.clear()
self._fallback_run_ids.clear()

View File

@ -473,16 +473,14 @@ class ToolProgressMiddleware(AgentMiddleware[AgentState]):
the previous run are also cleared so a single first-call problem in the new run the previous run are also cleared so a single first-call problem in the new run
cannot falsely trip WARNED against stale context from a run the model no longer sees. cannot falsely trip WARNED against stale context from a run the model no longer sees.
**Cross-run scoping vs LoopDetectionMiddleware**: this per-run reset is an intentional **Graph-entry scoping vs LoopDetectionMiddleware**: this reset at every
policy choice, not an oversight. Errors like ``rate_limited`` and ``transient`` are ``before_agent`` is an intentional policy choice, not an oversight. Errors like
time-bound: their root cause may resolve between user turns, so carrying a stale ``rate_limited`` and ``transient`` are time-bound, so carrying a stale counter into a
counter forward risks a false-positive BLOCKED on calls that would now succeed. later graph entry risks a false-positive BLOCKED on calls that would now succeed.
LoopDetectionMiddleware takes the opposite stance it retains ``_history`` across LoopDetectionMiddleware instead keys call-pattern state by ``(thread_id, run_id)``:
runs (only clearing other-run *pending* warnings at ``before_agent``), because separate user runs are isolated even on a cached graph, while repeated graph entries
call-pattern loops are time-invariant: a model that keeps issuing the same tool_calls in one Gateway run (including hidden goal continuations) share a loop budget. The two
regardless of results does so regardless of when the run started. The two middlewares middlewares therefore guard different failure modes and use different lifetimes.
therefore guard different failure modes (result quality vs. call pattern) and their
cross-run scoping policies intentionally differ as a consequence.
""" """
thread_id = self._thread_id(runtime) thread_id = self._thread_id(runtime)
with self._lock: with self._lock:

View File

@ -46,7 +46,7 @@ class LoopDetectionConfig(BaseModel):
max_tracked_threads: int = Field( max_tracked_threads: int = Field(
default=100, default=100,
ge=1, ge=1,
description="Maximum number of thread histories to keep in memory", description="Maximum number of thread/run histories to keep in memory (legacy field name)",
) )
tool_freq_warn: int = Field( tool_freq_warn: int = Field(
default=30, default=30,

View File

@ -14,6 +14,7 @@ from langchain_core.runnables import Runnable
from langchain_core.tools import tool as as_tool from langchain_core.tools import tool as as_tool
from pydantic import PrivateAttr from pydantic import PrivateAttr
from deerflow.agents.middlewares import loop_detection_middleware as loop_detection_module
from deerflow.agents.middlewares.loop_detection_middleware import ( from deerflow.agents.middlewares.loop_detection_middleware import (
_HARD_STOP_MSG, _HARD_STOP_MSG,
_MAX_PENDING_WARNINGS_PER_RUN, _MAX_PENDING_WARNINGS_PER_RUN,
@ -33,6 +34,10 @@ def _pending_key(thread_id="test-thread", run_id="test-run"):
return (thread_id, run_id) return (thread_id, run_id)
def _scope_key(thread_id="test-thread", run_id="test-run"):
return (thread_id, run_id)
def _make_request(messages, runtime): def _make_request(messages, runtime):
"""Build a minimal ModelRequest stand-in for wrap_model_call tests.""" """Build a minimal ModelRequest stand-in for wrap_model_call tests."""
request = MagicMock() request = MagicMock()
@ -307,21 +312,20 @@ class TestLoopDetection:
def test_missing_run_id_uses_per_runtime_pending_scope(self): def test_missing_run_id_uses_per_runtime_pending_scope(self):
"""When runtime.context has no ``run_id`` key at all, warning handling """When runtime.context has no ``run_id`` key at all, warning handling
falls back to a key scoped to the runtime object's identity — falls back to a key scoped to the LangGraph invocation instead of a
mirroring ``TokenBudgetMiddleware._get_run_id``'s fallback — instead shared literal like the old ``"default"``."""
of a shared literal like the old ``"default"``, which would collide
across concurrent runs that both lack a run_id (the ``_stop_reason``
dict this same key derivation feeds is keyed by run_id alone, with
no thread scoping)."""
mw = LoopDetectionMiddleware(warn_threshold=3, hard_limit=10) mw = LoopDetectionMiddleware(warn_threshold=3, hard_limit=10)
runtime = MagicMock() runtime = SimpleNamespace(
runtime.context = {"thread_id": "test-thread"} context={"thread_id": "test-thread"},
control=object(),
execution_info=None,
)
call = [_bash_call("ls")] call = [_bash_call("ls")]
for _ in range(3): for _ in range(3):
mw._apply(_make_state(tool_calls=call), runtime) mw._apply(_make_state(tool_calls=call), runtime)
fallback_run_id = str(id(runtime)) fallback_run_id = mw._get_run_id(runtime)
assert mw._pending_warnings.get(_pending_key(run_id=fallback_run_id)) assert mw._pending_warnings.get(_pending_key(run_id=fallback_run_id))
request = _make_request([AIMessage(content="hi")], runtime) request = _make_request([AIMessage(content="hi")], runtime)
@ -333,8 +337,59 @@ class TestLoopDetection:
assert "LOOP DETECTED" in loop_warnings[0].content assert "LOOP DETECTED" in loop_warnings[0].content
assert not mw._pending_warnings.get(_pending_key(run_id=fallback_run_id)) assert not mw._pending_warnings.get(_pending_key(run_id=fallback_run_id))
def test_before_agent_clears_stale_pending_warnings_for_thread(self): def test_missing_run_id_shares_scope_across_runtime_wrappers(self):
"""Starting a new run drops stale warnings from prior runs in the same thread.""" """LangGraph replaces ``Runtime`` per node but preserves ``control``."""
mw = LoopDetectionMiddleware()
invocation_control = object()
first_runtime = SimpleNamespace(
context={"thread_id": "test-thread"},
control=invocation_control,
execution_info=None,
)
later_runtime = SimpleNamespace(
context={"thread_id": "test-thread"},
control=invocation_control,
execution_info=None,
)
assert mw._get_run_id(first_runtime) == mw._get_run_id(later_runtime)
def test_missing_run_id_fallback_survives_reused_object_address(self, monkeypatch):
"""A later invocation must not inherit a freed anchor's fallback key."""
mw = LoopDetectionMiddleware()
monkeypatch.setattr(loop_detection_module, "id", lambda _value: 42, raising=False)
first_runtime = SimpleNamespace(
context={"thread_id": "test-thread"},
control=object(),
execution_info=None,
)
first_run_id = mw._get_run_id(first_runtime)
mw.after_agent({"messages": []}, first_runtime)
assert not mw._fallback_run_ids
later_runtime = SimpleNamespace(
context={"thread_id": "test-thread"},
control=object(),
execution_info=None,
)
assert mw._get_run_id(later_runtime) != first_run_id
def test_missing_run_id_fallback_map_is_bounded_on_abnormal_exits(self):
mw = LoopDetectionMiddleware(max_tracked_threads=2)
for _ in range(10):
runtime = SimpleNamespace(
context={"thread_id": "test-thread"},
control=object(),
execution_info=None,
)
mw._get_run_id(runtime)
assert len(mw._fallback_run_ids) == mw._max_fallback_run_ids == 4
def test_before_agent_preserves_pending_warning_for_sibling_run(self):
"""An overlapping run must not erase a warning owned by another run."""
mw = LoopDetectionMiddleware(warn_threshold=3, hard_limit=10) mw = LoopDetectionMiddleware(warn_threshold=3, hard_limit=10)
runtime_a = _make_runtime(run_id="run-A") runtime_a = _make_runtime(run_id="run-A")
runtime_b = _make_runtime(run_id="run-B") runtime_b = _make_runtime(run_id="run-B")
@ -345,6 +400,13 @@ class TestLoopDetection:
assert mw._pending_warnings.get(_pending_key(run_id="run-A")) assert mw._pending_warnings.get(_pending_key(run_id="run-A"))
mw.before_agent({"messages": []}, runtime_b) mw.before_agent({"messages": []}, runtime_b)
assert mw._pending_warnings.get(_pending_key(run_id="run-A"))
request = _make_request([AIMessage(content="hi")], runtime_a)
captured, handler = _capture_handler()
mw.wrap_model_call(request, handler)
loop_warnings = [message for message in captured[0].messages if isinstance(message, HumanMessage) and message.name == "loop_warning"]
assert len(loop_warnings) == 1
assert not mw._pending_warnings.get(_pending_key(run_id="run-A")) assert not mw._pending_warnings.get(_pending_key(run_id="run-A"))
def test_after_agent_clears_current_run_pending_warnings(self): def test_after_agent_clears_current_run_pending_warnings(self):
@ -580,9 +642,9 @@ class TestLoopDetection:
runtime_new = _make_runtime("thread-new") runtime_new = _make_runtime("thread-new")
mw._apply(_make_state(tool_calls=call), runtime_new) mw._apply(_make_state(tool_calls=call), runtime_new)
assert "thread-0" not in mw._history assert _scope_key("thread-0") not in mw._history
assert "thread-0" not in mw._tool_name_history assert _scope_key("thread-0") not in mw._tool_name_history
assert "thread-new" in mw._history assert _scope_key("thread-new") in mw._history
assert len(mw._history) == 3 assert len(mw._history) == 3
def test_warned_hashes_are_pruned_to_sliding_window(self): def test_warned_hashes_are_pruned_to_sliding_window(self):
@ -595,9 +657,9 @@ class TestLoopDetection:
mw._apply(_make_state(tool_calls=call), runtime) mw._apply(_make_state(tool_calls=call), runtime)
mw._apply(_make_state(tool_calls=call), runtime) mw._apply(_make_state(tool_calls=call), runtime)
assert len(mw._history["test-thread"]) <= 4 assert len(mw._history[_scope_key()]) <= 4
assert set(mw._warned["test-thread"]).issubset(set(mw._history["test-thread"])) assert set(mw._warned[_scope_key()]).issubset(set(mw._history[_scope_key()]))
assert len(mw._warned["test-thread"]) <= 4 assert len(mw._warned[_scope_key()]) <= 4
def test_pending_warning_keys_are_capped(self): def test_pending_warning_keys_are_capped(self):
"""Abnormal same-thread runs cannot grow pending-warning keys forever.""" """Abnormal same-thread runs cannot grow pending-warning keys forever."""
@ -644,12 +706,190 @@ class TestLoopDetection:
def test_fallback_thread_id_when_missing(self): def test_fallback_thread_id_when_missing(self):
"""When runtime context has no thread_id, should use 'default'.""" """When runtime context has no thread_id, should use 'default'."""
mw = LoopDetectionMiddleware(warn_threshold=2) mw = LoopDetectionMiddleware(warn_threshold=2)
runtime = MagicMock() runtime = SimpleNamespace(context={}, control=object(), execution_info=None)
runtime.context = {}
call = [_bash_call("ls")] call = [_bash_call("ls")]
mw._apply(_make_state(tool_calls=call), runtime) mw._apply(_make_state(tool_calls=call), runtime)
assert "default" in mw._history assert ("default", mw._get_run_id(runtime)) in mw._history
class TestRunScopedTracking:
def test_identical_call_history_isolated_between_runs_on_same_thread(self):
"""Separate user runs on a cached graph must not share repeat counts."""
mw = LoopDetectionMiddleware(
warn_threshold=2,
hard_limit=3,
tool_freq_warn=100,
tool_freq_hard_limit=200,
)
first_run = _make_runtime("shared-thread", "run-1")
second_run = _make_runtime("shared-thread", "run-2")
state = _make_state(tool_calls=[_bash_call("pwd")])
assert mw._apply(state, first_run) is None
mw.after_agent({"messages": []}, first_run)
assert mw._apply(state, second_run) is None
assert not mw._pending_warnings.get(_pending_key("shared-thread", "run-2"))
assert mw._history[_scope_key("shared-thread", "run-1")] == mw._history[_scope_key("shared-thread", "run-2")]
assert len(mw._history[_scope_key("shared-thread", "run-1")]) == 1
def test_same_run_accumulates_across_agent_hook_cycles_for_goal_continuation(self):
"""One Gateway run may re-enter the graph for hidden goal continuations."""
mw = LoopDetectionMiddleware(
warn_threshold=2,
hard_limit=10,
tool_freq_warn=100,
tool_freq_hard_limit=200,
)
runtime = _make_runtime("goal-thread", "goal-run")
state = _make_state(tool_calls=[_bash_call("pwd")])
assert mw._apply(state, runtime) is None
mw.after_agent({"messages": []}, runtime)
assert mw._apply(state, runtime) is None
assert mw._pending_warnings.get(_pending_key("goal-thread", "goal-run"))
assert len(mw._history[_scope_key("goal-thread", "goal-run")]) == 2
def test_tool_frequency_history_isolated_between_runs_on_same_thread(self):
mw = LoopDetectionMiddleware(
warn_threshold=100,
hard_limit=200,
tool_freq_warn=2,
tool_freq_hard_limit=3,
)
first_run = _make_runtime("shared-thread", "run-1")
second_run = _make_runtime("shared-thread", "run-2")
assert mw._apply(_make_state(tool_calls=[_bash_call("first")]), first_run) is None
mw.after_agent({"messages": []}, first_run)
assert mw._apply(_make_state(tool_calls=[_bash_call("second")]), second_run) is None
assert not mw._pending_warnings.get(_pending_key("shared-thread", "run-2"))
assert mw._tool_name_counter[_scope_key("shared-thread", "run-1")]["bash"] == 1
assert mw._tool_name_counter[_scope_key("shared-thread", "run-2")]["bash"] == 1
def test_warned_hash_is_run_scoped(self):
"""A warning in one run must not suppress the same warning in a later run."""
mw = LoopDetectionMiddleware(
warn_threshold=2,
hard_limit=100,
tool_freq_warn=100,
tool_freq_hard_limit=200,
)
state = _make_state(tool_calls=[_bash_call("pwd")])
for run_id in ("run-1", "run-2"):
runtime = _make_runtime("shared-thread", run_id)
assert mw._apply(state, runtime) is None
assert mw._apply(state, runtime) is None
assert mw._pending_warnings.get(_pending_key("shared-thread", run_id))
mw.after_agent({"messages": []}, runtime)
assert len(mw._warned[_scope_key("shared-thread", "run-1")]) == 1
assert len(mw._warned[_scope_key("shared-thread", "run-2")]) == 1
def test_tool_frequency_warning_suppression_is_run_scoped(self):
mw = LoopDetectionMiddleware(
warn_threshold=100,
hard_limit=200,
tool_freq_warn=2,
tool_freq_hard_limit=100,
)
for run_id in ("run-1", "run-2"):
runtime = _make_runtime("shared-thread", run_id)
for suffix in ("first", "second"):
assert mw._apply(_make_state(tool_calls=[_bash_call(f"{run_id}-{suffix}")]), runtime) is None
assert mw._pending_warnings.get(_pending_key("shared-thread", run_id))
mw.after_agent({"messages": []}, runtime)
assert mw._tool_freq_warned[_scope_key("shared-thread", "run-1")] == {"bash"}
assert mw._tool_freq_warned[_scope_key("shared-thread", "run-2")] == {"bash"}
def test_reset_thread_clears_every_run_scope_and_preserves_other_threads(self):
mw = LoopDetectionMiddleware(
warn_threshold=2,
hard_limit=100,
tool_freq_warn=3,
tool_freq_hard_limit=100,
)
target_runs = [
_make_runtime("thread-A", "run-A1"),
_make_runtime("thread-A", "run-A2"),
]
other_run = _make_runtime("thread-B", "run-B1")
state = _make_state(tool_calls=[_bash_call("pwd")])
for runtime in [*target_runs, other_run]:
mw._apply(state, runtime)
mw._apply(state, runtime)
mw.reset(thread_id="thread-A")
expected_keys = {_scope_key("thread-B", "run-B1")}
for mapping in (
mw._history,
mw._warned,
mw._tool_name_history,
mw._tool_name_counter,
mw._tool_freq_warned,
mw._pending_warnings,
mw._pending_warning_touch_order,
):
assert set(mapping) == expected_keys
def test_reset_thread_clears_its_unconsumed_stop_reasons_only(self):
mw = LoopDetectionMiddleware(
warn_threshold=1,
hard_limit=2,
tool_freq_warn=100,
tool_freq_hard_limit=200,
)
runtimes = [
_make_runtime("thread-A", "run-A1"),
_make_runtime("thread-A", "run-A2"),
_make_runtime("thread-B", "run-B1"),
]
state = _make_state(tool_calls=[_bash_call("pwd")])
for runtime in runtimes:
assert mw._apply(state, runtime) is None
result = mw._apply(state, runtime)
assert result is not None
mw.reset(thread_id="thread-A")
assert mw.consume_stop_reason("run-A1") is None
assert mw.consume_stop_reason("run-A2") is None
assert mw.consume_stop_reason("run-B1") == "loop_capped"
def test_lru_evicts_one_run_scope_without_dropping_sibling_run_warning(self):
mw = LoopDetectionMiddleware(
warn_threshold=100,
hard_limit=200,
max_tracked_threads=2,
tool_freq_warn=100,
tool_freq_hard_limit=200,
)
run_1 = _make_runtime("shared-thread", "run-1")
run_2 = _make_runtime("shared-thread", "run-2")
run_3 = _make_runtime("shared-thread", "run-3")
mw._apply(_make_state(tool_calls=[_bash_call("one")]), run_1)
mw._queue_pending_warning(run_1, "run-1 warning")
mw._apply(_make_state(tool_calls=[_bash_call("two")]), run_2)
mw._queue_pending_warning(run_2, "run-2 warning")
mw._apply(_make_state(tool_calls=[_bash_call("one-again")]), run_1)
mw._apply(_make_state(tool_calls=[_bash_call("three")]), run_3)
assert list(mw._history) == [
_scope_key("shared-thread", "run-1"),
_scope_key("shared-thread", "run-3"),
]
assert _scope_key("shared-thread", "run-2") not in mw._tool_name_history
assert _scope_key("shared-thread", "run-2") not in mw._tool_name_counter
assert _pending_key("shared-thread", "run-1") in mw._pending_warnings
assert _pending_key("shared-thread", "run-2") not in mw._pending_warnings
class TestLoopDetectionRunEvents: class TestLoopDetectionRunEvents:
@ -864,6 +1104,94 @@ class TestLoopDetectionRunEvents:
class TestLoopDetectionAgentGraphIntegration: class TestLoopDetectionAgentGraphIntegration:
def test_reused_agent_graph_without_run_id_gets_one_budget_per_invocation(self):
"""Library embedders may omit run_id while reusing one compiled graph."""
@as_tool
def bash(command: str) -> str:
"""Run a fake shell command."""
return f"ran: {command}"
mw = LoopDetectionMiddleware(
warn_threshold=2,
hard_limit=3,
tool_freq_warn=100,
tool_freq_hard_limit=200,
)
model = _CapturingFakeMessagesListChatModel(
responses=[
AIMessage(content="", tool_calls=[{"name": "bash", "id": "run-1-a", "args": {"command": "pwd"}}]),
AIMessage(content="", tool_calls=[{"name": "bash", "id": "run-1-b", "args": {"command": "pwd"}}]),
AIMessage(content="first final answer"),
AIMessage(content="", tool_calls=[{"name": "bash", "id": "run-2-a", "args": {"command": "pwd"}}]),
AIMessage(content="", tool_calls=[{"name": "bash", "id": "run-2-b", "args": {"command": "pwd"}}]),
AIMessage(content="second final answer"),
],
)
graph = create_agent(model=model, tools=[bash], middleware=[mw])
shared_context = {"thread_id": "library-thread"}
first = graph.invoke(
{"messages": [("user", "where am I?")]},
context=shared_context,
config={"recursion_limit": 15},
)
second = graph.invoke(
{"messages": [("user", "where am I now?")]},
context=shared_context,
config={"recursion_limit": 15},
)
assert first["messages"][-1].content == "first final answer"
assert second["messages"][-1].content == "second final answer"
loop_warnings_by_call = [[message for message in messages if isinstance(message, HumanMessage) and message.name == "loop_warning"] for messages in model.seen_messages]
assert [len(warnings) for warnings in loop_warnings_by_call] == [0, 0, 1, 0, 0, 1]
assert not mw._fallback_run_ids
def test_reused_agent_graph_isolates_loop_history_between_runs(self):
"""A cached graph must give each new run a fresh loop-detection budget."""
@as_tool
def bash(command: str) -> str:
"""Run a fake shell command."""
return f"ran: {command}"
mw = LoopDetectionMiddleware(
warn_threshold=2,
hard_limit=3,
tool_freq_warn=100,
tool_freq_hard_limit=200,
)
model = _CapturingFakeMessagesListChatModel(
responses=[
AIMessage(content="", tool_calls=[{"name": "bash", "id": "run-1-call", "args": {"command": "pwd"}}]),
AIMessage(content="first final answer"),
AIMessage(content="", tool_calls=[{"name": "bash", "id": "run-2-call", "args": {"command": "pwd"}}]),
AIMessage(content="second final answer"),
],
)
graph = create_agent(model=model, tools=[bash], middleware=[mw])
first = graph.invoke(
{"messages": [("user", "where am I?")]},
context={"thread_id": "cached-thread", "run_id": "run-1"},
config={"recursion_limit": 10},
)
second = graph.invoke(
{"messages": [("user", "where am I now?")]},
context={"thread_id": "cached-thread", "run_id": "run-2"},
config={"recursion_limit": 10},
)
assert first["messages"][-1].content == "first final answer"
assert second["messages"][-1].content == "second final answer"
assert len(model.seen_messages) == 4
assert not any(isinstance(message, HumanMessage) and message.name == "loop_warning" for request_messages in model.seen_messages for message in request_messages)
assert set(mw._history) == {
_scope_key("cached-thread", "run-1"),
_scope_key("cached-thread", "run-2"),
}
def test_loop_warning_is_transient_in_real_agent_graph(self): def test_loop_warning_is_transient_in_real_agent_graph(self):
"""after_model queues the warning; wrap_model_call injects it request-only.""" """after_model queues the warning; wrap_model_call injects it request-only."""
@ -1263,7 +1591,7 @@ class TestToolFrequencyDetection:
# Reset only thread-A # Reset only thread-A
mw.reset(thread_id="thread-A") mw.reset(thread_id="thread-A")
assert "thread-A" not in mw._tool_name_history assert _scope_key("thread-A") not in mw._tool_name_history
# thread-B state should still be intact — 3rd call queues a warn. # thread-B state should still be intact — 3rd call queues a warn.
result = mw._apply(_make_state(tool_calls=[self._read_call("/b_2.py")]), runtime_b) result = mw._apply(_make_state(tool_calls=[self._read_call("/b_2.py")]), runtime_b)
@ -1310,8 +1638,8 @@ class TestToolFrequencyDetection:
# Two other threads push it out of the LRU window (max_tracked_threads=2). # Two other threads push it out of the LRU window (max_tracked_threads=2).
mw._apply(_make_state(tool_calls=[_bash_call("ls")]), _make_runtime("thread-a")) mw._apply(_make_state(tool_calls=[_bash_call("ls")]), _make_runtime("thread-a"))
mw._apply(_make_state(tool_calls=[_bash_call("ls")]), _make_runtime("thread-b")) mw._apply(_make_state(tool_calls=[_bash_call("ls")]), _make_runtime("thread-b"))
assert "thread-evicted" not in mw._tool_name_history assert _scope_key("thread-evicted") not in mw._tool_name_history
assert "thread-evicted" not in mw._tool_name_counter assert _scope_key("thread-evicted") not in mw._tool_name_counter
# Thread id reused: its first fresh read_file must not force-stop. # Thread id reused: its first fresh read_file must not force-stop.
result = mw._apply(_make_state(tool_calls=[self._read_call("/fresh.py")]), evicted) result = mw._apply(_make_state(tool_calls=[self._read_call("/fresh.py")]), evicted)
@ -1327,7 +1655,7 @@ class TestToolFrequencyDetection:
for i in range(2): for i in range(2):
mw._apply(_make_state(tool_calls=[self._read_call(f"/file_{i}.py")]), runtime) mw._apply(_make_state(tool_calls=[self._read_call(f"/file_{i}.py")]), runtime)
mw.reset(thread_id="thread-A") mw.reset(thread_id="thread-A")
assert "thread-A" not in mw._tool_name_counter assert _scope_key("thread-A") not in mw._tool_name_counter
result = mw._apply(_make_state(tool_calls=[self._read_call("/fresh.py")]), runtime) result = mw._apply(_make_state(tool_calls=[self._read_call("/fresh.py")]), runtime)
assert result is None assert result is None
@ -1444,7 +1772,7 @@ class TestToolCallBatchDecisions:
assert result["messages"][0].tool_calls == [] assert result["messages"][0].tool_calls == []
assert mw.consume_stop_reason("test-run") == "loop_capped" assert mw.consume_stop_reason("test-run") == "loop_capped"
assert not mw._pending_warnings assert not mw._pending_warnings
assert not mw._tool_freq_warned.get("test-thread") assert not mw._tool_freq_warned.get(_scope_key())
journal.record_middleware.assert_called_once() journal.record_middleware.assert_called_once()
recorded = journal.record_middleware.call_args.kwargs recorded = journal.record_middleware.call_args.kwargs
assert recorded["action"] == "hard_stop" assert recorded["action"] == "hard_stop"
@ -1466,7 +1794,7 @@ class TestToolCallBatchDecisions:
assert decision is not None and decision.hard_stop assert decision is not None and decision.hard_stop
assert decision.tool_names == ("bash",) assert decision.tool_names == ("bash",)
assert decision.count == 3 assert decision.count == 3
assert "read_file" not in mw._tool_freq_warned["test-thread"] assert "read_file" not in mw._tool_freq_warned[_scope_key()]
def test_identical_call_warning_cannot_mask_frequency_hard_stop(self): def test_identical_call_warning_cannot_mask_frequency_hard_stop(self):
mw = LoopDetectionMiddleware(warn_threshold=2, hard_limit=5, tool_freq_warn=2, tool_freq_hard_limit=3) mw = LoopDetectionMiddleware(warn_threshold=2, hard_limit=5, tool_freq_warn=2, tool_freq_hard_limit=3)
@ -1479,7 +1807,7 @@ class TestToolCallBatchDecisions:
assert decision is not None and decision.hard_stop assert decision is not None and decision.hard_stop
assert decision.detection_layer == "tool_frequency" assert decision.detection_layer == "tool_frequency"
assert decision.count == 3 assert decision.count == 3
assert not mw._warned.get("test-thread") assert not mw._warned.get(_scope_key())
def test_warning_batch_counts_all_calls_and_only_marks_selected_warning(self): def test_warning_batch_counts_all_calls_and_only_marks_selected_warning(self):
mw = LoopDetectionMiddleware(tool_freq_warn=2, tool_freq_hard_limit=10) mw = LoopDetectionMiddleware(tool_freq_warn=2, tool_freq_hard_limit=10)
@ -1490,9 +1818,9 @@ class TestToolCallBatchDecisions:
assert first is not None and first.action == "warn" assert first is not None and first.action == "warn"
assert first.tool_names == ("read_file",) assert first.tool_names == ("read_file",)
assert list(mw._tool_name_history["test-thread"]) == ["read_file"] * 3 + ["bash"] * 2 assert list(mw._tool_name_history[_scope_key()]) == ["read_file"] * 3 + ["bash"] * 2
assert dict(mw._tool_name_counter["test-thread"]) == {"read_file": 3, "bash": 2} assert dict(mw._tool_name_counter[_scope_key()]) == {"read_file": 3, "bash": 2}
assert mw._tool_freq_warned["test-thread"] == {"read_file"} assert mw._tool_freq_warned[_scope_key()] == {"read_file"}
second = mw._track_and_check(_make_state(tool_calls=[self._call("bash", 2)]), runtime) second = mw._track_and_check(_make_state(tool_calls=[self._call("bash", 2)]), runtime)
assert second is not None and second.action == "warn" assert second is not None and second.action == "warn"
@ -1508,8 +1836,8 @@ class TestToolCallBatchDecisions:
hash_warning = mw._track_and_check(_make_state(tool_calls=calls), runtime) hash_warning = mw._track_and_check(_make_state(tool_calls=calls), runtime)
assert hash_warning is not None and hash_warning.detection_layer == "identical_call_set" assert hash_warning is not None and hash_warning.detection_layer == "identical_call_set"
assert mw._tool_name_counter["test-thread"]["read_file"] == 2 assert mw._tool_name_counter[_scope_key()]["read_file"] == 2
assert not mw._tool_freq_warned.get("test-thread") assert not mw._tool_freq_warned.get(_scope_key())
freq_warning = mw._track_and_check(_make_state(tool_calls=[self._call("read_file", 1)]), runtime) freq_warning = mw._track_and_check(_make_state(tool_calls=[self._call("read_file", 1)]), runtime)
assert freq_warning is not None and freq_warning.detection_layer == "tool_frequency" assert freq_warning is not None and freq_warning.detection_layer == "tool_frequency"
@ -1523,8 +1851,8 @@ class TestToolCallBatchDecisions:
first = mw._track_and_check(_make_state(tool_calls=calls), runtime) first = mw._track_and_check(_make_state(tool_calls=calls), runtime)
assert first is not None and first.tool_names == ("a",) assert first is not None and first.tool_names == ("a",)
assert mw._tool_name_counter["test-thread"]["a"] == 1 assert mw._tool_name_counter[_scope_key()]["a"] == 1
assert "a" not in mw._tool_freq_warned["test-thread"] assert "a" not in mw._tool_freq_warned[_scope_key()]
second = mw._track_and_check(_make_state(tool_calls=[self._call("a", 5)]), runtime) second = mw._track_and_check(_make_state(tool_calls=[self._call("a", 5)]), runtime)
assert second is not None and second.action == "warn" assert second is not None and second.action == "warn"
assert second.tool_names == ("a",) assert second.tool_names == ("a",)
@ -1541,8 +1869,8 @@ class TestToolCallBatchDecisions:
second = mw._track_and_check(_make_state(tool_calls=[self._call("b", 3)]), runtime) second = mw._track_and_check(_make_state(tool_calls=[self._call("b", 3)]), runtime)
assert second is not None and second.tool_names == ("b",) assert second is not None and second.tool_names == ("b",)
assert mw._tool_name_counter["test-thread"]["a"] == 1 assert mw._tool_name_counter[_scope_key()]["a"] == 1
assert "a" not in mw._tool_freq_warned["test-thread"] assert "a" not in mw._tool_freq_warned[_scope_key()]
third = mw._track_and_check(_make_state(tool_calls=[self._call("a", 4)]), runtime) third = mw._track_and_check(_make_state(tool_calls=[self._call("a", 4)]), runtime)
assert third is not None and third.tool_names == ("a",) assert third is not None and third.tool_names == ("a",)
@ -1561,15 +1889,15 @@ class TestToolCallBatchDecisions:
runtime, runtime,
) )
assert first is not None and first.tool_names == ("bash",) assert first is not None and first.tool_names == ("bash",)
assert mw._tool_freq_warned["test-thread"] == {"bash"} assert mw._tool_freq_warned[_scope_key()] == {"bash"}
second = mw._track_and_check( second = mw._track_and_check(
_make_state(tool_calls=[self._call("read_file", i) for i in range(3)]), _make_state(tool_calls=[self._call("read_file", i) for i in range(3)]),
runtime, runtime,
) )
assert second is not None and second.tool_names == ("read_file",) assert second is not None and second.tool_names == ("read_file",)
assert mw._tool_name_counter["test-thread"]["bash"] == 2 assert mw._tool_name_counter[_scope_key()]["bash"] == 2
assert "bash" not in mw._tool_freq_warned["test-thread"] assert "bash" not in mw._tool_freq_warned[_scope_key()]
third = mw._track_and_check( third = mw._track_and_check(
_make_state(tool_calls=[self._call("bash", i) for i in range(3, 6)]), _make_state(tool_calls=[self._call("bash", i) for i in range(3, 6)]),

View File

@ -1236,6 +1236,7 @@ loop_detection:
warn_threshold: 3 warn_threshold: 3
hard_limit: 5 hard_limit: 5
window_size: 20 window_size: 20
# Compatibility name: bounds retained (thread_id, run_id) histories, not whole threads.
max_tracked_threads: 100 max_tracked_threads: 100
tool_freq_warn: 30 tool_freq_warn: 30
tool_freq_hard_limit: 50 tool_freq_hard_limit: 50