mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-11 06:28:58 +00:00
fix(summarization): stop fraction triggers from crashing the agent build (#4901)
* fix(summarization): resolve fraction triggers from declared context_window, degrade instead of crashing the agent build A fraction trigger/keep clause requires profile["max_input_tokens"], which any third-party OpenAI-compatible model lacks, so SummarizationMiddleware construction raised ValueError out of create_summarization_middleware and failed the whole agent build (#3103). - factory: translate a declared model context_window into the langchain profile (metadata-only, never reaches the provider payload); explicit caller/override profiles win - summarization factory: drop unusable fraction trigger clauses (absolute clauses survive), fall a fraction keep back to the messages default, and disable compaction with an actionable warning only when no usable trigger clause remains — the agent build never dies from summarization config - docs: config.example.yaml, ModelConfig.context_window, summarization.md * refactor(summarization): share the default keep constant with the fraction fallback The fraction-keep degradation fallback hardcoded ("messages", 20), duplicating SummarizationConfig.keep's default_factory literal. Move the value to a shared DEFAULT_KEEP constant so the two cannot drift apart. * fix(summarization): keep trigger-null + fraction-keep constructing after degradation A trigger of None with a fraction keep hit the all-clauses-dropped branch (has_usable_trigger=False) and disabled compaction, and the accompanying warning claimed configured triggers were all fraction-based when none were configured. Only report nothing-usable when trigger clauses actually existed; trigger:null keeps constructing the never-firing middleware with the degraded keep, matching its behavior outside the degradation path. * fix(summarization): address review — keep manual compaction, validate ContextSize, pin wiring Review follow-ups on #4901: - When every configured trigger is a dropped fraction clause, keep constructing the never-firing middleware (trigger=None) instead of returning None: manual /compact runs with force=True and never consults trigger clauses, so it must keep working for a profile-less model rather than reporting 'compaction is disabled'. The warning now says auto-compaction will not fire while manual compaction remains. - ContextSize gains a config-load validator: fraction values must be in (0,1] (a percent-style 80 instead of 0.8 previously produced a threshold the context could never reach — a silently inert trigger), absolute values must be positive. - New un-monkeypatched integration test pins the shipped wiring (context_window declared -> real factory attaches profile -> fraction clause survives -> middleware constructs), which the stubbed middleware-side tests and kwarg-capturing factory-side tests each stopped short of. - Docs (summarization.md + config.example.yaml) clarify that the fraction resolves against the summary/anchor model's context_window (summarization.model_name when set, else the run model), including the mismatch caveat for a larger-window summary model. * fix(summarization): reject non-finite ContextSize values at config load YAML .nan / .inf pass pydantic's float parsing, and nan <= 0 is False, so the positivity check alone let them through as dead thresholds (count >= nan is always False) — the same silent-inert-trigger class the range validator was added to close. Guard with math.isfinite first, consistent with the existing non-finite guards on mem0 timeout_seconds and poll_after_seconds. * fix(summarization): merge context_window into inferred profile, require whole message counts - construct the model first, then merge max_input_tokens into the provider-inferred langchain profile: passing profile= to the constructor replaced the whole inferred metadata (tool_calling, structured_output, io capabilities, output limits) with the single key. An explicitly configured profile is still never clobbered. - reject non-integral ContextSize values for type=messages at config load: langchain slices the message list with them, so a float index raised TypeError mid-compaction. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
parent
c139ba108f
commit
ae82f426bf
@ -95,6 +95,18 @@ summarization:
|
|||||||
value: 0.8 # 80% of max input tokens
|
value: 0.8 # 80% of max input tokens
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The percentage resolves from the **summary model's** declared `context_window`
|
||||||
|
— the anchor that generates summaries: `summarization.model_name` when set,
|
||||||
|
otherwise the run's own model. Declare `context_window` on that models entry
|
||||||
|
in `config.yaml`. Third-party OpenAI-compatible models carry no built-in
|
||||||
|
capacity profile, so without a declared `context_window` the fraction clause
|
||||||
|
is dropped with a warning at agent build — any remaining absolute clauses
|
||||||
|
(`tokens` / `messages`) keep working. Caveat: when a separate summary model
|
||||||
|
is configured, its window sizes the threshold — a 64k run model paired with
|
||||||
|
a 128k-window summary model resolves `fraction: 0.8` to ~102k tokens and
|
||||||
|
auto-summarization cannot fire before the run model overflows; in that setup
|
||||||
|
prefer absolute `tokens` thresholds sized for the run model.
|
||||||
|
|
||||||
**Multiple Triggers:**
|
**Multiple Triggers:**
|
||||||
```yaml
|
```yaml
|
||||||
trigger:
|
trigger:
|
||||||
|
|||||||
@ -4,6 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import html
|
import html
|
||||||
import logging
|
import logging
|
||||||
|
from collections.abc import Mapping
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from typing import Any, Protocol, override, runtime_checkable
|
from typing import Any, Protocol, override, runtime_checkable
|
||||||
|
|
||||||
@ -18,6 +19,7 @@ from langgraph.runtime import Runtime
|
|||||||
|
|
||||||
from deerflow.agents.middlewares.dynamic_context_middleware import is_dynamic_context_reminder
|
from deerflow.agents.middlewares.dynamic_context_middleware import is_dynamic_context_reminder
|
||||||
from deerflow.config.app_config import get_app_config
|
from deerflow.config.app_config import get_app_config
|
||||||
|
from deerflow.config.summarization_config import DEFAULT_KEEP
|
||||||
from deerflow.extensions.notify import notify_context_compacted
|
from deerflow.extensions.notify import notify_context_compacted
|
||||||
from deerflow.models import create_chat_model
|
from deerflow.models import create_chat_model
|
||||||
from deerflow.utils.messages import is_real_user_message
|
from deerflow.utils.messages import is_real_user_message
|
||||||
@ -799,6 +801,73 @@ def _build_summary_anchor(candidate_names: list[str | None], app_config: Any) ->
|
|||||||
return None, None
|
return None, None
|
||||||
|
|
||||||
|
|
||||||
|
def _anchor_profile_max_input_tokens(model: Any) -> int | None:
|
||||||
|
"""Pre-construction mirror of the parent's ``_get_profile_limits`` validation.
|
||||||
|
|
||||||
|
Same rules the parent will apply moments later: ``model.profile`` must be a
|
||||||
|
``Mapping`` carrying an ``int`` ``max_input_tokens``. Anything else counts as
|
||||||
|
"no usable profile".
|
||||||
|
"""
|
||||||
|
profile = getattr(model, "profile", None)
|
||||||
|
if not isinstance(profile, Mapping):
|
||||||
|
return None
|
||||||
|
max_input_tokens = profile.get("max_input_tokens")
|
||||||
|
return max_input_tokens if isinstance(max_input_tokens, int) else None
|
||||||
|
|
||||||
|
|
||||||
|
def _drop_unusable_fraction_clauses(
|
||||||
|
anchor_model: Any,
|
||||||
|
trigger: Any,
|
||||||
|
keep: tuple[str, int | float],
|
||||||
|
) -> tuple[Any, tuple[str, int | float], bool]:
|
||||||
|
"""Drop fraction clauses the anchor model cannot resolve (no usable profile).
|
||||||
|
|
||||||
|
LangChain's parent constructor raises ``ValueError`` for a fraction clause when
|
||||||
|
``profile["max_input_tokens"]`` is unavailable, which on a third-party
|
||||||
|
OpenAI-compatible model without a declared ``context_window`` would otherwise
|
||||||
|
fail the whole agent build (#3103). Fraction trigger clauses are dropped
|
||||||
|
(absolute clauses survive), and a fraction ``keep`` falls back to the messages
|
||||||
|
default.
|
||||||
|
|
||||||
|
Returns ``(trigger, keep, has_usable_trigger)``; ``has_usable_trigger`` is
|
||||||
|
``False`` only when trigger clauses were configured and every one of them was
|
||||||
|
a dropped fraction clause. A ``trigger`` that was ``None`` to begin with passes
|
||||||
|
through unchanged with ``has_usable_trigger=True``, preserving the long-standing
|
||||||
|
"enabled but never auto-triggers" configuration.
|
||||||
|
"""
|
||||||
|
clauses = list(trigger) if isinstance(trigger, list) else ([] if trigger is None else [trigger])
|
||||||
|
has_fraction_trigger = any(isinstance(clause, tuple) and clause[0] == "fraction" for clause in clauses)
|
||||||
|
keep_is_fraction = isinstance(keep, tuple) and keep[0] == "fraction"
|
||||||
|
if not (has_fraction_trigger or keep_is_fraction):
|
||||||
|
return trigger, keep, True
|
||||||
|
if _anchor_profile_max_input_tokens(anchor_model) is not None:
|
||||||
|
return trigger, keep, True
|
||||||
|
|
||||||
|
kept = [clause for clause in clauses if not (isinstance(clause, tuple) and clause[0] == "fraction")]
|
||||||
|
dropped = [clause for clause in clauses if isinstance(clause, tuple) and clause[0] == "fraction"]
|
||||||
|
if dropped:
|
||||||
|
logger.warning(
|
||||||
|
"Dropped summarization fraction trigger clause(s) %s: the summary model exposes no context window to resolve them against. Declare `context_window` on the model in config.yaml, or use absolute token/message thresholds.",
|
||||||
|
dropped,
|
||||||
|
)
|
||||||
|
new_keep = keep
|
||||||
|
if keep_is_fraction:
|
||||||
|
# The shared constant keeps this fallback identical to SummarizationConfig's
|
||||||
|
# documented default keep.
|
||||||
|
new_keep = DEFAULT_KEEP
|
||||||
|
logger.warning(
|
||||||
|
"Summarization keep %s is unusable without a model context window; falling back to %s. Declare `context_window` on the model in config.yaml to use fraction retention.",
|
||||||
|
keep,
|
||||||
|
new_keep,
|
||||||
|
)
|
||||||
|
if not kept:
|
||||||
|
# No trigger clause survived, but only treat that as "nothing usable" when
|
||||||
|
# clauses were configured at all: a trigger of None keeps constructing the
|
||||||
|
# never-firing middleware, exactly as it does outside this degradation path.
|
||||||
|
return None, new_keep, not clauses
|
||||||
|
return (kept if isinstance(trigger, list) else kept[0]), new_keep, True
|
||||||
|
|
||||||
|
|
||||||
def create_summarization_middleware(
|
def create_summarization_middleware(
|
||||||
*,
|
*,
|
||||||
app_config: Any | None = None,
|
app_config: Any | None = None,
|
||||||
@ -858,10 +927,32 @@ def create_summarization_middleware(
|
|||||||
logger.warning("Summarization is enabled but no summary model could be constructed; compaction is unavailable for this build")
|
logger.warning("Summarization is enabled but no summary model could be constructed; compaction is unavailable for this build")
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
# LangChain's SummarizationMiddleware raises ValueError at construction when a
|
||||||
|
# fraction clause is configured but the anchor exposes no usable profile
|
||||||
|
# (``profile["max_input_tokens"]``) — the default for any third-party
|
||||||
|
# OpenAI-compatible model whose ``context_window`` was not declared in
|
||||||
|
# config.yaml (#3103: `trigger: fraction` used to fail the whole agent build).
|
||||||
|
# Degrade instead: drop the unusable fraction clauses (absolute ones survive)
|
||||||
|
# and fall the keep policy back to its messages default. When every configured
|
||||||
|
# trigger clause is dropped, construction continues with ``trigger=None`` —
|
||||||
|
# the never-firing shape — so manual compaction (``/compact``, which runs with
|
||||||
|
# ``force=True`` and never consults trigger clauses) keeps working for a
|
||||||
|
# profile-less model instead of reporting "compaction is disabled". The factory
|
||||||
|
# attaches a profile from a declared ``context_window``, so this path is
|
||||||
|
# reached only when the model's capacity is genuinely unknown.
|
||||||
|
trigger, keep_tuple, has_usable_trigger = _drop_unusable_fraction_clauses(anchor_model, trigger, keep or config.keep.to_tuple())
|
||||||
|
if not has_usable_trigger:
|
||||||
|
logger.warning(
|
||||||
|
"Every configured summarization trigger is fraction-based but anchor model %r "
|
||||||
|
"exposes no context window (no `context_window` on the model in config.yaml, no provider profile); "
|
||||||
|
"auto-compaction will not fire for this build. Declare `context_window` on the model to enable fraction "
|
||||||
|
"triggers. Manual compaction (/compact) remains available.",
|
||||||
|
anchor_name,
|
||||||
|
)
|
||||||
kwargs: dict[str, Any] = {
|
kwargs: dict[str, Any] = {
|
||||||
"model": anchor_model,
|
"model": anchor_model,
|
||||||
"trigger": trigger,
|
"trigger": trigger,
|
||||||
"keep": keep or config.keep.to_tuple(),
|
"keep": keep_tuple,
|
||||||
}
|
}
|
||||||
if config.trim_tokens_to_summarize is not None:
|
if config.trim_tokens_to_summarize is not None:
|
||||||
kwargs["trim_tokens_to_summarize"] = config.trim_tokens_to_summarize
|
kwargs["trim_tokens_to_summarize"] = config.trim_tokens_to_summarize
|
||||||
|
|||||||
@ -37,9 +37,11 @@ class ModelConfig(BaseModel):
|
|||||||
gt=0,
|
gt=0,
|
||||||
description=(
|
description=(
|
||||||
"Positive total context window size in tokens (prompt + completion). Used to compute the real-time "
|
"Positive total context window size in tokens (prompt + completion). Used to compute the real-time "
|
||||||
"context usage percentage displayed in the chat UI. Distinct from `max_tokens`, which is the "
|
"context usage percentage displayed in the chat UI, and attached to the model's langchain profile "
|
||||||
"per-call output cap passed to the provider. Leave unset if unknown; the UI will hide the "
|
"(`max_input_tokens`) so fraction-based summarization triggers can resolve their thresholds for "
|
||||||
"percentage."
|
"third-party OpenAI-compatible models that carry no built-in profile. Distinct from `max_tokens`, "
|
||||||
|
"which is the per-call output cap passed to the provider. Leave unset if unknown; the UI will hide "
|
||||||
|
"the percentage and fraction summarization clauses will degrade with a warning."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
stream_chunk_timeout: float | None = Field(
|
stream_chunk_timeout: float | None = Field(
|
||||||
|
|||||||
@ -1,11 +1,16 @@
|
|||||||
"""Configuration for conversation summarization."""
|
"""Configuration for conversation summarization."""
|
||||||
|
|
||||||
|
import math
|
||||||
from typing import Literal
|
from typing import Literal
|
||||||
|
|
||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field, model_validator
|
||||||
|
|
||||||
ContextSizeType = Literal["fraction", "tokens", "messages"]
|
ContextSizeType = Literal["fraction", "tokens", "messages"]
|
||||||
DEFAULT_SKILL_FILE_READ_TOOL_NAMES: tuple[str, ...] = ("read_file", "read", "view", "cat")
|
DEFAULT_SKILL_FILE_READ_TOOL_NAMES: tuple[str, ...] = ("read_file", "read", "view", "cat")
|
||||||
|
#: Documented default retention policy after summarization. Shared with the
|
||||||
|
#: summarization middleware's fraction-keep degradation fallback so the two
|
||||||
|
#: cannot drift apart.
|
||||||
|
DEFAULT_KEEP: tuple[ContextSizeType, int] = ("messages", 20)
|
||||||
|
|
||||||
|
|
||||||
class ContextSize(BaseModel):
|
class ContextSize(BaseModel):
|
||||||
@ -14,6 +19,34 @@ class ContextSize(BaseModel):
|
|||||||
type: ContextSizeType = Field(description="Type of context size specification")
|
type: ContextSizeType = Field(description="Type of context size specification")
|
||||||
value: int | float = Field(description="Value for the context size specification")
|
value: int | float = Field(description="Value for the context size specification")
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def _validate_value_range(self) -> "ContextSize":
|
||||||
|
"""Reject value ranges that would silently produce a dead threshold.
|
||||||
|
|
||||||
|
A fraction written percent-style (``value: 80`` instead of ``0.8``) resolves
|
||||||
|
to ``int(max_input_tokens * 80)`` — a threshold the context can never reach,
|
||||||
|
so the trigger silently never fires. Non-finite floats (YAML ``.nan`` /
|
||||||
|
``.inf`` pass pydantic's float parsing) are dead thresholds the same way
|
||||||
|
(``count >= nan`` is always False), and ``nan <= 0`` is False so the
|
||||||
|
positivity check alone would not catch them. Failing at config load turns
|
||||||
|
these foot-guns into actionable errors, consistent with how fraction
|
||||||
|
clauses degrade (loudly) elsewhere. Absolute ``tokens`` values must simply
|
||||||
|
be positive to describe a usable threshold, while ``messages`` values must
|
||||||
|
additionally be whole numbers: langchain slices the message list with them
|
||||||
|
(``messages[-keep:]``), and a float index raises ``TypeError: list indices
|
||||||
|
must be integers or slices, not float`` mid-compaction.
|
||||||
|
"""
|
||||||
|
if not math.isfinite(self.value):
|
||||||
|
raise ValueError(f"ContextSize value must be finite (got {self.value!r})")
|
||||||
|
if self.type == "fraction":
|
||||||
|
if not 0 < self.value <= 1:
|
||||||
|
raise ValueError(f"fraction ContextSize value must be in (0, 1] (got {self.value!r}) — write 0.8 for 80%, not 80")
|
||||||
|
elif self.type == "messages" and not isinstance(self.value, int):
|
||||||
|
raise ValueError(f"messages ContextSize value must be a whole number of messages (got {self.value!r}) — it slices the message list, so a float index would raise TypeError at compaction time")
|
||||||
|
elif self.value <= 0:
|
||||||
|
raise ValueError(f"{self.type} ContextSize value must be positive (got {self.value!r})")
|
||||||
|
return self
|
||||||
|
|
||||||
def to_tuple(self) -> tuple[ContextSizeType, int | float]:
|
def to_tuple(self) -> tuple[ContextSizeType, int | float]:
|
||||||
"""Convert to tuple format expected by SummarizationMiddleware."""
|
"""Convert to tuple format expected by SummarizationMiddleware."""
|
||||||
return (self.type, self.value)
|
return (self.type, self.value)
|
||||||
@ -41,7 +74,7 @@ class SummarizationConfig(BaseModel):
|
|||||||
"{'type': 'fraction', 'value': 0.8} triggers at 80% of model's max input tokens",
|
"{'type': 'fraction', 'value': 0.8} triggers at 80% of model's max input tokens",
|
||||||
)
|
)
|
||||||
keep: ContextSize = Field(
|
keep: ContextSize = Field(
|
||||||
default_factory=lambda: ContextSize(type="messages", value=20),
|
default_factory=lambda: ContextSize(type=DEFAULT_KEEP[0], value=DEFAULT_KEEP[1]),
|
||||||
description="Context retention policy after summarization. Specifies how much history to preserve. "
|
description="Context retention policy after summarization. Specifies how much history to preserve. "
|
||||||
"Examples: {'type': 'messages', 'value': 20} keeps 20 messages, "
|
"Examples: {'type': 'messages', 'value': 20} keeps 20 messages, "
|
||||||
"{'type': 'tokens', 'value': 3000} keeps 3000 tokens, "
|
"{'type': 'tokens', 'value': 3000} keeps 3000 tokens, "
|
||||||
|
|||||||
@ -308,10 +308,29 @@ def create_chat_model(name: str | None = None, thinking_enabled: bool = False, *
|
|||||||
if "stream_usage" in getattr(model_class, "model_fields", {}):
|
if "stream_usage" in getattr(model_class, "model_fields", {}):
|
||||||
model_settings_from_config["stream_usage"] = True
|
model_settings_from_config["stream_usage"] = True
|
||||||
|
|
||||||
|
# Translate the declared context window into the langchain profile so
|
||||||
|
# profile-dependent features (e.g. SummarizationMiddleware fraction triggers,
|
||||||
|
# which resolve thresholds from profile["max_input_tokens"]) work for
|
||||||
|
# third-party OpenAI-compatible models whose SDK ships no profile of its
|
||||||
|
# own (#3103). ``profile`` is a metadata-only BaseChatModel field
|
||||||
|
# (exclude=True) and never reaches the provider request payload. An
|
||||||
|
# explicit profile from a caller or model_overrides is never clobbered.
|
||||||
|
translate_context_window = bool(model_config.context_window) and "profile" not in kwargs and "profile" not in model_settings_from_config
|
||||||
|
|
||||||
_warn_unknown_model_settings(model_class, name, model_settings_from_config)
|
_warn_unknown_model_settings(model_class, name, model_settings_from_config)
|
||||||
|
|
||||||
model_instance = model_class(**kwargs, **model_settings_from_config)
|
model_instance = model_class(**kwargs, **model_settings_from_config)
|
||||||
|
|
||||||
|
if translate_context_window:
|
||||||
|
# Applied *after* construction and merged into the provider's inferred
|
||||||
|
# profile: passing ``profile`` to the constructor would REPLACE the whole
|
||||||
|
# inferred metadata (tool_calling, structured_output, io capabilities,
|
||||||
|
# output limits) with just this one key. The declared window wins on
|
||||||
|
# max_input_tokens itself — the operator set it precisely because the
|
||||||
|
# inferred (or absent) value doesn't match their gateway.
|
||||||
|
inferred_profile = getattr(model_instance, "profile", None)
|
||||||
|
model_instance.profile = {**(inferred_profile or {}), "max_input_tokens": model_config.context_window}
|
||||||
|
|
||||||
if attach_tracing:
|
if attach_tracing:
|
||||||
callbacks = build_tracing_callbacks()
|
callbacks = build_tracing_callbacks()
|
||||||
if callbacks:
|
if callbacks:
|
||||||
|
|||||||
@ -155,6 +155,85 @@ def test_context_window_never_reaches_the_provider_client(monkeypatch):
|
|||||||
assert "context_window" not in FakeChatModel.captured_kwargs
|
assert "context_window" not in FakeChatModel.captured_kwargs
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_window_attaches_langchain_profile(monkeypatch):
|
||||||
|
"""The declared context window is translated into the langchain ``profile``
|
||||||
|
so profile-dependent features (e.g. SummarizationMiddleware fraction triggers,
|
||||||
|
which resolve thresholds from ``profile["max_input_tokens"]``) work for
|
||||||
|
third-party OpenAI-compatible models whose SDK ships no profile of its own
|
||||||
|
(#3103: `trigger: fraction` used to crash the whole agent build)."""
|
||||||
|
model = _make_model("windowed")
|
||||||
|
model.context_window = 200_000
|
||||||
|
cfg = _make_app_config([model])
|
||||||
|
_patch_factory(monkeypatch, cfg)
|
||||||
|
|
||||||
|
FakeChatModel.captured_kwargs = {}
|
||||||
|
created = factory_module.create_chat_model(name="windowed")
|
||||||
|
|
||||||
|
assert "profile" not in FakeChatModel.captured_kwargs
|
||||||
|
assert created.profile == {"max_input_tokens": 200_000}
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_window_merges_into_inferred_profile(monkeypatch):
|
||||||
|
"""A provider-inferred profile must survive the context_window translation:
|
||||||
|
passing ``profile`` to the constructor would REPLACE the whole inferred
|
||||||
|
metadata (tool_calling, structured_output, output limits) with the single
|
||||||
|
key, changing LangChain feature selection. The declared window wins on
|
||||||
|
``max_input_tokens`` itself — the operator declared it because the inferred
|
||||||
|
value doesn't match their gateway."""
|
||||||
|
inferred = {
|
||||||
|
"tool_calling": True,
|
||||||
|
"structured_output": True,
|
||||||
|
"max_output_tokens": 16_384,
|
||||||
|
"max_input_tokens": 999_999,
|
||||||
|
}
|
||||||
|
|
||||||
|
class _InferredProfileChatModel(FakeChatModel):
|
||||||
|
def __init__(self, **kwargs):
|
||||||
|
super().__init__(**kwargs)
|
||||||
|
self.profile = dict(inferred)
|
||||||
|
|
||||||
|
model = _make_model("windowed")
|
||||||
|
model.context_window = 200_000
|
||||||
|
cfg = _make_app_config([model])
|
||||||
|
_patch_factory(monkeypatch, cfg, model_class=_InferredProfileChatModel)
|
||||||
|
|
||||||
|
created = factory_module.create_chat_model(name="windowed")
|
||||||
|
|
||||||
|
assert created.profile == {
|
||||||
|
"tool_calling": True,
|
||||||
|
"structured_output": True,
|
||||||
|
"max_output_tokens": 16_384,
|
||||||
|
"max_input_tokens": 200_000,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_unset_context_window_leaves_profile_unset(monkeypatch):
|
||||||
|
"""No declared window -> no invented profile; fraction triggers degrade
|
||||||
|
with a warning instead of silently assuming a capacity."""
|
||||||
|
cfg = _make_app_config([_make_model("opaque")])
|
||||||
|
_patch_factory(monkeypatch, cfg)
|
||||||
|
|
||||||
|
FakeChatModel.captured_kwargs = {}
|
||||||
|
created = factory_module.create_chat_model(name="opaque")
|
||||||
|
|
||||||
|
assert "profile" not in FakeChatModel.captured_kwargs
|
||||||
|
assert created.profile is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_window_does_not_clobber_explicit_profile(monkeypatch):
|
||||||
|
"""A caller-supplied profile wins over the context_window translation."""
|
||||||
|
model = _make_model("windowed")
|
||||||
|
model.context_window = 200_000
|
||||||
|
cfg = _make_app_config([model])
|
||||||
|
_patch_factory(monkeypatch, cfg)
|
||||||
|
|
||||||
|
FakeChatModel.captured_kwargs = {}
|
||||||
|
created = factory_module.create_chat_model(name="windowed", profile={"max_input_tokens": 100})
|
||||||
|
|
||||||
|
assert FakeChatModel.captured_kwargs.get("profile") == {"max_input_tokens": 100}
|
||||||
|
assert created.profile == {"max_input_tokens": 100}
|
||||||
|
|
||||||
|
|
||||||
def test_appends_all_tracing_callbacks(monkeypatch):
|
def test_appends_all_tracing_callbacks(monkeypatch):
|
||||||
cfg = _make_app_config([_make_model("alpha")])
|
cfg = _make_app_config([_make_model("alpha")])
|
||||||
_patch_factory(monkeypatch, cfg)
|
_patch_factory(monkeypatch, cfg)
|
||||||
|
|||||||
@ -10,13 +10,17 @@ from langchain_core.language_models import BaseChatModel
|
|||||||
from langchain_core.messages import AIMessage, HumanMessage, RemoveMessage, SystemMessage, ToolMessage
|
from langchain_core.messages import AIMessage, HumanMessage, RemoveMessage, SystemMessage, ToolMessage
|
||||||
from langchain_core.outputs import ChatGeneration, ChatResult
|
from langchain_core.outputs import ChatGeneration, ChatResult
|
||||||
from langgraph.constants import TAG_NOSTREAM
|
from langgraph.constants import TAG_NOSTREAM
|
||||||
|
from pydantic import ValidationError
|
||||||
|
|
||||||
from deerflow.agents.memory.summarization_hook import memory_flush_hook
|
from deerflow.agents.memory.summarization_hook import memory_flush_hook
|
||||||
from deerflow.agents.middlewares.dynamic_context_middleware import _DYNAMIC_CONTEXT_REMINDER_KEY, DynamicContextMiddleware, is_dynamic_context_reminder
|
from deerflow.agents.middlewares.dynamic_context_middleware import _DYNAMIC_CONTEXT_REMINDER_KEY, DynamicContextMiddleware, is_dynamic_context_reminder
|
||||||
from deerflow.agents.middlewares.summarization_middleware import DeerFlowSummarizationMiddleware, SummarizationEvent, SummaryGenerationError, create_summarization_middleware
|
from deerflow.agents.middlewares.summarization_middleware import DeerFlowSummarizationMiddleware, SummarizationEvent, SummaryGenerationError, create_summarization_middleware
|
||||||
from deerflow.agents.thread_state import ThreadState
|
from deerflow.agents.thread_state import ThreadState
|
||||||
|
from deerflow.config.app_config import AppConfig
|
||||||
from deerflow.config.memory_config import MemoryConfig
|
from deerflow.config.memory_config import MemoryConfig
|
||||||
from deerflow.config.summarization_config import SummarizationConfig
|
from deerflow.config.model_config import ModelConfig
|
||||||
|
from deerflow.config.sandbox_config import SandboxConfig
|
||||||
|
from deerflow.config.summarization_config import ContextSize, SummarizationConfig
|
||||||
|
|
||||||
|
|
||||||
def _messages() -> list:
|
def _messages() -> list:
|
||||||
@ -1077,11 +1081,11 @@ def test_before_summarization_hook_not_fired_when_summary_fails(monkeypatch: pyt
|
|||||||
assert captured == []
|
assert captured == []
|
||||||
|
|
||||||
|
|
||||||
def _factory_app_config(model_names, *, summary_model_name=None):
|
def _factory_app_config(model_names, *, summary_model_name=None, summarization_kwargs=None):
|
||||||
"""AppConfig-shaped stub for the factory: summarization enabled + ordered models."""
|
"""AppConfig-shaped stub for the factory: summarization enabled + ordered models."""
|
||||||
models = [SimpleNamespace(name=name) for name in model_names]
|
models = [SimpleNamespace(name=name) for name in model_names]
|
||||||
return SimpleNamespace(
|
return SimpleNamespace(
|
||||||
summarization=SummarizationConfig(enabled=True, model_name=summary_model_name),
|
summarization=SummarizationConfig(enabled=True, model_name=summary_model_name, **(summarization_kwargs or {})),
|
||||||
memory=MemoryConfig(enabled=False),
|
memory=MemoryConfig(enabled=False),
|
||||||
models=models,
|
models=models,
|
||||||
get_model_config=lambda name: next((model for model in models if model.name == name), None),
|
get_model_config=lambda name: next((model for model in models if model.name == name), None),
|
||||||
@ -1141,6 +1145,184 @@ def test_factory_configured_constructor_failure_falls_back_to_run_model(monkeypa
|
|||||||
assert result.summary_text == "from-run-model"
|
assert result.summary_text == "from-run-model"
|
||||||
|
|
||||||
|
|
||||||
|
def _profileless_anchor_stub() -> MagicMock:
|
||||||
|
"""Anchor stub whose ``.profile`` is unusable (not a Mapping), like any
|
||||||
|
third-party OpenAI-compatible client constructed without a profile."""
|
||||||
|
model = MagicMock()
|
||||||
|
model.with_config.return_value = model
|
||||||
|
model.invoke.return_value = SimpleNamespace(text="summary")
|
||||||
|
model.ainvoke = AsyncMock(return_value=SimpleNamespace(text="summary"))
|
||||||
|
return model
|
||||||
|
|
||||||
|
|
||||||
|
def test_factory_fraction_only_trigger_degrades_to_manual_compaction_only(monkeypatch, caplog: pytest.LogCaptureFixture) -> None:
|
||||||
|
"""#3103 mechanism (b): a fraction trigger whose anchor exposes no usable profile
|
||||||
|
must not raise out of ``create_summarization_middleware`` (which used to fail the
|
||||||
|
whole agent build). With no absolute clause to keep, the middleware still
|
||||||
|
constructs as never-firing so manual compaction (/compact, force=True, never
|
||||||
|
consults trigger clauses) keeps working; the warning names the config fix."""
|
||||||
|
fake_model = _profileless_anchor_stub()
|
||||||
|
monkeypatch.setattr("deerflow.agents.middlewares.summarization_middleware.create_chat_model", lambda **kw: fake_model)
|
||||||
|
cfg = _factory_app_config(("models0",), summarization_kwargs={"trigger": ContextSize(type="fraction", value=0.8)})
|
||||||
|
|
||||||
|
with caplog.at_level("WARNING", logger="deerflow.agents.middlewares.summarization_middleware"):
|
||||||
|
middleware = create_summarization_middleware(app_config=cfg, run_model_name="models0", keep=("messages", 2))
|
||||||
|
|
||||||
|
assert middleware is not None # degraded, not raised — and not disabled either
|
||||||
|
assert "context_window" in caplog.text # the warning names the fix
|
||||||
|
assert "Manual compaction" in caplog.text # and says manual /compact still works
|
||||||
|
result = middleware.compact_state({"messages": _messages()}, _runtime(), force=True)
|
||||||
|
assert result is not None # forced compaction never consults trigger clauses
|
||||||
|
assert result.summary_text == "summary"
|
||||||
|
|
||||||
|
|
||||||
|
def test_factory_drops_only_fraction_clauses_and_keeps_absolute_ones(monkeypatch, caplog: pytest.LogCaptureFixture) -> None:
|
||||||
|
"""Mixed [fraction, messages] triggers degrade to the messages clause alone:
|
||||||
|
construction succeeds and message-count compaction still fires."""
|
||||||
|
fake_model = _profileless_anchor_stub()
|
||||||
|
monkeypatch.setattr("deerflow.agents.middlewares.summarization_middleware.create_chat_model", lambda **kw: fake_model)
|
||||||
|
cfg = _factory_app_config(
|
||||||
|
("models0",),
|
||||||
|
summarization_kwargs={"trigger": [ContextSize(type="fraction", value=0.8), ContextSize(type="messages", value=3)]},
|
||||||
|
)
|
||||||
|
|
||||||
|
with caplog.at_level("WARNING", logger="deerflow.agents.middlewares.summarization_middleware"):
|
||||||
|
middleware = create_summarization_middleware(app_config=cfg, run_model_name="models0", keep=("messages", 2))
|
||||||
|
|
||||||
|
assert middleware is not None # the absolute clause kept the middleware alive
|
||||||
|
assert "context_window" in caplog.text
|
||||||
|
result = middleware.compact_state({"messages": _messages()}, _runtime(), force=True)
|
||||||
|
assert result is not None
|
||||||
|
assert result.summary_text == "summary"
|
||||||
|
|
||||||
|
|
||||||
|
def test_factory_keep_fraction_falls_back_to_messages_default(monkeypatch, caplog: pytest.LogCaptureFixture) -> None:
|
||||||
|
"""A fraction ``keep`` against a profile-less anchor falls back to the
|
||||||
|
messages default instead of failing construction."""
|
||||||
|
fake_model = _profileless_anchor_stub()
|
||||||
|
monkeypatch.setattr("deerflow.agents.middlewares.summarization_middleware.create_chat_model", lambda **kw: fake_model)
|
||||||
|
cfg = _factory_app_config(
|
||||||
|
("models0",),
|
||||||
|
summarization_kwargs={"trigger": ContextSize(type="messages", value=3), "keep": ContextSize(type="fraction", value=0.3)},
|
||||||
|
)
|
||||||
|
|
||||||
|
with caplog.at_level("WARNING", logger="deerflow.agents.middlewares.summarization_middleware"):
|
||||||
|
middleware = create_summarization_middleware(app_config=cfg, run_model_name="models0")
|
||||||
|
|
||||||
|
assert middleware is not None
|
||||||
|
assert middleware.keep == ("messages", 20) # SummarizationConfig's documented default
|
||||||
|
|
||||||
|
|
||||||
|
def test_factory_null_trigger_with_fraction_keep_still_constructs(monkeypatch, caplog: pytest.LogCaptureFixture) -> None:
|
||||||
|
"""``trigger: null`` + fraction ``keep``: the long-standing "enabled but never
|
||||||
|
auto-triggers" setup must keep constructing — with the keep degraded to the
|
||||||
|
messages default — rather than disabling compaction. On main this exact config
|
||||||
|
crashes the agent build (fraction keep needs a profile)."""
|
||||||
|
fake_model = _profileless_anchor_stub()
|
||||||
|
monkeypatch.setattr("deerflow.agents.middlewares.summarization_middleware.create_chat_model", lambda **kw: fake_model)
|
||||||
|
cfg = _factory_app_config(("models0",), summarization_kwargs={"keep": ContextSize(type="fraction", value=0.3)})
|
||||||
|
|
||||||
|
with caplog.at_level("WARNING", logger="deerflow.agents.middlewares.summarization_middleware"):
|
||||||
|
middleware = create_summarization_middleware(app_config=cfg, run_model_name="models0")
|
||||||
|
|
||||||
|
assert middleware is not None # never-firing but constructed, same as any trigger: null setup
|
||||||
|
assert middleware.keep == ("messages", 20)
|
||||||
|
assert "context_window" in caplog.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_factory_fraction_trigger_survives_when_anchor_has_profile(monkeypatch) -> None:
|
||||||
|
"""With a usable profile on the anchor (the factory attaches one from
|
||||||
|
``context_window``), the fraction clause is kept as configured — construction
|
||||||
|
succeeding is itself the regression pin (#3103: it used to raise)."""
|
||||||
|
model = _StaticChatModel(profile={"max_input_tokens": 65536})
|
||||||
|
assert model.profile == {"max_input_tokens": 65536}
|
||||||
|
monkeypatch.setattr("deerflow.agents.middlewares.summarization_middleware.create_chat_model", lambda **kw: model)
|
||||||
|
cfg = _factory_app_config(("models0",), summarization_kwargs={"trigger": ContextSize(type="fraction", value=0.8)})
|
||||||
|
|
||||||
|
middleware = create_summarization_middleware(app_config=cfg, run_model_name="models0", keep=("messages", 2))
|
||||||
|
|
||||||
|
assert middleware is not None
|
||||||
|
result = middleware.compact_state({"messages": _messages()}, _runtime(), force=True)
|
||||||
|
assert result is not None
|
||||||
|
assert result.summary_text == "ok"
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_size_rejects_percent_style_fraction_value() -> None:
|
||||||
|
"""A fraction written percent-style (80 instead of 0.8) would resolve to
|
||||||
|
int(capacity * 80) — a threshold the context can never reach, so the trigger
|
||||||
|
silently never fires. Config load is the failure point, not a dead trigger."""
|
||||||
|
with pytest.raises(ValidationError, match="fraction ContextSize value must be in"):
|
||||||
|
ContextSize(type="fraction", value=80)
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_size_rejects_non_positive_absolute_values() -> None:
|
||||||
|
with pytest.raises(ValidationError, match="tokens ContextSize value must be positive"):
|
||||||
|
ContextSize(type="tokens", value=0)
|
||||||
|
with pytest.raises(ValidationError, match="messages ContextSize value must be positive"):
|
||||||
|
ContextSize(type="messages", value=-5)
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_size_rejects_non_finite_values() -> None:
|
||||||
|
"""YAML .nan / .inf pass pydantic's float parsing but never describe a usable
|
||||||
|
threshold (``count >= nan`` is always False, and ``nan <= 0`` is False so the
|
||||||
|
positivity check alone would not catch them) — they must fail at config load."""
|
||||||
|
with pytest.raises(ValidationError, match="ContextSize value must be finite"):
|
||||||
|
ContextSize(type="tokens", value=float("nan"))
|
||||||
|
with pytest.raises(ValidationError, match="ContextSize value must be finite"):
|
||||||
|
ContextSize(type="fraction", value=float("inf"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_size_rejects_fractional_message_counts() -> None:
|
||||||
|
"""``messages`` values slice the message list at compaction time
|
||||||
|
(``messages[-keep:]``) — a float raises ``TypeError: list indices must be
|
||||||
|
integers or slices, not float`` mid-compaction, so config load must reject
|
||||||
|
it first. Even an integral float (``20.0``) is a float index to a slice."""
|
||||||
|
with pytest.raises(ValidationError, match="messages ContextSize value must be a whole number"):
|
||||||
|
ContextSize(type="messages", value=1.5)
|
||||||
|
with pytest.raises(ValidationError, match="messages ContextSize value must be a whole number"):
|
||||||
|
ContextSize(type="messages", value=20.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_size_accepts_boundary_values() -> None:
|
||||||
|
assert ContextSize(type="fraction", value=1).to_tuple() == ("fraction", 1)
|
||||||
|
assert ContextSize(type="fraction", value=0.8).to_tuple() == ("fraction", 0.8)
|
||||||
|
assert ContextSize(type="messages", value=20).to_tuple() == ("messages", 20)
|
||||||
|
|
||||||
|
|
||||||
|
def test_factory_wiring_context_window_to_fraction_trigger_end_to_end() -> None:
|
||||||
|
"""Pins the two halves of the fix together without monkeypatching
|
||||||
|
``create_chat_model``: a ``context_window``-declared model gets a profile from
|
||||||
|
the real model factory, the fraction clause survives
|
||||||
|
``_drop_unusable_fraction_clauses``, and the middleware constructs with the
|
||||||
|
trigger intact. The middleware-side tests stub the factory and the
|
||||||
|
factory-side tests stop at captured kwargs — this is the automated pin of the
|
||||||
|
shipped contract (the manual E2E in the PR body was the only wiring proof)."""
|
||||||
|
model = ModelConfig(
|
||||||
|
name="gw-64k",
|
||||||
|
display_name="gw-64k",
|
||||||
|
description=None,
|
||||||
|
use="langchain_openai:ChatOpenAI",
|
||||||
|
model="some-64k-model",
|
||||||
|
base_url="https://third-party-gateway.example.com/v1",
|
||||||
|
api_key="sk-test",
|
||||||
|
supports_thinking=False,
|
||||||
|
supports_reasoning_effort=False,
|
||||||
|
supports_vision=False,
|
||||||
|
context_window=65_536,
|
||||||
|
)
|
||||||
|
cfg = AppConfig(
|
||||||
|
models=[model],
|
||||||
|
sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider"),
|
||||||
|
summarization=SummarizationConfig(enabled=True, trigger=ContextSize(type="fraction", value=0.8)),
|
||||||
|
memory=MemoryConfig(enabled=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
middleware = create_summarization_middleware(app_config=cfg, run_model_name="gw-64k")
|
||||||
|
|
||||||
|
assert middleware is not None # on main this raises: no profile without the factory translation
|
||||||
|
assert middleware.model.profile == {"max_input_tokens": 65_536}
|
||||||
|
|
||||||
|
|
||||||
class _RaisingTextResponse:
|
class _RaisingTextResponse:
|
||||||
"""A provider response whose ``.text`` accessor fails — a realistic malformed result."""
|
"""A provider response whose ``.text`` accessor fails — a realistic malformed result."""
|
||||||
|
|
||||||
|
|||||||
@ -102,8 +102,12 @@ max_recursion_limit: 1000
|
|||||||
# Two token fields look similar but mean different things:
|
# Two token fields look similar but mean different things:
|
||||||
# - `max_tokens` is the per-call OUTPUT cap passed to the provider.
|
# - `max_tokens` is the per-call OUTPUT cap passed to the provider.
|
||||||
# - `context_window` is a positive integer for the total context capacity
|
# - `context_window` is a positive integer for the total context capacity
|
||||||
# (prompt + completion) and drives the real-time "% context used" indicator
|
# (prompt + completion). It drives the real-time "% context used" indicator
|
||||||
# in the chat UI.
|
# in the chat UI and feeds the model's langchain profile, which fraction-based
|
||||||
|
# summarization triggers (`summarization.trigger: [{type: fraction, ...}]`)
|
||||||
|
# resolve their thresholds from. Third-party OpenAI-compatible models carry no
|
||||||
|
# built-in profile, so without `context_window` a fraction trigger degrades
|
||||||
|
# (dropped with a warning) instead of crashing the agent build.
|
||||||
# Leave `context_window` unset if the provider limit is unknown; the percentage
|
# Leave `context_window` unset if the provider limit is unknown; the percentage
|
||||||
# will not render. Verify configured values against the provider's model docs.
|
# will not render. Verify configured values against the provider's model docs.
|
||||||
#
|
#
|
||||||
@ -1760,7 +1764,14 @@ summarization:
|
|||||||
# Uncomment to also trigger when message count reaches 50
|
# Uncomment to also trigger when message count reaches 50
|
||||||
# - type: messages
|
# - type: messages
|
||||||
# value: 50
|
# value: 50
|
||||||
# Uncomment to trigger when 80% of model's max input tokens is reached
|
# Uncomment to trigger when 80% of model's max input tokens is reached.
|
||||||
|
# The percentage resolves from the SUMMARY model's declared `context_window`
|
||||||
|
# (summarization.model_name when set, else the run's own model): declare it
|
||||||
|
# on that models entry — third-party OpenAI-compatible models carry no
|
||||||
|
# built-in profile. Without it the fraction clause is dropped with a warning
|
||||||
|
# and any remaining absolute clauses keep working. If a separate summary
|
||||||
|
# model with a larger window is configured, prefer absolute `tokens`
|
||||||
|
# thresholds sized for the run model instead.
|
||||||
# - type: fraction
|
# - type: fraction
|
||||||
# value: 0.8
|
# value: 0.8
|
||||||
|
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user