mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-28 14:10:37 +00:00
* fix(models): deep-merge when_thinking_enabled on the legacy path The legacy thinking path (profiles without a `reasoning:` block, i.e. every existing config.yaml) applied `when_thinking_enabled` with a shallow `dict.update`, so a template such as `extra_body.thinking.type=enabled` replaced the profile's whole `extra_body` and silently dropped sibling keys like GLM's `extra_body.tool_stream` — but only while thinking was on. The disable branch of the same function already deep-merged into `extra_body`, and the contract path added in #5780 deep-merges through `_merge_settings` for exactly this reason (its docstring names `tool_stream`). The on/off asymmetry cannot be intended: the same profile sent `tool_stream` with thinking off and lost it with thinking on. Route the enable template through `_merge_settings` as well. Precedence is unchanged (template values win on conflicts); the only observable difference is that keys present only in the profile now survive a thinking-on request. Everything else on the legacy path — the synthesized `reasoning_effort=minimal`, the disable payloads, the supports_thinking guard — is untouched. Tests pin both directions for a base extra_body beside a thinking template, recursive merging of nested chat_template_kwargs, and template precedence. * docs(changelog): reference #5894 in the when_thinking_enabled entries * fix(models): keep the template's vLLM switch spelling authoritative when merging Review follow-up. Deep-merging the thinking template into a profile keeps both spellings of vLLM's toggle when they differ — base `chat_template_kwargs.enable_thinking: false` plus a template using the legacy `thinking: true` alias — and VllmChatModel's normalizer maps the alias onto `enable_thinking` only when the latter is absent, so the profile's `false` won and thinking stayed off despite thinking_enabled=True. The previous shallow replacement never hit this because it discarded the base switch. The contract path had the same trap since it already deep-merged, and both disable branches had the mirror image (a base `enable_thinking: true` outlived a template `thinking: false`). Route every thinking on/off merge through _merge_thinking_payload: it drops the profile's *other* spelling of the switch before the deep merge, so the payload's spelling decides in both directions on both paths, and unrelated chat_template_kwargs still survive. Tests cover both spelling combinations, both directions, and both paths, asserting the switch as the provider will send it after normalization. * fix(models): deep-merge the legacy disable template and mirror the vLLM switch instead of pruning Review of the previous head found two regressions. The legacy when_thinking_disabled branch was still dict.update-ed, so a profile that declares both templates (the shape of most extra_body examples) kept tool_stream while thinking was on and lost it while off. And the merge pruned the profile's other vLLM switch spelling for every model class, although only VllmChatModel maps `thinking` onto `enable_thinking`: on a plain ChatOpenAI profile the prune removed the only key vLLM reads, so thinking-off could no longer switch reasoning off. Route when_thinking_disabled through _merge_thinking_payload too, and replace the prune with mirroring: after the deep merge, the value the payload declares for its switch spelling is copied onto the profile's other spelling when the profile carries it. VllmChatModel collapses the pair, plain OpenAI-compatible classes send both, and either way the server reads the template's intent. No spelling is invented, a payload that spells both is left alone, and non-mapping template values are forwarded unchanged instead of raising. _VLLM_THINKING_SWITCHES is now the single definition used by the disable, toggle and dialect helpers. Docs state the merge semantics (keys are never removed; a template can only add or override) and drop the "byte-for-byte" wording; tests cover the disable template on/off, nested inheritance, non-mapping values, the raw payload for a non-normalizing class, and the no-invention rule. * fix(models): copy thinking templates on merge and document the never-remove rule Review follow-up. Templates merged into the constructor settings were stored by reference wherever the profile lacked the key, so the kwargs handed to the model class aliased the cached ModelConfig; the mirroring step now writes into that mapping, so make _merge_settings and _deep_merge_dicts deep-copy every template mapping they store. Pinned at every depth by a test that also checks the profile is unchanged. Deep-merging the legacy when_thinking_disabled template means it can no longer clear a key the profile's base extra_body sets: a base extra_body.thinking {type: enabled, budget_tokens} reaches the provider as {type: disabled, budget_tokens}, which Anthropic-style APIs reject. The synthesized disable payloads and the contract path already behaved this way. State the rule and the migration (enable-only keys belong in when_thinking_enabled) in the changelog, models/AGENTS.md and configuration.mdx, and pin it with a test for both shapes. Route the legacy synthesized OpenAI disable through _merge_thinking_payload too (identical output), narrow the docstring and guidance wording about the native-Anthropic disable, and make the two vLLM switch suites resolve the real VllmChatModel / ChatOpenAI classes so a class-gated merge would fail the case it is named after.
626 lines
34 KiB
Python
626 lines
34 KiB
Python
import copy
|
|
import logging
|
|
|
|
from langchain.chat_models import BaseChatModel
|
|
from langchain_openai.chat_models.base import BaseChatOpenAI
|
|
|
|
from deerflow.config import get_app_config
|
|
from deerflow.config.app_config import AppConfig
|
|
from deerflow.models.reasoning import ReasoningContract, ReasoningPolicyError, resolve_reasoning_contract, resolve_reasoning_request
|
|
from deerflow.reflection import resolve_class
|
|
from deerflow.tracing import build_tracing_callbacks
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
# ``ModelConfig`` fields that describe the profile rather than configure the
|
|
# provider client. They are stripped before the constructor sees the profile.
|
|
_MODEL_METADATA_FIELDS: frozenset[str] = frozenset(
|
|
{
|
|
"use",
|
|
"name",
|
|
"display_name",
|
|
"description",
|
|
"supports_thinking",
|
|
"supports_reasoning_effort",
|
|
# Declarative reasoning contract (issue #5073); consumed by the
|
|
# normalization layer, never a provider kwarg.
|
|
"reasoning",
|
|
"when_thinking_enabled",
|
|
"when_thinking_disabled",
|
|
"thinking",
|
|
"supports_vision",
|
|
# Runtime/UI metadata used to size the context indicator. Provider
|
|
# clients do not accept this as a model-constructor argument.
|
|
"context_window",
|
|
# Presentation-only metadata (consumed by the console's cost
|
|
# display) — must never reach the provider client, which would
|
|
# forward unknown kwargs into the completion request payload.
|
|
"pricing",
|
|
"request_admission",
|
|
}
|
|
)
|
|
|
|
|
|
def _deep_merge_dicts(base: dict | None, override: dict) -> dict:
|
|
"""Recursively merge two dictionaries without mutating the inputs."""
|
|
merged = dict(base or {})
|
|
for key, value in override.items():
|
|
if isinstance(value, dict) and isinstance(merged.get(key), dict):
|
|
merged[key] = _deep_merge_dicts(merged[key], value)
|
|
elif isinstance(value, dict):
|
|
# Copy, don't alias: *override* is usually a template straight from
|
|
# the cached ``ModelConfig`` (see ``_merge_settings``).
|
|
merged[key] = copy.deepcopy(value)
|
|
else:
|
|
merged[key] = value
|
|
return merged
|
|
|
|
|
|
# The two spellings of the vLLM/Qwen chat-template thinking switch: the legacy
|
|
# ``thinking`` alias DeerFlow documented first and ``enable_thinking``, which
|
|
# vLLM's Qwen reasoning parser reads. ``VllmChatModel`` maps the alias onto
|
|
# ``enable_thinking`` just before sending; other OpenAI-compatible classes
|
|
# forward both keys as written.
|
|
_VLLM_THINKING_SWITCHES = ("thinking", "enable_thinking")
|
|
|
|
|
|
def _vllm_disable_chat_template_kwargs(chat_template_kwargs: dict) -> dict:
|
|
"""Build the disable payload for vLLM/Qwen chat template kwargs."""
|
|
return {key: False for key in _VLLM_THINKING_SWITCHES if key in chat_template_kwargs}
|
|
|
|
|
|
def _merge_settings(settings: dict, payload: dict) -> None:
|
|
"""Deep-merge *payload* into the constructor settings in place.
|
|
|
|
Nested dicts (``extra_body``, ``thinking``, ``chat_template_kwargs``) merge
|
|
recursively so a synthesized thinking block never clobbers unrelated keys an
|
|
operator put beside it (e.g. GLM's ``extra_body.tool_stream``).
|
|
"""
|
|
for key, value in payload.items():
|
|
if isinstance(value, dict) and isinstance(settings.get(key), dict):
|
|
settings[key] = _deep_merge_dicts(settings[key], value)
|
|
elif isinstance(value, dict):
|
|
# Never hand the constructor (or a later in-place edit here) the
|
|
# template's own mapping: payloads come straight from the cached
|
|
# ``ModelConfig``, and a shared reference would leak one call's
|
|
# adjustments into every later ``create_chat_model`` for that profile.
|
|
settings[key] = copy.deepcopy(value)
|
|
else:
|
|
settings[key] = value
|
|
|
|
|
|
def _chat_template_kwargs(settings: dict) -> dict | None:
|
|
"""``settings["extra_body"]["chat_template_kwargs"]`` when both levels are mappings, else ``None``."""
|
|
extra_body = settings.get("extra_body")
|
|
if not isinstance(extra_body, dict):
|
|
return None
|
|
chat_template_kwargs = extra_body.get("chat_template_kwargs")
|
|
return chat_template_kwargs if isinstance(chat_template_kwargs, dict) else None
|
|
|
|
|
|
def _merge_thinking_payload(settings: dict, payload: dict) -> None:
|
|
"""Deep-merge a thinking on/off *payload*, keeping its vLLM switch authoritative.
|
|
|
|
Keys are never removed: a template can only add or override, so a profile's
|
|
unrelated ``extra_body`` / ``chat_template_kwargs`` entries survive in both
|
|
directions. The one thing a plain deep merge gets wrong is the vLLM switch,
|
|
which has two spellings (see ``_VLLM_THINKING_SWITCHES``): a profile that
|
|
spells it differently from the payload (base ``enable_thinking: false``,
|
|
template ``thinking: true``) would keep both keys with opposite values, and
|
|
whichever key the server reads might be the profile's. So after merging, the
|
|
value the payload declares is mirrored onto the *other* spelling when the
|
|
profile carries it. Both keys then agree: ``VllmChatModel`` collapses them to
|
|
``enable_thinking``, and a plain OpenAI-compatible class sends both, so the
|
|
server sees the payload's intent under whichever name it honors. No spelling
|
|
is invented (a template using only ``thinking`` never grows
|
|
``enable_thinking``), and a payload that spells both itself is left as
|
|
written. Non-mapping template values are forwarded unchanged, as before.
|
|
|
|
Deep-merging also means a template cannot *remove* a key by omitting it:
|
|
a base ``extra_body.thinking.budget_tokens`` survives a disable template
|
|
that only sets ``thinking.type: disabled``. Enable-only keys therefore
|
|
belong in ``when_thinking_enabled``, not in the profile's base settings.
|
|
"""
|
|
_merge_settings(settings, payload)
|
|
payload_kwargs = _chat_template_kwargs(payload)
|
|
merged_kwargs = _chat_template_kwargs(settings)
|
|
if payload_kwargs is None or merged_kwargs is None:
|
|
return
|
|
declared = {key: payload_kwargs[key] for key in _VLLM_THINKING_SWITCHES if key in payload_kwargs}
|
|
if len(declared) != 1:
|
|
return
|
|
value = next(iter(declared.values()))
|
|
# In-place is safe: ``_merge_settings`` / ``_deep_merge_dicts`` copy every
|
|
# template mapping they store, so ``merged_kwargs`` is never the config's own dict.
|
|
for key in _VLLM_THINKING_SWITCHES:
|
|
if key not in declared and key in merged_kwargs:
|
|
merged_kwargs[key] = value
|
|
|
|
|
|
def _set_dotted_setting(settings: dict, path: str, value: object) -> None:
|
|
"""Write *value* at a dotted *path* (``reasoning_effort`` or ``extra_body.thinking.effort``)."""
|
|
head, _, rest = path.partition(".")
|
|
if not rest:
|
|
settings[head] = value
|
|
return
|
|
nested: dict = {}
|
|
cursor = nested
|
|
parts = rest.split(".")
|
|
for part in parts[:-1]:
|
|
cursor[part] = {}
|
|
cursor = cursor[part]
|
|
cursor[parts[-1]] = value
|
|
_merge_settings(settings, {head: nested})
|
|
|
|
|
|
def _infer_dialect(effective_wte: dict) -> str:
|
|
"""Infer the thinking payload dialect from a ``when_thinking_enabled`` template.
|
|
|
|
Mirrors the historical disable-path heuristics so ``dialect: auto`` behaves
|
|
like the legacy factory did for the same template shape.
|
|
"""
|
|
extra_body = effective_wte.get("extra_body") or {}
|
|
thinking = extra_body.get("thinking")
|
|
if isinstance(thinking, dict) and thinking.get("type"):
|
|
return "openai_extra_body"
|
|
chat_template_kwargs = extra_body.get("chat_template_kwargs") or {}
|
|
if isinstance(chat_template_kwargs, dict) and any(key in chat_template_kwargs for key in _VLLM_THINKING_SWITCHES):
|
|
return "vllm_chat_template"
|
|
native_thinking = effective_wte.get("thinking")
|
|
if isinstance(native_thinking, dict) and native_thinking.get("type"):
|
|
return "anthropic"
|
|
if "reasoning" in effective_wte:
|
|
return "ollama"
|
|
return "none"
|
|
|
|
|
|
def _vllm_toggle_chat_template_kwargs(chat_template_kwargs: dict, *, enabled: bool) -> dict:
|
|
"""Mirror whichever vLLM/Qwen switch the template declares; default to ``enable_thinking``."""
|
|
toggled = {key: enabled for key in _VLLM_THINKING_SWITCHES if key in chat_template_kwargs}
|
|
return toggled or {"enable_thinking": enabled}
|
|
|
|
|
|
def _dialect_payload(dialect: str, *, enabled: bool, chat_template_kwargs: dict | None = None) -> dict:
|
|
"""Constructor settings that switch thinking on or off for one payload dialect."""
|
|
if dialect == "openai_extra_body":
|
|
return {"extra_body": {"thinking": {"type": "enabled" if enabled else "disabled"}}}
|
|
if dialect == "anthropic":
|
|
return {"thinking": {"type": "enabled" if enabled else "disabled"}}
|
|
if dialect == "vllm_chat_template":
|
|
return {"extra_body": {"chat_template_kwargs": _vllm_toggle_chat_template_kwargs(chat_template_kwargs or {}, enabled=enabled)}}
|
|
if dialect == "ollama":
|
|
return {"reasoning": enabled}
|
|
return {}
|
|
|
|
|
|
def _apply_legacy_thinking_settings(
|
|
settings: dict,
|
|
model_config,
|
|
*,
|
|
thinking_enabled: bool,
|
|
has_thinking_settings: bool,
|
|
effective_wte: dict,
|
|
requested_reasoning_effort: str | None,
|
|
is_codex_model: bool,
|
|
) -> None:
|
|
"""Historical thinking/effort payload path for profiles without a ``reasoning:`` contract.
|
|
|
|
Kept as-is so existing configurations retain their behavior (issue #5073
|
|
acceptance criterion), including the synthesized ``reasoning_effort=minimal``
|
|
on the OpenAI-compatible disable path that the contract path drops. The one
|
|
deliberate change is how the operator's templates are applied: both
|
|
``when_thinking_enabled`` and ``when_thinking_disabled`` used to be
|
|
``dict.update``-ed onto the settings, so a template's ``extra_body`` replaced
|
|
the profile's whole ``extra_body`` and silently dropped sibling keys such as
|
|
GLM's ``tool_stream``. Both now go through ``_merge_thinking_payload`` like
|
|
the synthesized ``extra_body`` disable payloads and the contract path: keys
|
|
are never removed (so enable-only keys such as ``budget_tokens`` belong in
|
|
``when_thinking_enabled``, not in the base ``extra_body``), template values
|
|
win on conflicts, and the template's vLLM switch stays authoritative across
|
|
its two spellings. The native-Anthropic disable still assigns
|
|
``settings["thinking"]`` outright: that mapping is a tagged union keyed by
|
|
``type``, and nothing else lives beside it at the top level.
|
|
"""
|
|
if requested_reasoning_effort is not None and not is_codex_model:
|
|
settings["reasoning_effort"] = requested_reasoning_effort
|
|
if thinking_enabled and has_thinking_settings and effective_wte:
|
|
_merge_thinking_payload(settings, effective_wte)
|
|
if not thinking_enabled:
|
|
if model_config.when_thinking_disabled is not None:
|
|
# User-provided disable settings win on conflicts; profile keys survive.
|
|
_merge_thinking_payload(settings, model_config.when_thinking_disabled)
|
|
elif has_thinking_settings and effective_wte.get("extra_body", {}).get("thinking", {}).get("type"):
|
|
# OpenAI-compatible gateway: thinking is nested under extra_body
|
|
_merge_thinking_payload(settings, {"extra_body": {"thinking": {"type": "disabled"}}})
|
|
settings["reasoning_effort"] = "minimal"
|
|
elif has_thinking_settings and (disable_chat_template_kwargs := _vllm_disable_chat_template_kwargs(effective_wte.get("extra_body", {}).get("chat_template_kwargs") or {})):
|
|
# vLLM uses chat template kwargs to switch thinking on/off.
|
|
_merge_thinking_payload(settings, {"extra_body": {"chat_template_kwargs": disable_chat_template_kwargs}})
|
|
elif has_thinking_settings and effective_wte.get("thinking", {}).get("type"):
|
|
# Native langchain_anthropic: thinking is a direct constructor parameter
|
|
settings["thinking"] = {"type": "disabled"}
|
|
|
|
|
|
def _apply_contract_thinking_settings(
|
|
settings: dict,
|
|
contract: ReasoningContract,
|
|
model_config,
|
|
*,
|
|
thinking_enabled: bool,
|
|
effective_wte: dict,
|
|
requested_reasoning_effort: str | None,
|
|
codex_handles_generic_effort: bool,
|
|
) -> None:
|
|
"""Thinking/effort payload path for profiles that declare a ``reasoning:`` contract.
|
|
|
|
``thinking_enabled`` and ``requested_reasoning_effort`` are the *resolved*
|
|
values from :func:`resolve_reasoning_request`, so a required-thinking model
|
|
never enters the disable branch and the effort value is already one the
|
|
provider accepts.
|
|
|
|
* Enabled: the dialect's enable payload deep-merged under the operator's
|
|
``when_thinking_enabled`` / ``thinking`` template (template wins), plus
|
|
``clear_thinking`` when the contract states a history requirement and the
|
|
dialect nests thinking under ``extra_body``.
|
|
* Disabled: ``when_thinking_disabled`` when present, else the dialect's
|
|
disable payload. No effort value is ever synthesized here.
|
|
* Effort: written last at ``effort.path`` so a validated request wins over
|
|
a template value; ``None`` leaves any profile/template value in place. A
|
|
model without an effort contract never forwards ``reasoning_effort``.
|
|
"""
|
|
dialect = contract.dialect if contract.dialect != "auto" else _infer_dialect(effective_wte)
|
|
# Both directions mirror whichever vLLM switch the template declares, so a
|
|
# template using the older ``thinking`` key never also grows ``enable_thinking``.
|
|
chat_template_kwargs = (effective_wte.get("extra_body") or {}).get("chat_template_kwargs") or {}
|
|
if thinking_enabled:
|
|
payload = _deep_merge_dicts(_dialect_payload(dialect, enabled=True, chat_template_kwargs=chat_template_kwargs), effective_wte)
|
|
if contract.history is not None and dialect == "openai_extra_body":
|
|
payload = _deep_merge_dicts(payload, {"extra_body": {"thinking": {"clear_thinking": contract.history == "clear"}}})
|
|
_merge_thinking_payload(settings, payload)
|
|
elif model_config.when_thinking_disabled is not None:
|
|
_merge_thinking_payload(settings, model_config.when_thinking_disabled)
|
|
else:
|
|
_merge_thinking_payload(settings, _dialect_payload(dialect, enabled=False, chat_template_kwargs=chat_template_kwargs))
|
|
if contract.effort is None or contract.effort.path != "reasoning_effort":
|
|
# A custom path is the only effort wire format for this contract.
|
|
# Drop a generic key supplied by model_overrides as well as stale
|
|
# settings on a ModelConfig mutated after load-time validation.
|
|
settings.pop("reasoning_effort", None)
|
|
if contract.effort is not None and requested_reasoning_effort is not None and not codex_handles_generic_effort:
|
|
_set_dotted_setting(settings, contract.effort.path, requested_reasoning_effort)
|
|
|
|
|
|
def _declares_api_base(model_class: type) -> bool:
|
|
"""Whether *model_class* declares ``api_base`` as its own constructor field.
|
|
|
|
``langchain_deepseek:ChatDeepSeek`` (and therefore ``PatchedChatDeepSeek``) does, so for it
|
|
``api_base`` is the canonical endpoint key and must be passed through untouched. Every other
|
|
``BaseChatOpenAI`` subclass inherits only ``openai_api_base`` (alias ``base_url``).
|
|
"""
|
|
return "api_base" in getattr(model_class, "model_fields", {})
|
|
|
|
|
|
def _normalize_openai_base_url(model_class: type, model_settings_from_config: dict) -> None:
|
|
"""Map the common ``api_base`` alias to ``base_url`` for OpenAI-compatible clients.
|
|
|
|
``BaseChatOpenAI`` subclasses accept the OpenAI endpoint override as ``base_url`` (with
|
|
``openai_api_base`` as a legacy alias). Several providers in ``config.example.yaml`` use
|
|
``api_base`` for *other* model classes, so users frequently copy ``api_base`` onto such a model
|
|
by mistake. Because ``ModelConfig`` is ``extra="allow"``, the bad key is not caught at
|
|
config-load time — it is forwarded to the constructor, which does not reject it but transfers it
|
|
into ``model_kwargs``; that is then spread into every ``Completions.create()`` call and rejected
|
|
by the OpenAI SDK at *request* time with an opaque ``unexpected keyword argument 'api_base'``
|
|
error (and the endpoint override is silently dropped). Rename it here so the model works as the
|
|
user intended.
|
|
|
|
Gated on ``issubclass(model_class, BaseChatOpenAI)`` rather than a class-path allowlist, so any
|
|
OpenAI-compatible subclass is covered automatically — the divert-and-crash behaviour is a
|
|
property of the base class, not of the two paths that used to be listed. Classes that declare
|
|
``api_base`` themselves are skipped: there the key is canonical, not a typo.
|
|
"""
|
|
if not issubclass(model_class, BaseChatOpenAI) or _declares_api_base(model_class):
|
|
return
|
|
if "api_base" not in model_settings_from_config:
|
|
return
|
|
if "base_url" in model_settings_from_config or "openai_api_base" in model_settings_from_config:
|
|
# Canonical key already present; drop the alias to avoid a duplicate-intent kwarg.
|
|
model_settings_from_config.pop("api_base", None)
|
|
logger.warning("Model config sets both an endpoint key (base_url/openai_api_base) and 'api_base'; using the former and ignoring 'api_base'.")
|
|
return
|
|
model_settings_from_config["base_url"] = model_settings_from_config.pop("api_base")
|
|
logger.debug("Normalized model config key 'api_base' -> 'base_url' for OpenAI-compatible client.")
|
|
|
|
|
|
def _warn_unknown_model_settings(model_class, model_name: str, model_settings_from_config: dict) -> None:
|
|
"""Warn about config keys the OpenAI client will silently divert into ``model_kwargs``.
|
|
|
|
``ModelConfig`` is ``extra="allow"``, so a typo'd key (e.g. ``maxx_tokens``) is not caught at
|
|
config-load time. LangChain's OpenAI client does not reject an unknown constructor kwarg — it
|
|
emits a ``UserWarning`` and transfers the key into ``model_kwargs``, which is then spread into
|
|
every ``Completions.create()`` call and rejected by the OpenAI SDK at *request* time with an
|
|
opaque ``unexpected keyword argument`` error that is very hard to trace back to a config typo.
|
|
|
|
This turns that latent failure into an explicit, actionable log line at model-build time. It is
|
|
**scoped to the OpenAI-compatible family** — that is where the ``model_kwargs``
|
|
divert-and-crash behavior occurs and where the known field/alias set is accurate. The family is
|
|
``issubclass(model_class, BaseChatOpenAI)``: the divert is implemented in that base class, so
|
|
every subclass inherits it. Other providers (e.g. ``ChatAnthropic``) route extra kwargs
|
|
differently and would false-positive against this allow-list, so they are intentionally left
|
|
alone. Best-effort and non-fatal: it only fires when the class exposes a pydantic
|
|
``model_fields`` schema, treats both field names and their aliases as valid, and allow-lists the
|
|
standard passthrough kwargs the factory injects and the OpenAI client accepts.
|
|
"""
|
|
if not issubclass(model_class, BaseChatOpenAI):
|
|
return
|
|
known = getattr(model_class, "model_fields", None)
|
|
if not known:
|
|
return
|
|
valid_names = set(known.keys())
|
|
for field in known.values():
|
|
alias = getattr(field, "alias", None)
|
|
if alias:
|
|
valid_names.add(alias)
|
|
# Standard kwargs the factory injects or the OpenAI client accepts beyond declared fields.
|
|
valid_names |= {
|
|
"model",
|
|
"model_kwargs",
|
|
"extra_body",
|
|
"default_headers",
|
|
"default_query",
|
|
"stream_usage",
|
|
"stream_chunk_timeout",
|
|
"reasoning_effort",
|
|
}
|
|
unknown = sorted(k for k in model_settings_from_config if k not in valid_names)
|
|
if unknown:
|
|
logger.warning(
|
|
"Model '%s' (%s): config key(s) %s are not recognized parameters of the model class and will be forwarded as-is; this may raise at request time. Check for typos (e.g. 'maxx_tokens' -> 'max_tokens').",
|
|
model_name,
|
|
getattr(model_class, "__name__", "?"),
|
|
unknown,
|
|
)
|
|
|
|
|
|
# Default chunk-gap budget for OpenAI-compatible streaming responses.
|
|
#
|
|
# langchain-openai raises ``StreamChunkTimeoutError`` after this many seconds
|
|
# without receiving a chunk. Its own default is 120s, which is too aggressive for
|
|
# reasoning models (DeepSeek-R1, Doubao-thinking, GPT-5) whose first chunk can
|
|
# legitimately take 90~150s. We default to 240s so the streaming layer rarely
|
|
# trips on long thinking pauses; the LLMErrorHandlingMiddleware still retries
|
|
# (budget=2) if a real stall happens. Users can override per-model in config.yaml.
|
|
_DEFAULT_STREAM_CHUNK_TIMEOUT_SECONDS: float = 240.0
|
|
|
|
|
|
def _apply_stream_chunk_timeout_default(model_class: type, model_settings_from_config: dict) -> None:
|
|
"""Inject a generous ``stream_chunk_timeout`` for OpenAI-compatible clients.
|
|
|
|
``stream_chunk_timeout`` is a field of langchain-openai's ``BaseChatOpenAI``, so
|
|
it is accepted by ``ChatOpenAI`` and by every DeerFlow provider that subclasses
|
|
it: ``PatchedChatOpenAI`` plus the self-hosted / reasoning adapters
|
|
``VllmChatModel``, ``MindIEChatModel``, ``PatchedChatDeepSeek``,
|
|
``PatchedChatMiMo``, ``PatchedChatStepFun`` and ``PatchedChatMiniMax``. We gate on
|
|
``issubclass(model_class, BaseChatOpenAI)`` rather than an explicit class-path
|
|
allowlist so any OpenAI-compatible subclass inherits the default (and honors an
|
|
explicit override) automatically. Issue #3189 was reported against ``mimo-v2.5``
|
|
(``PatchedChatMiMo``); the original fix (#3195) matched only ``ChatOpenAI`` /
|
|
``PatchedChatOpenAI``, so those subclasses kept langchain-openai's aggressive
|
|
built-in chunk-gap timeout and — worse — silently discarded a user's explicit
|
|
``stream_chunk_timeout``.
|
|
|
|
Behaviour:
|
|
|
|
* ``BaseChatOpenAI`` subclass: an explicit value in ``config.yaml`` is preserved.
|
|
An explicit ``null`` is dropped upstream by ``model_dump(exclude_none=True)``
|
|
and therefore treated as "unset", so the default is injected.
|
|
* Any other client (e.g. ``ChatAnthropic``): drop the key so it is never
|
|
forwarded to a constructor that does not declare it. The kwarg is not a
|
|
declared field of these clients: depending on the client it is either
|
|
silently dropped (``ChatAnthropic`` declares ``extra="ignore"``) or, for
|
|
other OpenAI-style clients, diverted into ``model_kwargs`` and rejected
|
|
at request time. Either way the user's intent is lost, so we drop it
|
|
proactively instead.
|
|
"""
|
|
if not issubclass(model_class, BaseChatOpenAI):
|
|
model_settings_from_config.pop("stream_chunk_timeout", None)
|
|
return
|
|
if "stream_chunk_timeout" in model_settings_from_config:
|
|
return
|
|
model_settings_from_config["stream_chunk_timeout"] = _DEFAULT_STREAM_CHUNK_TIMEOUT_SECONDS
|
|
|
|
|
|
def create_chat_model(name: str | None = None, thinking_enabled: bool = False, *, app_config: AppConfig | None = None, attach_tracing: bool = True, model_overrides: dict | None = None, **kwargs) -> BaseChatModel:
|
|
"""Create a chat model instance from the config.
|
|
|
|
Args:
|
|
name: The name of the model to create. If None, the first model in the config will be used.
|
|
thinking_enabled: Enable the model's extended-thinking mode when supported.
|
|
app_config: Explicit application config; falls back to the cached global if omitted.
|
|
model_overrides: Optional per-caller sampling overrides (e.g. a custom
|
|
agent's ``temperature`` / ``max_tokens``) layered on top of the
|
|
model profile. ``None`` values are ignored so an unset override
|
|
never clobbers a profile value. Applied before the thinking / Codex
|
|
transforms so provider-specific normalization (e.g. Codex dropping
|
|
``max_tokens``) still governs an overridden value.
|
|
attach_tracing: When True (default), attach tracing callbacks (Langfuse,
|
|
LangSmith) directly to the model instance. Standalone callers — anything
|
|
that invokes the model outside a LangGraph run that already wires tracing
|
|
at the invocation root (``MemoryUpdater``, ad-hoc utilities, etc.) — keep
|
|
this default so the model-level callback still produces traces. Callers
|
|
that already attach tracing at the graph root (``make_lead_agent``, the
|
|
in-graph ``TitleMiddleware``) MUST pass ``attach_tracing=False``; otherwise
|
|
the same LLM call emits duplicate spans (one rooted at the graph, one at
|
|
the model) and ``session_id`` / ``user_id`` metadata never reach the trace
|
|
because the model becomes a nested observation whose ``langfuse_*`` keys
|
|
get stripped.
|
|
|
|
Returns:
|
|
A chat model instance.
|
|
"""
|
|
config = app_config or get_app_config()
|
|
if name is None:
|
|
name = config.models[0].name
|
|
model_config = config.get_model_config(name)
|
|
if model_config is None:
|
|
raise ValueError(f"Model {name} not found in config") from None
|
|
model_class = resolve_class(model_config.use, BaseChatModel)
|
|
model_settings_from_config = model_config.model_dump(exclude_none=True, exclude=set(_MODEL_METADATA_FIELDS))
|
|
if isinstance(model_config.reasoning, bool | str):
|
|
# Before the capability contract, ``reasoning`` was an unrestricted
|
|
# provider kwarg. ChatOllama uses the boolean form and, for gpt-oss
|
|
# style models, the ``low|medium|high`` level string; keep forwarding
|
|
# both on the legacy path even though contract metadata is excluded
|
|
# above.
|
|
model_settings_from_config["reasoning"] = model_config.reasoning
|
|
# Layer per-caller sampling overrides (e.g. a custom agent's temperature /
|
|
# max_tokens) on top of the profile. Ignore None so an unset override never
|
|
# clobbers a configured profile value. Applied here — before the thinking
|
|
# and Codex transforms below — so provider-specific normalization (Codex
|
|
# dropping max_tokens, thinking disable-paths) still governs the merged
|
|
# value exactly as it would a profile-native one.
|
|
if model_overrides:
|
|
model_settings_from_config.update({key: value for key, value in model_overrides.items() if value is not None})
|
|
# The per-request reasoning effort layers the same way. The regular lead-agent
|
|
# build forwards the key even when None (neither the request nor the custom
|
|
# agent chose one), so it must leave kwargs: a profile that also yields
|
|
# reasoning_effort would otherwise hand the constructor the keyword twice.
|
|
# Codex validates and maps the requested value itself below.
|
|
from deerflow.models.openai_codex_provider import CodexChatModel
|
|
|
|
is_codex_model = issubclass(model_class, CodexChatModel)
|
|
requested_reasoning_effort = kwargs.pop("reasoning_effort", None)
|
|
# Compute effective when_thinking_enabled by merging in the `thinking` shortcut field.
|
|
# The `thinking` shortcut is equivalent to setting when_thinking_enabled["thinking"].
|
|
has_thinking_settings = (model_config.when_thinking_enabled is not None) or (model_config.thinking is not None)
|
|
effective_wte: dict = dict(model_config.when_thinking_enabled) if model_config.when_thinking_enabled else {}
|
|
if model_config.thinking is not None:
|
|
merged_thinking = {**(effective_wte.get("thinking") or {}), **model_config.thinking}
|
|
effective_wte = {**effective_wte, "thinking": merged_thinking}
|
|
|
|
# One normalized reasoning policy for every caller (issue #5073): the lead
|
|
# agent, subagents, summarization, title generation and one-shot utilities
|
|
# all land here, so required thinking, unsupported thinking and restricted
|
|
# effort vocabularies are enforced in exactly one place.
|
|
contract = resolve_reasoning_contract(model_config)
|
|
# Codex's legacy/default-path adapter writes the generic constructor key.
|
|
# A declared custom path takes precedence even for a Codex subclass.
|
|
codex_handles_generic_effort = is_codex_model and (contract.source == "legacy" or contract.effort is None or contract.effort.path == "reasoning_effort")
|
|
if contract.source == "legacy" and thinking_enabled and has_thinking_settings and not contract.supports_thinking:
|
|
raise ValueError(f"Model {name} does not support thinking. Set `supports_thinking` to true in the `config.yaml` to enable thinking.") from None
|
|
try:
|
|
resolved_reasoning = resolve_reasoning_request(contract, thinking_enabled=thinking_enabled, reasoning_effort=requested_reasoning_effort)
|
|
except ReasoningPolicyError as exc:
|
|
raise ValueError(f"Model {name}: {exc}") from exc
|
|
if resolved_reasoning.adjustments:
|
|
logger.info("Model '%s': reasoning request adjusted by its capability contract (%s)", name, ", ".join(resolved_reasoning.adjustments))
|
|
requested_reasoning_effort = resolved_reasoning.reasoning_effort
|
|
if contract.source == "legacy":
|
|
# Profiles without a ``reasoning:`` block keep the historical payload
|
|
# path, driven by the caller's raw thinking flag; the only change from
|
|
# the pre-contract factory is that its templates deep-merge (see the
|
|
# function docstring) instead of replacing whole settings.
|
|
_apply_legacy_thinking_settings(
|
|
model_settings_from_config,
|
|
model_config,
|
|
thinking_enabled=thinking_enabled,
|
|
has_thinking_settings=has_thinking_settings,
|
|
effective_wte=effective_wte,
|
|
requested_reasoning_effort=requested_reasoning_effort,
|
|
is_codex_model=is_codex_model,
|
|
)
|
|
if not contract.supports_reasoning_effort:
|
|
requested_reasoning_effort = None
|
|
model_settings_from_config.pop("reasoning_effort", None)
|
|
else:
|
|
thinking_enabled = resolved_reasoning.thinking_enabled
|
|
_apply_contract_thinking_settings(
|
|
model_settings_from_config,
|
|
contract,
|
|
model_config,
|
|
thinking_enabled=thinking_enabled,
|
|
effective_wte=effective_wte,
|
|
requested_reasoning_effort=requested_reasoning_effort,
|
|
codex_handles_generic_effort=codex_handles_generic_effort,
|
|
)
|
|
|
|
# Normalize the api_base -> base_url alias FIRST, so the downstream OpenAI-compatible
|
|
# heuristics (stream_usage default below / stream_chunk_timeout) see the canonical endpoint key.
|
|
_normalize_openai_base_url(model_class, model_settings_from_config)
|
|
_apply_stream_chunk_timeout_default(model_class, model_settings_from_config)
|
|
|
|
# For Codex Responses API models: map thinking mode to reasoning_effort
|
|
if is_codex_model:
|
|
# The ChatGPT Codex endpoint currently rejects max_tokens/max_output_tokens.
|
|
model_settings_from_config.pop("max_tokens", None)
|
|
|
|
if codex_handles_generic_effort:
|
|
# Use explicit reasoning_effort from frontend if provided (low/medium/high)
|
|
if not thinking_enabled:
|
|
model_settings_from_config["reasoning_effort"] = "none"
|
|
elif requested_reasoning_effort in ("low", "medium", "high", "xhigh"):
|
|
model_settings_from_config["reasoning_effort"] = requested_reasoning_effort
|
|
elif "reasoning_effort" not in model_settings_from_config:
|
|
model_settings_from_config["reasoning_effort"] = "medium"
|
|
|
|
# For MindIE models: enforce conservative retry defaults.
|
|
# Timeout normalization is handled inside MindIEChatModel itself.
|
|
if getattr(model_class, "__name__", "") == "MindIEChatModel":
|
|
# Enforce max_retries constraint to prevent cascading timeouts.
|
|
model_settings_from_config["max_retries"] = model_settings_from_config.get("max_retries", 1)
|
|
|
|
# Ensure stream_usage is enabled so that token usage metadata is available
|
|
# in streaming responses. LangChain's BaseChatOpenAI only defaults
|
|
# stream_usage=True when no custom base_url/api_base is set, so models
|
|
# hitting third-party endpoints (e.g. doubao, deepseek) silently lose
|
|
# usage data. We default it to True unless explicitly configured.
|
|
if "stream_usage" not in model_settings_from_config and "stream_usage" not in kwargs:
|
|
if "stream_usage" in getattr(model_class, "model_fields", {}):
|
|
model_settings_from_config["stream_usage"] = True
|
|
|
|
# Translate the declared context window into the langchain profile so
|
|
# profile-dependent features (e.g. SummarizationMiddleware fraction triggers,
|
|
# which resolve thresholds from profile["max_input_tokens"]) work for
|
|
# third-party OpenAI-compatible models whose SDK ships no profile of its
|
|
# own (#3103). ``profile`` is a metadata-only BaseChatModel field
|
|
# (exclude=True) and never reaches the provider request payload. An
|
|
# explicit profile from a caller or model_overrides is never clobbered.
|
|
translate_context_window = bool(model_config.context_window) and "profile" not in kwargs and "profile" not in model_settings_from_config
|
|
|
|
if model_config.request_admission is not None:
|
|
from deerflow.models.request_admission import get_request_admission
|
|
|
|
if "rate_limiter" in kwargs or "rate_limiter" in model_settings_from_config:
|
|
raise ValueError("request_admission cannot be combined with a custom rate_limiter")
|
|
model_settings_from_config["rate_limiter"] = get_request_admission(name, model_config.request_admission)
|
|
# SDK-internal retries do not re-enter BaseChatModel's admission hook.
|
|
# Keep retries at the middleware layer where each attempt is paced.
|
|
if "max_retries" in model_class.model_fields:
|
|
kwargs.pop("max_retries", None)
|
|
model_settings_from_config["max_retries"] = 0
|
|
|
|
_warn_unknown_model_settings(model_class, name, model_settings_from_config)
|
|
|
|
# 配置提供默认值,调用方显式传入的非空参数统一覆盖配置。
|
|
# 先合并再展开,避免同名字段通过两个 **dict 传入时触发 TypeError。
|
|
effective_model_settings = dict(model_settings_from_config)
|
|
effective_model_settings.update({key: value for key, value in kwargs.items() if value is not None})
|
|
model_instance = model_class(**effective_model_settings)
|
|
|
|
if translate_context_window:
|
|
# Applied *after* construction and merged into the provider's inferred
|
|
# profile: passing ``profile`` to the constructor would REPLACE the whole
|
|
# inferred metadata (tool_calling, structured_output, io capabilities,
|
|
# output limits) with just this one key. The declared window wins on
|
|
# max_input_tokens itself — the operator set it precisely because the
|
|
# inferred (or absent) value doesn't match their gateway.
|
|
inferred_profile = getattr(model_instance, "profile", None)
|
|
model_instance.profile = {**(inferred_profile or {}), "max_input_tokens": model_config.context_window}
|
|
|
|
if attach_tracing:
|
|
callbacks = build_tracing_callbacks()
|
|
if callbacks:
|
|
existing_callbacks = model_instance.callbacks or []
|
|
model_instance.callbacks = [*existing_callbacks, *callbacks]
|
|
logger.debug(f"Tracing attached to model '{name}' with providers={len(callbacks)}")
|
|
return model_instance
|