deer-flow/frontend/tests/unit/core/models/reasoning.test.ts
Nan Gao bcaf6db36a
feat(models): add declarative reasoning capability contract (#5780)
* feat(models): add declarative reasoning capability contract (#5073)

Introduce an optional per-model `reasoning:` block beside the legacy
`supports_thinking` / `supports_reasoning_effort` booleans. It declares
thinking availability (unsupported | optional | required), the accepted
effort values with aliases, default, and serialization path, the payload
dialect, and the reasoning-history requirement. When present the booleans
are derived from it and contradictory profiles fail at config load.

`deerflow.models.reasoning` normalizes legacy and declared profiles into
one immutable contract and resolves every caller's generic request against
it. `create_chat_model` is the single enforcement point: required-thinking
models never receive a synthesized disable payload, unsupported effort
values never reach the provider, and the dialect synthesizes the on/off
payload when no template exists. Legacy profiles keep the historical path
byte-for-byte. The lead agent and subagent descriptor resolve the same
policy first so run metadata reports effective values.

`/api/models` and `DeerFlowClient` project the contract as `reasoning`;
the frontend derives mode clamping and effort choices from it through
`core/models/reasoning.ts`, and user preferences accept provider-specific
effort tokens. The setup wizard's Z.AI GLM-5.3-Flash profile migrates to
the contract, restoring its low/high/max effort control.

* docs(models): move the reasoning contract guidance into models/AGENTS.md

backend/AGENTS.md sits within 46 bytes of the 96 KiB effective-chain hard
limit for the middlewares, runtime, sandbox, and subagents guides, so the
new section pushed scripts/check_agent_guidance.py over it. The models
guide is the nearest file for the factory and the contract module, and
its chain has ample room.

* fix(models): address review on the reasoning contract

- vLLM dialect: the enable path now mirrors whichever chat_template_kwargs
  switch the template declares, like the disable path already did, so a
  template using the older `thinking` key no longer also grows
  `enable_thinking`.
- Validate operator-supplied effort values at `effort.path` inside
  `when_thinking_enabled`, `when_thinking_disabled`, and the `thinking`
  shortcut against the contract, not only the profile-level value; they are
  forwarded when the caller chooses nothing, so the "unknown values never
  reach the provider" guarantee now holds for them too.
- Constrain `effort.path` to dotted identifiers and reject single-segment
  paths that would replace a whole mapping (`extra_body`, `thinking`,
  `model_kwargs`, `default_headers`, `default_query`).
- Z.AI GLM-5.3-Flash profile: `default: high` instead of `max`, because the
  default also governs summarization, title generation, and subagent calls
  that never choose an effort; `max` stays selectable. Documented in the
  config example, the configuration manual, and the design note.

* fix(models): close reasoning contract review gaps

* fix(models): preserve native Ollama reasoning config

* fix(models): keep native reasoning strings and project reasoning into the descriptor

- `ModelConfig.reasoning` also accepts the level string ChatOllama takes
  for gpt-oss style models (`low` / `medium` / `high`); the factory forwards
  both the boolean and the string on the legacy path, so profiles that
  passed `reasoning: high` through before the contract field existed keep
  loading and keep sending it.
- The assembly descriptor no longer excludes `reasoning` from
  `model_parameters`: the native ChatOllama value and a declared contract's
  dialect/history/effort change the request payload, so they now move the
  assembly fingerprint.
- e2e: the sidecar spec pinned the pre-contract body where a model without
  effort control still received `reasoning_effort: minimal`; the preset is
  now dropped for such models (the Gateway discarded it anyway), so the
  assertion expects the key to be absent.
2026-09-24 20:39:30 +08:00

161 lines
5.0 KiB
TypeScript

import { describe, expect, it } from "@rstest/core";
import {
getReasoningCapabilities,
getReasoningEffortOptions,
getResolvedMode,
isThinkingRequired,
reasoningEffortForMode,
resolveReasoningEffort,
supportsThinking,
} from "@/core/models/reasoning";
import type { Model } from "@/core/models/types";
function model(overrides: Partial<Model>): Model {
return {
id: "m",
name: "m",
model: "m",
display_name: "M",
...overrides,
};
}
const legacyThinking = model({
supports_thinking: true,
supports_reasoning_effort: true,
});
const legacyPlain = model({});
const glm = model({
supports_thinking: true,
supports_reasoning_effort: true,
reasoning: {
thinking: "required",
effort: {
values: ["low", "high", "max"],
default: "max",
aliases: { minimal: "low", medium: "high" },
},
history: "clear",
source: "contract",
},
});
const optionalNoEffort = model({
reasoning: {
thinking: "optional",
effort: null,
history: null,
source: "contract",
},
});
const restrictedNoDefault = model({
reasoning: {
thinking: "optional",
effort: { values: ["low", "high"], default: null, aliases: {} },
history: null,
source: "contract",
},
});
describe("getReasoningCapabilities", () => {
it("falls back to the legacy booleans when the Gateway omits the contract", () => {
expect(getReasoningCapabilities(legacyThinking)).toEqual({
thinking: "optional",
effort: {
values: ["minimal", "low", "medium", "high"],
default: null,
aliases: {},
},
history: null,
source: "legacy",
});
expect(getReasoningCapabilities(legacyPlain).thinking).toBe("unsupported");
expect(getReasoningCapabilities(legacyPlain).effort).toBeNull();
});
it("treats a missing model as unsupported", () => {
expect(getReasoningCapabilities(undefined).thinking).toBe("unsupported");
expect(supportsThinking(undefined)).toBe(false);
expect(getReasoningEffortOptions(null)).toEqual([]);
});
it("prefers the projected contract over the booleans", () => {
expect(getReasoningCapabilities(glm).source).toBe("contract");
expect(isThinkingRequired(glm)).toBe(true);
expect(getReasoningEffortOptions(glm)).toEqual(["low", "high", "max"]);
});
});
describe("resolveReasoningEffort", () => {
it("keeps an unset value unset so the Gateway applies the contract default", () => {
expect(resolveReasoningEffort(glm, undefined)).toBeUndefined();
});
it("keeps advertised legacy values and drops remembered contract-only values", () => {
expect(resolveReasoningEffort(legacyThinking, "minimal")).toBe("minimal");
expect(resolveReasoningEffort(legacyThinking, "xhigh")).toBeUndefined();
expect(
resolveReasoningEffort(
legacyThinking,
resolveReasoningEffort(glm, "max"),
),
).toBeUndefined();
expect(resolveReasoningEffort(legacyPlain, "high")).toBeUndefined();
});
it("maps generic presets through the contract aliases", () => {
expect(resolveReasoningEffort(glm, "minimal")).toBe("low");
expect(resolveReasoningEffort(glm, "medium")).toBe("high");
expect(resolveReasoningEffort(glm, "max")).toBe("max");
});
it("falls back to the contract default for unknown values", () => {
expect(resolveReasoningEffort(glm, "xhigh")).toBe("max");
expect(
resolveReasoningEffort(restrictedNoDefault, "medium"),
).toBeUndefined();
});
it("drops any value for a model without effort control", () => {
expect(resolveReasoningEffort(optionalNoEffort, "high")).toBeUndefined();
});
});
describe("getResolvedMode", () => {
it("clamps every mode to flash on a model without thinking", () => {
expect(getResolvedMode("pro", legacyPlain)).toBe("flash");
expect(getResolvedMode(undefined, legacyPlain)).toBe("flash");
});
it("keeps the requested mode on an optional-thinking model", () => {
expect(getResolvedMode("flash", legacyThinking)).toBe("flash");
expect(getResolvedMode("ultra", legacyThinking)).toBe("ultra");
expect(getResolvedMode(undefined, legacyThinking)).toBe("pro");
});
it("never yields flash on a required-thinking model", () => {
expect(getResolvedMode("flash", glm)).toBe("thinking");
expect(getResolvedMode(undefined, glm)).toBe("pro");
expect(getResolvedMode("ultra", glm)).toBe("ultra");
});
});
describe("reasoningEffortForMode", () => {
it("uses the generic presets for legacy models", () => {
expect(reasoningEffortForMode("flash", legacyThinking)).toBe("minimal");
expect(reasoningEffortForMode("thinking", legacyThinking)).toBe("low");
expect(reasoningEffortForMode("pro", legacyThinking)).toBe("medium");
expect(reasoningEffortForMode("ultra", legacyThinking)).toBe("high");
});
it("maps the presets onto a restricted contract", () => {
expect(reasoningEffortForMode("thinking", glm)).toBe("low");
expect(reasoningEffortForMode("pro", glm)).toBe("high");
expect(reasoningEffortForMode("ultra", glm)).toBe("high");
});
});