From 4470932118a6ea193542bc74e864eee4622b3ba3 Mon Sep 17 00:00:00 2001 From: wutongyuonce <147830929+wutongyuonce@users.noreply.github.com> Date: Sat, 12 Sep 2026 10:57:40 +0800 Subject: [PATCH] feat(extensions): allow constructor kwargs on config-declared middlewares (#5312) * feat(extensions): allow constructor kwargs on config-declared middlewares extensions.middlewares entries may be a class-path string or {class, kwargs}. String entries keep the zero-argument constructor. Unknown fields and blank class paths fail at config validation. Constructor errors still fail at agent creation. Fixes #5311 * fix(extensions): coerce middleware kwargs to JSON types YAML timestamps became datetime objects while JSON kept strings, so constructors and to_file_dict() json.dump saw different types. Validate kwargs as JSON types at config load, stringify dates, reject NaN and other non-JSON values, and cover the raw-dict loader branch. * style(extensions): wrap middleware Field description for ruff E501 make lint failed: the middlewares description was 289 chars (limit 240). Wrap it and run ruff format on the two files this PR last touched. * docs: compact configured middleware guidance to satisfy size limit --------- Co-authored-by: Willem Jiang --- README.md | 2 +- backend/CONTRIBUTING.md | 8 +- .../deerflow/agents/middlewares/AGENTS.md | 2 +- .../middlewares/configured_extensions.py | 33 ++-- .../harness/deerflow/config/AGENTS.md | 2 +- .../deerflow/config/extensions_config.py | 85 ++++++++- backend/tests/test_app_config_reload.py | 31 ++++ backend/tests/test_configured_extensions.py | 161 ++++++++++++++++++ config.example.yaml | 14 +- .../src/content/en/harness/customization.mdx | 5 +- .../src/content/en/harness/middlewares.mdx | 5 +- .../src/content/zh/harness/customization.mdx | 5 +- .../src/content/zh/harness/middlewares.mdx | 5 +- 13 files changed, 334 insertions(+), 24 deletions(-) create mode 100644 backend/tests/test_configured_extensions.py diff --git a/README.md b/README.md index 79d8739b0..c72b13e88 100644 --- a/README.md +++ b/README.md @@ -1027,7 +1027,7 @@ as the heading; their content remains available to the agent. Advanced deployments can enable pluggable authorization with `authorization.enabled` in `config.yaml`. A configured `AuthorizationProvider` filters denied tools before they reach the model or deferred-tool catalog, then the same provider is checked again before every business-tool execution through the existing guardrail middleware. Gateway `threads:*` and `runs:*` route permissions are derived from the same provider, while existing owner checks and admin-only management gates remain in force. Every HTTP route that starts or enables a future Agent run requires `runs:create`: this includes the stateless `POST /api/runs/stream` and `POST /api/runs/wait` endpoints plus scheduled-task create, update, resume, and manual-trigger mutations. Scheduled-task mutations retain their existing `threads:write` requirement, and the stateless routes separately enforce ownership when the optional thread ID is supplied in the request body. A generated `tool_search` may bypass the second tool check only when it fronts the current build's already-filtered deferred catalog. Model access follows the same provider: the Gateway `models` list is filtered per principal, `model:use` is enforced on model detail requests and again when the runtime resolves the agent's model, and a denied default model falls back to the first remaining candidate that also passes `model:use`. The built-in RBAC provider supports per-role `tools`, `routes`, `models`, `skills`, and `sandbox` allow/deny policies and validates that `default_role` names a configured role; authorization is disabled by default. See `config.example.yaml` and the [authorization RFC](docs/plans/2026-07-10-pluggable-authorization-rfc.md). -Advanced deployments can also extend the agent runtime itself by declaring zero-argument `AgentMiddleware` classes under `extensions.middlewares` in `config.yaml` or `extensions_config.json`. DeerFlow loads the same configured class list into the lead-agent and subagent pipelines after their built-in runtime middlewares and loop/token guards, but before the terminal-response/safety/clarification tail, so enterprise forks can add domain guardrails, tool-call governance, or observability hooks without patching the built-in middleware builders. Missing packages, invalid classes, and broken modules fail loudly at agent creation. Treat `config.yaml` and `extensions_config.json` as trusted operator-controlled files: middleware paths are code execution, just like custom tool, model, sandbox, guardrail, MCP server, and MCP interceptor declarations. Gateway skill/MCP toggle endpoints preserve this field but do not expose an API write path for `extensions.middlewares`. Per-context parameterization and separate lead-only/subagent-only middleware lists are not supported yet. +Advanced deployments can also extend the agent runtime itself by declaring `AgentMiddleware` classes under `extensions.middlewares` in `config.yaml` or `extensions_config.json`. Each entry is a `module.path:ClassName` string (zero-argument constructor) or an object `{class, kwargs}` whose `kwargs` are passed to the constructor. `kwargs` values must be JSON types (object, array, string, number, boolean, or null); YAML dates and timestamps are coerced to ISO strings so they match JSON. DeerFlow loads the same configured list into the lead-agent and subagent pipelines after their built-in runtime middlewares and loop/token guards, but before the terminal-response/safety/clarification tail, so enterprise forks can add domain guardrails, tool-call governance, or observability hooks without patching the built-in middleware builders. Missing packages, invalid classes, broken modules, and constructor errors fail loudly at agent creation. Treat `config.yaml` and `extensions_config.json` as trusted operator-controlled files: middleware paths are code execution, just like custom tool, model, sandbox, guardrail, MCP server, and MCP interceptor declarations. Gateway skill/MCP toggle endpoints preserve this field but do not expose an API write path for `extensions.middlewares`. Separate lead-only/subagent-only middleware lists are not supported yet. For packaged and configurable runtime integrations, use DeerFlow's extension manager. It accepts a Python package requirement, a public HTTPS Git URL, or a local directory, installs the diff --git a/backend/CONTRIBUTING.md b/backend/CONTRIBUTING.md index ff01ad547..de9719142 100644 --- a/backend/CONTRIBUTING.md +++ b/backend/CONTRIBUTING.md @@ -299,12 +299,18 @@ class MyMiddleware(AgentMiddleware[AgentState]): Lifecycle hooks can return a dictionary of state updates, which LangChain merges into the agent state, or `None` when they only observe state. -2. Register the zero-argument middleware class in `config.yaml`: +2. Register the middleware class in `config.yaml`. A class path uses the + zero-argument constructor; `{class, kwargs}` passes constructor arguments. + `kwargs` values must be JSON types (object, array, string, number, boolean, + or null); YAML dates and timestamps are coerced to ISO strings so they match JSON: ```yaml extensions: middlewares: - deerflow.agents.middlewares.my_middleware:MyMiddleware + - class: deerflow.agents.middlewares.my_middleware:MyMiddleware + kwargs: + max_tool_calls: 5 ``` Configured middleware runs after the built-in middleware and optional loop/token diff --git a/backend/packages/harness/deerflow/agents/middlewares/AGENTS.md b/backend/packages/harness/deerflow/agents/middlewares/AGENTS.md index 9f09c5771..0d828b9a3 100644 --- a/backend/packages/harness/deerflow/agents/middlewares/AGENTS.md +++ b/backend/packages/harness/deerflow/agents/middlewares/AGENTS.md @@ -104,7 +104,7 @@ Before changing a later authorization phase, read the [authorization RFC](../../ fallback identity, cleanup/LRU/reset, severity ordering, and test invariants. 30. **TokenBudgetMiddleware** - *(optional, if `token_budget.enabled`)* Enforces per-run token limits 31. **Custom middlewares** - *(optional)* Any `custom_middlewares` passed to `build_middlewares` are injected here, before config-declared extensions and the terminal-response/safety/clarification tail -32. **Configured extension middlewares** - *(optional, if `extensions.middlewares` is set in `config.yaml` or `extensions_config.json`)* Zero-argument `AgentMiddleware` classes loaded from `module.path:ClassName` entries via `deerflow.reflection.resolve_class`. Missing packages, invalid classes, and broken modules fail loudly at agent creation. These run after built-ins/programmatic custom middleware and after the lead/subagent loop/token guards, but before the terminal-response/safety/clarification tail; subagents receive the same configured extension middleware class list before their safety tail. Treat these files as trusted operator config because middleware paths instantiate arbitrary code. Gateway skill/MCP toggle endpoints preserve this field through `to_file_dict()` but must not add a write path for `extensions.middlewares` without an explicit trust-boundary review. Lead-only vs subagent-only middleware lists and per-context constructor parameters are not expressible in this MVP. +32. **Configured extension middlewares** - `extensions.middlewares` in `config.yaml` or `extensions_config.json` optionally accepts `module.path:ClassName` strings or `{class, kwargs}` objects. `deerflow.reflection.resolve_class` loads `AgentMiddleware` classes; import, class, and constructor errors fail agent creation. `kwargs` must be JSON-compatible; YAML dates/timestamps become ISO strings. Order: built-ins/custom and loop/token guards → extensions → terminal-response/safety/clarification tail. Subagents share the list before their safety tail; separate lead/subagent lists are unsupported. Trusted operator config only: paths instantiate arbitrary code. Gateway skill/MCP toggles preserve it via `to_file_dict()`; adding an API write path requires explicit trust-boundary review. 33. **TerminalResponseMiddleware** - When a provider returns an empty terminal `AIMessage` after tool execution, injects a hidden recovery prompt and retries the model once; a second empty response is replaced in checkpoint state by a visible error fallback marked for the run worker, so the run finishes as an error instead of a silent success 34. **ModelLengthFinishReasonMiddleware** - Records `stop_reason=model_length_capped` when provider-specific length detectors match a terminal `AIMessage` without tool-call intent (`finish_reason=length` / `MAX_TOKENS`, or `stop_reason=max_tokens`), preserving the original assistant content and never reparsing textual tool-call-like envelopes 35. **SafetyFinishReasonMiddleware** - *(optional, if `safety_finish_reason.enabled`)* Suppresses tool execution when the provider safety-terminated the response (e.g. `finish_reason=content_filter`); registered after terminal-response/custom/configured middlewares so LangChain's reverse-order `after_model` dispatch runs it first diff --git a/backend/packages/harness/deerflow/agents/middlewares/configured_extensions.py b/backend/packages/harness/deerflow/agents/middlewares/configured_extensions.py index ef811426f..0c82acaa8 100644 --- a/backend/packages/harness/deerflow/agents/middlewares/configured_extensions.py +++ b/backend/packages/harness/deerflow/agents/middlewares/configured_extensions.py @@ -1,10 +1,11 @@ """Config-declared agent middleware loading.""" import logging -from typing import TYPE_CHECKING +from typing import TYPE_CHECKING, Any from langchain.agents.middleware import AgentMiddleware +from deerflow.config.extensions_config import ConfiguredMiddlewareSpec from deerflow.reflection import resolve_class if TYPE_CHECKING: @@ -13,22 +14,34 @@ if TYPE_CHECKING: logger = logging.getLogger(__name__) +def _middleware_constructor_args(entry: str | ConfiguredMiddlewareSpec | dict[str, Any]) -> tuple[str, dict[str, Any]]: + """Return ``(class_path, kwargs)`` for one config entry.""" + if isinstance(entry, str): + return entry, {} + if isinstance(entry, ConfiguredMiddlewareSpec): + return entry.class_path, dict(entry.kwargs) + parsed = ConfiguredMiddlewareSpec.model_validate(entry) + return parsed.class_path, dict(parsed.kwargs) + + def load_configured_extension_middlewares(app_config: "AppConfig") -> list[AgentMiddleware]: """Instantiate config-declared agent middlewares. - Each entry is a zero-argument ``AgentMiddleware`` class path in - ``module.path:ClassName`` format. Import, attribute, and subclass validation - intentionally go through the shared reflection resolver so failures carry - the same actionable dependency hints as models, tools, sandbox providers, - and guardrail providers. + Each entry is a ``module.path:ClassName`` string or a + ``ConfiguredMiddlewareSpec`` (``class`` plus optional ``kwargs``). + Import, attribute, and subclass validation intentionally go through the + shared reflection resolver so failures carry the same actionable + dependency hints as models, tools, sandbox providers, and guardrail + providers. Constructor errors fail loudly at agent creation. """ middlewares: list[AgentMiddleware] = [] - for middleware_path in list(app_config.extensions.middlewares or []): - middleware_cls = resolve_class(middleware_path, AgentMiddleware) + for entry in list(app_config.extensions.middlewares or []): + class_path, kwargs = _middleware_constructor_args(entry) + middleware_cls = resolve_class(class_path, AgentMiddleware) try: - middleware = middleware_cls() + middleware = middleware_cls(**kwargs) except Exception: - logger.exception("Failed to instantiate configured extension middleware %s", middleware_path) + logger.exception("Failed to instantiate configured extension middleware %s", class_path) raise middlewares.append(middleware) return middlewares diff --git a/backend/packages/harness/deerflow/config/AGENTS.md b/backend/packages/harness/deerflow/config/AGENTS.md index feec437e5..9dc181c70 100644 --- a/backend/packages/harness/deerflow/config/AGENTS.md +++ b/backend/packages/harness/deerflow/config/AGENTS.md @@ -66,6 +66,6 @@ Extensions are optional only in the fallback *search* mode (priority 3-4 above): - `mcpServers` - Map of server name → config (enabled, type, command, args, env, url, headers, oauth, description, `routing`, `tools`, `tool_call_timeout`, `session_init_timeout`). `routing.mode="prefer"` emits `` prompt guidance; if `tool_search` defers the hinted tool, `McpRoutingMiddleware` can also auto-promote matching deferred schemas before the model call. It does not hard-disable other tools. `session_init_timeout` (default `DEFAULT_MCP_SESSION_INIT_TIMEOUT` = 60s, `null` to disable) bounds server bring-up: tool discovery and persistent stdio session initialization, so a hung server cannot block agent construction indefinitely; durable HTTP/SSE task calls use it for their ephemeral session initialization too. `tool_call_timeout` bounds individual stdio calls and durable-task calls on every transport; other HTTP/SSE tools use transport-level timeouts. - `tool_search.auto_promote_top_k` - Global MCP routing auto-promote breadth. Default `3`, clamped to `1..5`; applies only when `tool_search.enabled=true` and only to deferred MCP tools with `routing.mode="prefer"` and non-empty keywords. For lead agents the deferred catalog is built from the full configured MCP set; auto-promotion never grants authority because an active skill's runtime policy still filters model-visible schemas, `tool_search` results, and execution. - `skills` - Map of skill name → state (enabled) -- `middlewares` - Zero-argument `AgentMiddleware` class paths for lead and subagent runtime extension. `config.yaml -> extensions` can override these fields after validation; overrides are replace-per-field, not list concatenation. +- `middlewares` - `AgentMiddleware` entries for lead and subagent runtime extension: class-path strings or `{class, kwargs}` objects. `kwargs` values must be JSON types; YAML dates and timestamps are coerced to ISO strings so they match JSON. `config.yaml -> extensions` can override these fields after validation; overrides are replace-per-field, not list concatenation. Gateway API endpoints and `DeerFlowClient` methods can modify MCP servers and skill state at runtime; their `extensions_config.json` writes use the shared atomic replacement helper, while `middlewares` remains an operator-controlled config-file extension point. diff --git a/backend/packages/harness/deerflow/config/extensions_config.py b/backend/packages/harness/deerflow/config/extensions_config.py index 24bbf8e10..09a5218a5 100644 --- a/backend/packages/harness/deerflow/config/extensions_config.py +++ b/backend/packages/harness/deerflow/config/extensions_config.py @@ -3,12 +3,14 @@ import errno import json import logging +import math import os import stat import tempfile import threading from collections.abc import Iterator from contextlib import contextmanager +from datetime import date, datetime, time from pathlib import Path from typing import Any, Literal @@ -23,6 +25,7 @@ from deerflow.constants import ( logger = logging.getLogger(__name__) +_JSON_KWARGS_ERROR = "middleware kwargs values must be JSON types (object, array, string, number, boolean, or null)" _non_atomic_fallback_targets: set[Path] = set() _non_atomic_fallback_targets_lock = threading.Lock() @@ -307,12 +310,76 @@ class SkillStateConfig(BaseModel): enabled: bool = Field(default=True, description="Whether this skill is enabled") +def _coerce_json_kwargs_value(value: Any) -> Any: + """Keep JSON types; stringify YAML timestamps so they match JSON strings.""" + if value is None or isinstance(value, (str, bool, int)): + return value + if isinstance(value, float): + if not math.isfinite(value): + raise ValueError(_JSON_KWARGS_ERROR) + return value + if isinstance(value, datetime): + return value.isoformat() + if isinstance(value, date): + return value.isoformat() + if isinstance(value, time): + return value.isoformat() + if isinstance(value, dict): + if not all(isinstance(key, str) and key.strip() for key in value): + raise ValueError("middleware kwargs keys must be non-empty strings") + return {key: _coerce_json_kwargs_value(item) for key, item in value.items()} + if isinstance(value, list): + return [_coerce_json_kwargs_value(item) for item in value] + raise ValueError(_JSON_KWARGS_ERROR) + + +class ConfiguredMiddlewareSpec(BaseModel): + """One config-declared AgentMiddleware with optional constructor arguments.""" + + class_path: str = Field( + ..., + alias="class", + min_length=1, + description="AgentMiddleware class path in 'module.path:ClassName' form.", + ) + kwargs: dict[str, Any] = Field( + default_factory=dict, + description=("Keyword arguments passed to the middleware constructor. Values must be JSON types (object, array, string, number, boolean, or null); YAML dates and timestamps are coerced to ISO strings so they match JSON."), + ) + model_config = ConfigDict(extra="forbid", populate_by_name=True) + + @field_validator("class_path") + @classmethod + def _strip_class_path(cls, value: str) -> str: + stripped = value.strip() + if not stripped: + raise ValueError("middleware class path must be a non-empty string") + return stripped + + @field_validator("kwargs", mode="before") + @classmethod + def _kwargs_none_is_empty(cls, value: Any) -> Any: + return {} if value is None else value + + @field_validator("kwargs") + @classmethod + def _kwargs_are_json_object(cls, value: dict[str, Any]) -> dict[str, Any]: + coerced = _coerce_json_kwargs_value(value) + json.dumps(coerced) + return coerced + + class ExtensionsConfig(BaseModel): """Unified configuration for MCP servers and skills.""" - middlewares: list[str] = Field( + middlewares: list[str | ConfiguredMiddlewareSpec] = Field( default_factory=list, - description="AgentMiddleware class paths loaded into the lead-agent and subagent middleware chains. Each entry uses 'module.path:ClassName'.", + description=( + "AgentMiddleware entries loaded into the lead-agent and subagent middleware chains. " + "Each entry is a 'module.path:ClassName' string or an object with 'class' and optional " + "'kwargs'. kwargs values must be JSON types; YAML dates and timestamps are coerced to " + "ISO strings." + ), ) mcp_servers: dict[str, McpServerConfig] = Field( default_factory=dict, @@ -325,6 +392,20 @@ class ExtensionsConfig(BaseModel): ) model_config = ConfigDict(extra="allow", populate_by_name=True) + @field_validator("middlewares") + @classmethod + def _normalize_middleware_entries(cls, value: list[str | ConfiguredMiddlewareSpec]) -> list[str | ConfiguredMiddlewareSpec]: + normalized: list[str | ConfiguredMiddlewareSpec] = [] + for entry in value: + if isinstance(entry, str): + stripped = entry.strip() + if not stripped: + raise ValueError("middleware class path must be a non-empty string") + normalized.append(stripped) + continue + normalized.append(entry) + return normalized + @model_validator(mode="after") def _validate_task_server_names_fit_storage(self) -> "ExtensionsConfig": for server_name, server in self.mcp_servers.items(): diff --git a/backend/tests/test_app_config_reload.py b/backend/tests/test_app_config_reload.py index 9baa71ff8..aacad592a 100644 --- a/backend/tests/test_app_config_reload.py +++ b/backend/tests/test_app_config_reload.py @@ -278,6 +278,37 @@ def test_app_config_loads_extension_middlewares_from_extensions_config(tmp_path, assert config.extensions.middlewares == ["pkg.from_file:FileMiddleware"] +def test_app_config_loads_middleware_kwargs_from_config_yaml(tmp_path, monkeypatch): + from deerflow.config.extensions_config import ConfiguredMiddlewareSpec + + config_path = tmp_path / "config.yaml" + extensions_path = tmp_path / "extensions_config.json" + extensions_path.write_text( + json.dumps({"mcpServers": {}, "skills": {}, "middlewares": ["pkg.from_file:FileMiddleware"]}), + encoding="utf-8", + ) + _write_config_with_sections( + config_path, + { + "extensions": { + "middlewares": [ + "pkg.from_yaml:PlainMiddleware", + {"class": "pkg.from_yaml:KwargsMiddleware", "kwargs": {"max_tool_calls": 4}}, + ], + } + }, + ) + monkeypatch.setenv("DEER_FLOW_EXTENSIONS_CONFIG_PATH", str(extensions_path)) + + config = AppConfig.from_file(str(config_path)) + + assert config.extensions.middlewares[0] == "pkg.from_yaml:PlainMiddleware" + spec = config.extensions.middlewares[1] + assert isinstance(spec, ConfiguredMiddlewareSpec) + assert spec.class_path == "pkg.from_yaml:KwargsMiddleware" + assert spec.kwargs == {"max_tool_calls": 4} + + def test_app_config_defaults_empty_database_to_sqlite(tmp_path, monkeypatch): config_path = tmp_path / "config.yaml" extensions_path = tmp_path / "extensions_config.json" diff --git a/backend/tests/test_configured_extensions.py b/backend/tests/test_configured_extensions.py new file mode 100644 index 000000000..c6e851853 --- /dev/null +++ b/backend/tests/test_configured_extensions.py @@ -0,0 +1,161 @@ +"""Config-declared extension middleware loading, including constructor kwargs.""" + +import json +from datetime import date, datetime +from types import SimpleNamespace + +import pytest +from langchain.agents.middleware import AgentMiddleware +from pydantic import ValidationError + +from deerflow.agents.middlewares.configured_extensions import load_configured_extension_middlewares +from deerflow.config.extensions_config import ConfiguredMiddlewareSpec, ExtensionsConfig + + +class RecordingMiddleware(AgentMiddleware): + """Test double that records constructor kwargs.""" + + def __init__(self, max_tool_calls: int = 10): + super().__init__() + self.max_tool_calls = max_tool_calls + + +class ZeroArgMiddleware(AgentMiddleware): + def __init__(self): + super().__init__() + + +def _config(*entries) -> SimpleNamespace: + return SimpleNamespace(extensions=SimpleNamespace(middlewares=list(entries))) + + +def test_string_entry_still_zero_arg(): + loaded = load_configured_extension_middlewares(_config(f"{__name__}:ZeroArgMiddleware")) + + assert len(loaded) == 1 + assert isinstance(loaded[0], ZeroArgMiddleware) + + +def test_dict_entry_passes_constructor_kwargs(): + entry = ConfiguredMiddlewareSpec.model_validate({"class": f"{__name__}:RecordingMiddleware", "kwargs": {"max_tool_calls": 3}}) + + loaded = load_configured_extension_middlewares(_config(entry)) + + assert len(loaded) == 1 + assert isinstance(loaded[0], RecordingMiddleware) + assert loaded[0].max_tool_calls == 3 + + +def test_raw_dict_entry_passes_constructor_kwargs(): + loaded = load_configured_extension_middlewares(_config({"class": f"{__name__}:RecordingMiddleware", "kwargs": {"max_tool_calls": 2}})) + + assert len(loaded) == 1 + assert isinstance(loaded[0], RecordingMiddleware) + assert loaded[0].max_tool_calls == 2 + + +def test_malformed_raw_dict_fails_at_load(): + with pytest.raises(ValidationError): + load_configured_extension_middlewares(_config({"class": f"{__name__}:RecordingMiddleware", "apply_to": "lead"})) + + +def test_empty_kwargs_matches_zero_arg_constructor(): + entry = ConfiguredMiddlewareSpec.model_validate({"class": f"{__name__}:RecordingMiddleware"}) + + loaded = load_configured_extension_middlewares(_config(entry)) + + assert loaded[0].max_tool_calls == 10 + + +def test_unknown_constructor_kwarg_fails_loudly(): + entry = ConfiguredMiddlewareSpec.model_validate({"class": f"{__name__}:ZeroArgMiddleware", "kwargs": {"not_a_param": 1}}) + + with pytest.raises(TypeError): + load_configured_extension_middlewares(_config(entry)) + + +def test_extensions_config_keeps_string_entries(): + config = ExtensionsConfig.model_validate({"middlewares": ["pkg:Middleware"]}) + + assert config.middlewares == ["pkg:Middleware"] + + +def test_extensions_config_parses_class_and_kwargs(): + config = ExtensionsConfig.model_validate( + { + "middlewares": [ + "pkg:Plain", + {"class": "pkg:WithArgs", "kwargs": {"max_tool_calls": 5}}, + ] + } + ) + + assert config.middlewares[0] == "pkg:Plain" + spec = config.middlewares[1] + assert isinstance(spec, ConfiguredMiddlewareSpec) + assert spec.class_path == "pkg:WithArgs" + assert spec.kwargs == {"max_tool_calls": 5} + + +def test_extensions_config_rejects_unknown_entry_fields(): + with pytest.raises(ValidationError): + ExtensionsConfig.model_validate({"middlewares": [{"class": "pkg:Middleware", "apply_to": "lead"}]}) + + +def test_extensions_config_rejects_blank_class_path(): + with pytest.raises(ValidationError): + ExtensionsConfig.model_validate({"middlewares": [{"class": " "}]}) + + +def test_extensions_config_rejects_blank_string_entry(): + with pytest.raises(ValidationError): + ExtensionsConfig.model_validate({"middlewares": [" "]}) + + +def test_extensions_config_strips_string_entries(): + config = ExtensionsConfig.model_validate({"middlewares": [" pkg:Plain "]}) + + assert config.middlewares == ["pkg:Plain"] + + +def test_kwargs_yaml_date_normalizes_to_iso_string(): + spec = ConfiguredMiddlewareSpec.model_validate({"class": "pkg:Mw", "kwargs": {"cutoff": date(2026, 1, 1)}}) + + assert spec.kwargs == {"cutoff": "2026-01-01"} + json.dumps(ExtensionsConfig(middlewares=[spec]).to_file_dict()) + + +def test_kwargs_yaml_datetime_normalizes_to_iso_string(): + spec = ConfiguredMiddlewareSpec.model_validate({"class": "pkg:Mw", "kwargs": {"cutoff": datetime(2026, 1, 1, 12, 0, 0)}}) + + assert spec.kwargs == {"cutoff": "2026-01-01T12:00:00"} + + +def test_kwargs_reject_non_json_values(): + with pytest.raises(ValidationError, match="JSON types"): + ConfiguredMiddlewareSpec.model_validate({"class": "pkg:Mw", "kwargs": {"hook": object()}}) + + +def test_kwargs_reject_nan(): + with pytest.raises(ValidationError, match="JSON types"): + ConfiguredMiddlewareSpec.model_validate({"class": "pkg:Mw", "kwargs": {"n": float("nan")}}) + + +def test_to_file_dict_round_trips_kwargs_entries(): + config = ExtensionsConfig.model_validate( + { + "middlewares": [ + "pkg:Plain", + {"class": "pkg:WithArgs", "kwargs": {"max_tool_calls": 5}}, + ] + } + ) + + dumped = config.to_file_dict() + restored = ExtensionsConfig.model_validate(dumped) + + assert dumped["middlewares"][0] == "pkg:Plain" + assert dumped["middlewares"][1]["class"] == "pkg:WithArgs" + assert dumped["middlewares"][1]["kwargs"] == {"max_tool_calls": 5} + assert restored.middlewares[1].class_path == "pkg:WithArgs" + assert restored.middlewares[1].kwargs == {"max_tool_calls": 5} diff --git a/config.example.yaml b/config.example.yaml index 1e5d4b3fb..b96471573 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -43,9 +43,13 @@ logging: # Optional AgentMiddleware classes loaded into the lead and subagent runtime # middleware chains after built-in runtime middlewares, but before the # safety/clarification tail. Missing packages, invalid classes, and broken -# modules fail loudly at agent creation with an actionable import error. -# The same zero-argument class list applies to both lead and subagent runtimes; -# lead-only vs subagent-only configuration is not expressible yet. Treat these +# modules fail at agent creation with an actionable import error; constructor +# errors fail at agent creation with the underlying exception. +# The same list applies to both lead and subagent runtimes; lead-only vs +# subagent-only configuration is not expressible yet. Each entry is a class +# path or `{class, kwargs}` for constructor arguments. kwargs values must be +# JSON types (object, array, string, number, boolean, or null); YAML dates +# and timestamps are coerced to ISO strings so they match JSON. Treat these # files as trusted operator config because middleware classes execute code. # Uncomment this block to define middlewares in config.yaml. Leaving it commented # lets extensions_config.json remain the source of truth for this fixed-slot @@ -53,7 +57,9 @@ logging: # extensions: # middlewares: # - my_company.deerflow_middlewares:DomainGuardMiddleware -# - my_company.deerflow_middlewares:LatencyStampingMiddleware +# - class: my_company.deerflow_middlewares:LatencyStampingMiddleware +# kwargs: +# header: X-DeerFlow-Latency # ============================================================================ # Tracing / Observability (Monocle) diff --git a/frontend/src/content/en/harness/customization.mdx b/frontend/src/content/en/harness/customization.mdx index ac55e5f19..f15033a50 100644 --- a/frontend/src/content/en/harness/customization.mdx +++ b/frontend/src/content/en/harness/customization.mdx @@ -44,12 +44,15 @@ class AuditMiddleware(AgentMiddleware[AgentState]): Lifecycle hooks can return a dictionary of state updates, which LangChain merges into the agent state; return `None` when observing only. -For an operator-managed deployment, the class must have a zero-argument constructor and be importable by the Gateway process: +For an operator-managed deployment, the class must be importable by the Gateway process. A string entry uses the zero-argument constructor; `{class, kwargs}` passes constructor arguments. `kwargs` values must be JSON types (object, array, string, number, boolean, or null); YAML dates and timestamps are coerced to ISO strings so they match JSON: ```yaml extensions: middlewares: - my_company.deerflow_middlewares:AuditMiddleware + - class: my_company.deerflow_middlewares:AuditMiddleware + kwargs: + max_tool_calls: 5 ``` Configured middleware is loaded after the built-in middleware and optional loop/token guards. On the lead-agent pipeline, it runs before the terminal-response, model-length, safety, and clarification tail; subagents have no terminal-response, model-length, or clarification stage, so configured middleware is followed by the optional safety guard, `DurableContextMiddleware`, optional `SummarizationMiddleware`, then `SubagentDateContextMiddleware` and `SystemMessageCoalescingMiddleware`. Treat these class paths as trusted configuration because loading one executes Python code. Embedded callers can instead use `DeerFlowClient(middlewares=[AuditMiddleware()])`, which builds the full lead-agent chain and places middleware before its terminal-response, model-length, safety, and clarification tail. `create_deerflow_agent(extra_middleware=[AuditMiddleware()])` instead builds a smaller feature-based lead-agent chain; unanchored extras are placed immediately before `ClarificationMiddleware` (anchored extras follow their `@Next`/`@Prev` placement). Neither API forwards middleware to subagents. diff --git a/frontend/src/content/en/harness/middlewares.mdx b/frontend/src/content/en/harness/middlewares.mdx index 7f9d2a78a..d4310547f 100644 --- a/frontend/src/content/en/harness/middlewares.mdx +++ b/frontend/src/content/en/harness/middlewares.mdx @@ -257,12 +257,15 @@ class MyMiddleware(AgentMiddleware[AgentState]): Lifecycle hooks can return a dictionary of state updates, which LangChain merges into the agent state, or `None` when they only observe state. -For operator-managed deployments, register a zero-argument class by import path: +For operator-managed deployments, register a class by import path. A string uses the zero-argument constructor; `{class, kwargs}` passes constructor arguments. `kwargs` values must be JSON types (object, array, string, number, boolean, or null); YAML dates and timestamps are coerced to ISO strings so they match JSON: ```yaml extensions: middlewares: - my_company.deerflow_middlewares:MyMiddleware + - class: my_company.deerflow_middlewares:MyMiddleware + kwargs: + max_tool_calls: 5 ``` Configured middleware runs after the built-in middleware and optional loop/token guards. On the lead-agent pipeline, it runs before the terminal-response, model-length, safety, and clarification tail; subagents have no terminal-response, model-length, or clarification stage, so configured middleware is followed by the optional safety guard, `DurableContextMiddleware`, optional `SummarizationMiddleware`, then `SubagentDateContextMiddleware` and `SystemMessageCoalescingMiddleware`. Treat middleware paths as trusted configuration because loading one executes Python code. Embedded callers can instead use `DeerFlowClient(middlewares=[...])`, which builds the full lead-agent chain and places middleware before its terminal-response, model-length, safety, and clarification tail. `create_deerflow_agent(extra_middleware=[...])` instead builds a smaller feature-based lead-agent chain; unanchored extras are placed immediately before `ClarificationMiddleware` (anchored extras follow their `@Next`/`@Prev` placement). Neither API forwards middleware to subagents. diff --git a/frontend/src/content/zh/harness/customization.mdx b/frontend/src/content/zh/harness/customization.mdx index 75383e31d..93b77909a 100644 --- a/frontend/src/content/zh/harness/customization.mdx +++ b/frontend/src/content/zh/harness/customization.mdx @@ -43,12 +43,15 @@ class AuditMiddleware(AgentMiddleware[AgentState]): 生命周期钩子可以返回状态更新字典，LangChain 会将其合并到 Agent 状态中；仅观察时返回 `None`。 -对于运维配置的部署，该类必须提供零参数构造函数，并且 Gateway 进程必须能够导入它： +对于运维配置的部署，该类必须能被 Gateway 进程导入。字符串条目使用零参数构造；`{class, kwargs}` 会把 `kwargs` 传给构造函数。`kwargs` 的值必须是 JSON 类型（对象、数组、字符串、数字、布尔或 null）；YAML 日期和时间戳会转成 ISO 字符串，与 JSON 保持一致： ```yaml extensions: middlewares: - my_company.deerflow_middlewares:AuditMiddleware + - class: my_company.deerflow_middlewares:AuditMiddleware + kwargs: + max_tool_calls: 5 ``` 配置的中间件会在内置中间件及可选的循环/token 保护之后加载。对于主 Agent 链，它位于终态响应、模型长度、安全和澄清尾部之前；子 Agent 链没有终态响应、模型长度或澄清阶段，因此配置中间件之后会继续执行可选的安全防护、`DurableContextMiddleware`、可选的 `SummarizationMiddleware`，随后是 `SubagentDateContextMiddleware` 和 `SystemMessageCoalescingMiddleware`。中间件路径会执行 Python 代码，因此应视为可信配置。嵌入式调用方也可以使用 `DeerFlowClient(middlewares=[AuditMiddleware()])`；它构建完整的主 Agent 链，并将中间件放在其终态响应、模型长度、安全和澄清尾部之前。`create_deerflow_agent(extra_middleware=[AuditMiddleware()])` 则构建较小的按功能组装的主 Agent 链；未锚定的额外中间件会紧接在 `ClarificationMiddleware` 之前放置（锚定中间件遵循其 `@Next`/`@Prev` 位置）。两个 API 均不会将中间件转发给子 Agent。 diff --git a/frontend/src/content/zh/harness/middlewares.mdx b/frontend/src/content/zh/harness/middlewares.mdx index 25e6e18df..e99560a6d 100644 --- a/frontend/src/content/zh/harness/middlewares.mdx +++ b/frontend/src/content/zh/harness/middlewares.mdx @@ -240,12 +240,15 @@ class MyMiddleware(AgentMiddleware[AgentState]): 生命周期钩子可以返回状态更新字典，LangChain 会将其合并到 Agent 状态中；仅观察时返回 `None`。 -对于运维配置的部署，请通过导入路径注册零参数构造的类： +对于运维配置的部署，请通过导入路径注册类。字符串条目使用零参数构造；`{class, kwargs}` 会把 `kwargs` 传给构造函数。`kwargs` 的值必须是 JSON 类型（对象、数组、字符串、数字、布尔或 null）；YAML 日期和时间戳会转成 ISO 字符串，与 JSON 保持一致： ```yaml extensions: middlewares: - my_company.deerflow_middlewares:MyMiddleware + - class: my_company.deerflow_middlewares:MyMiddleware + kwargs: + max_tool_calls: 5 ``` 配置的中间件会在内置中间件及可选的循环/token 保护之后加载。对于主 Agent 链，它位于终态响应、模型长度、安全和澄清尾部之前；子 Agent 链没有终态响应、模型长度或澄清阶段，因此配置中间件之后会继续执行可选的安全防护、`DurableContextMiddleware`、可选的 `SummarizationMiddleware`，随后是 `SubagentDateContextMiddleware` 和 `SystemMessageCoalescingMiddleware`。中间件路径会执行 Python 代码，因此应视为可信配置。嵌入式调用方也可以使用 `DeerFlowClient(middlewares=[...])`；它构建完整的主 Agent 链，并将中间件放在其终态响应、模型长度、安全和澄清尾部之前。`create_deerflow_agent(extra_middleware=[...])` 则构建较小的按功能组装的主 Agent 链；未锚定的额外中间件会紧接在 `ClarificationMiddleware` 之前放置（锚定中间件遵循其 `@Next`/`@Prev` 位置）。两个 API 均不会将中间件转发给子 Agent。