mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-14 16:08:41 +00:00
feat(extensions): allow constructor kwargs on config-declared middlewares (#5312)
* feat(extensions): allow constructor kwargs on config-declared middlewares
extensions.middlewares entries may be a class-path string or {class, kwargs}.
String entries keep the zero-argument constructor. Unknown fields and blank
class paths fail at config validation. Constructor errors still fail at
agent creation.
Fixes #5311
* fix(extensions): coerce middleware kwargs to JSON types
YAML timestamps became datetime objects while JSON kept strings, so
constructors and to_file_dict() json.dump saw different types. Validate
kwargs as JSON types at config load, stringify dates, reject NaN and
other non-JSON values, and cover the raw-dict loader branch.
* style(extensions): wrap middleware Field description for ruff E501
make lint failed: the middlewares description was 289 chars (limit 240).
Wrap it and run ruff format on the two files this PR last touched.
* docs: compact configured middleware guidance to satisfy size limit
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
parent
76d072c584
commit
4470932118
@ -1027,7 +1027,7 @@ as the heading; their content remains available to the agent.
|
||||
|
||||
Advanced deployments can enable pluggable authorization with `authorization.enabled` in `config.yaml`. A configured `AuthorizationProvider` filters denied tools before they reach the model or deferred-tool catalog, then the same provider is checked again before every business-tool execution through the existing guardrail middleware. Gateway `threads:*` and `runs:*` route permissions are derived from the same provider, while existing owner checks and admin-only management gates remain in force. Every HTTP route that starts or enables a future Agent run requires `runs:create`: this includes the stateless `POST /api/runs/stream` and `POST /api/runs/wait` endpoints plus scheduled-task create, update, resume, and manual-trigger mutations. Scheduled-task mutations retain their existing `threads:write` requirement, and the stateless routes separately enforce ownership when the optional thread ID is supplied in the request body. A generated `tool_search` may bypass the second tool check only when it fronts the current build's already-filtered deferred catalog. Model access follows the same provider: the Gateway `models` list is filtered per principal, `model:use` is enforced on model detail requests and again when the runtime resolves the agent's model, and a denied default model falls back to the first remaining candidate that also passes `model:use`. The built-in RBAC provider supports per-role `tools`, `routes`, `models`, `skills`, and `sandbox` allow/deny policies and validates that `default_role` names a configured role; authorization is disabled by default. See `config.example.yaml` and the [authorization RFC](docs/plans/2026-07-10-pluggable-authorization-rfc.md).
|
||||
|
||||
Advanced deployments can also extend the agent runtime itself by declaring zero-argument `AgentMiddleware` classes under `extensions.middlewares` in `config.yaml` or `extensions_config.json`. DeerFlow loads the same configured class list into the lead-agent and subagent pipelines after their built-in runtime middlewares and loop/token guards, but before the terminal-response/safety/clarification tail, so enterprise forks can add domain guardrails, tool-call governance, or observability hooks without patching the built-in middleware builders. Missing packages, invalid classes, and broken modules fail loudly at agent creation. Treat `config.yaml` and `extensions_config.json` as trusted operator-controlled files: middleware paths are code execution, just like custom tool, model, sandbox, guardrail, MCP server, and MCP interceptor declarations. Gateway skill/MCP toggle endpoints preserve this field but do not expose an API write path for `extensions.middlewares`. Per-context parameterization and separate lead-only/subagent-only middleware lists are not supported yet.
|
||||
Advanced deployments can also extend the agent runtime itself by declaring `AgentMiddleware` classes under `extensions.middlewares` in `config.yaml` or `extensions_config.json`. Each entry is a `module.path:ClassName` string (zero-argument constructor) or an object `{class, kwargs}` whose `kwargs` are passed to the constructor. `kwargs` values must be JSON types (object, array, string, number, boolean, or null); YAML dates and timestamps are coerced to ISO strings so they match JSON. DeerFlow loads the same configured list into the lead-agent and subagent pipelines after their built-in runtime middlewares and loop/token guards, but before the terminal-response/safety/clarification tail, so enterprise forks can add domain guardrails, tool-call governance, or observability hooks without patching the built-in middleware builders. Missing packages, invalid classes, broken modules, and constructor errors fail loudly at agent creation. Treat `config.yaml` and `extensions_config.json` as trusted operator-controlled files: middleware paths are code execution, just like custom tool, model, sandbox, guardrail, MCP server, and MCP interceptor declarations. Gateway skill/MCP toggle endpoints preserve this field but do not expose an API write path for `extensions.middlewares`. Separate lead-only/subagent-only middleware lists are not supported yet.
|
||||
|
||||
For packaged and configurable runtime integrations, use DeerFlow's extension manager.
|
||||
It accepts a Python package requirement, a public HTTPS Git URL, or a local directory, installs the
|
||||
|
||||
@ -299,12 +299,18 @@ class MyMiddleware(AgentMiddleware[AgentState]):
|
||||
Lifecycle hooks can return a dictionary of state updates, which LangChain merges
|
||||
into the agent state, or `None` when they only observe state.
|
||||
|
||||
2. Register the zero-argument middleware class in `config.yaml`:
|
||||
2. Register the middleware class in `config.yaml`. A class path uses the
|
||||
zero-argument constructor; `{class, kwargs}` passes constructor arguments.
|
||||
`kwargs` values must be JSON types (object, array, string, number, boolean,
|
||||
or null); YAML dates and timestamps are coerced to ISO strings so they match JSON:
|
||||
|
||||
```yaml
|
||||
extensions:
|
||||
middlewares:
|
||||
- deerflow.agents.middlewares.my_middleware:MyMiddleware
|
||||
- class: deerflow.agents.middlewares.my_middleware:MyMiddleware
|
||||
kwargs:
|
||||
max_tool_calls: 5
|
||||
```
|
||||
|
||||
Configured middleware runs after the built-in middleware and optional loop/token
|
||||
|
||||
@ -104,7 +104,7 @@ Before changing a later authorization phase, read the [authorization RFC](../../
|
||||
fallback identity, cleanup/LRU/reset, severity ordering, and test invariants.
|
||||
30. **TokenBudgetMiddleware** - *(optional, if `token_budget.enabled`)* Enforces per-run token limits
|
||||
31. **Custom middlewares** - *(optional)* Any `custom_middlewares` passed to `build_middlewares` are injected here, before config-declared extensions and the terminal-response/safety/clarification tail
|
||||
32. **Configured extension middlewares** - *(optional, if `extensions.middlewares` is set in `config.yaml` or `extensions_config.json`)* Zero-argument `AgentMiddleware` classes loaded from `module.path:ClassName` entries via `deerflow.reflection.resolve_class`. Missing packages, invalid classes, and broken modules fail loudly at agent creation. These run after built-ins/programmatic custom middleware and after the lead/subagent loop/token guards, but before the terminal-response/safety/clarification tail; subagents receive the same configured extension middleware class list before their safety tail. Treat these files as trusted operator config because middleware paths instantiate arbitrary code. Gateway skill/MCP toggle endpoints preserve this field through `to_file_dict()` but must not add a write path for `extensions.middlewares` without an explicit trust-boundary review. Lead-only vs subagent-only middleware lists and per-context constructor parameters are not expressible in this MVP.
|
||||
32. **Configured extension middlewares** - `extensions.middlewares` in `config.yaml` or `extensions_config.json` optionally accepts `module.path:ClassName` strings or `{class, kwargs}` objects. `deerflow.reflection.resolve_class` loads `AgentMiddleware` classes; import, class, and constructor errors fail agent creation. `kwargs` must be JSON-compatible; YAML dates/timestamps become ISO strings. Order: built-ins/custom and loop/token guards → extensions → terminal-response/safety/clarification tail. Subagents share the list before their safety tail; separate lead/subagent lists are unsupported. Trusted operator config only: paths instantiate arbitrary code. Gateway skill/MCP toggles preserve it via `to_file_dict()`; adding an API write path requires explicit trust-boundary review.
|
||||
33. **TerminalResponseMiddleware** - When a provider returns an empty terminal `AIMessage` after tool execution, injects a hidden recovery prompt and retries the model once; a second empty response is replaced in checkpoint state by a visible error fallback marked for the run worker, so the run finishes as an error instead of a silent success
|
||||
34. **ModelLengthFinishReasonMiddleware** - Records `stop_reason=model_length_capped` when provider-specific length detectors match a terminal `AIMessage` without tool-call intent (`finish_reason=length` / `MAX_TOKENS`, or `stop_reason=max_tokens`), preserving the original assistant content and never reparsing textual tool-call-like envelopes
|
||||
35. **SafetyFinishReasonMiddleware** - *(optional, if `safety_finish_reason.enabled`)* Suppresses tool execution when the provider safety-terminated the response (e.g. `finish_reason=content_filter`); registered after terminal-response/custom/configured middlewares so LangChain's reverse-order `after_model` dispatch runs it first
|
||||
|
||||
@ -1,10 +1,11 @@
|
||||
"""Config-declared agent middleware loading."""
|
||||
|
||||
import logging
|
||||
from typing import TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from langchain.agents.middleware import AgentMiddleware
|
||||
|
||||
from deerflow.config.extensions_config import ConfiguredMiddlewareSpec
|
||||
from deerflow.reflection import resolve_class
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@ -13,22 +14,34 @@ if TYPE_CHECKING:
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _middleware_constructor_args(entry: str | ConfiguredMiddlewareSpec | dict[str, Any]) -> tuple[str, dict[str, Any]]:
|
||||
"""Return ``(class_path, kwargs)`` for one config entry."""
|
||||
if isinstance(entry, str):
|
||||
return entry, {}
|
||||
if isinstance(entry, ConfiguredMiddlewareSpec):
|
||||
return entry.class_path, dict(entry.kwargs)
|
||||
parsed = ConfiguredMiddlewareSpec.model_validate(entry)
|
||||
return parsed.class_path, dict(parsed.kwargs)
|
||||
|
||||
|
||||
def load_configured_extension_middlewares(app_config: "AppConfig") -> list[AgentMiddleware]:
|
||||
"""Instantiate config-declared agent middlewares.
|
||||
|
||||
Each entry is a zero-argument ``AgentMiddleware`` class path in
|
||||
``module.path:ClassName`` format. Import, attribute, and subclass validation
|
||||
intentionally go through the shared reflection resolver so failures carry
|
||||
the same actionable dependency hints as models, tools, sandbox providers,
|
||||
and guardrail providers.
|
||||
Each entry is a ``module.path:ClassName`` string or a
|
||||
``ConfiguredMiddlewareSpec`` (``class`` plus optional ``kwargs``).
|
||||
Import, attribute, and subclass validation intentionally go through the
|
||||
shared reflection resolver so failures carry the same actionable
|
||||
dependency hints as models, tools, sandbox providers, and guardrail
|
||||
providers. Constructor errors fail loudly at agent creation.
|
||||
"""
|
||||
middlewares: list[AgentMiddleware] = []
|
||||
for middleware_path in list(app_config.extensions.middlewares or []):
|
||||
middleware_cls = resolve_class(middleware_path, AgentMiddleware)
|
||||
for entry in list(app_config.extensions.middlewares or []):
|
||||
class_path, kwargs = _middleware_constructor_args(entry)
|
||||
middleware_cls = resolve_class(class_path, AgentMiddleware)
|
||||
try:
|
||||
middleware = middleware_cls()
|
||||
middleware = middleware_cls(**kwargs)
|
||||
except Exception:
|
||||
logger.exception("Failed to instantiate configured extension middleware %s", middleware_path)
|
||||
logger.exception("Failed to instantiate configured extension middleware %s", class_path)
|
||||
raise
|
||||
middlewares.append(middleware)
|
||||
return middlewares
|
||||
|
||||
@ -66,6 +66,6 @@ Extensions are optional only in the fallback *search* mode (priority 3-4 above):
|
||||
- `mcpServers` - Map of server name → config (enabled, type, command, args, env, url, headers, oauth, description, `routing`, `tools`, `tool_call_timeout`, `session_init_timeout`). `routing.mode="prefer"` emits `<mcp_routing_hints>` prompt guidance; if `tool_search` defers the hinted tool, `McpRoutingMiddleware` can also auto-promote matching deferred schemas before the model call. It does not hard-disable other tools. `session_init_timeout` (default `DEFAULT_MCP_SESSION_INIT_TIMEOUT` = 60s, `null` to disable) bounds server bring-up: tool discovery and persistent stdio session initialization, so a hung server cannot block agent construction indefinitely; durable HTTP/SSE task calls use it for their ephemeral session initialization too. `tool_call_timeout` bounds individual stdio calls and durable-task calls on every transport; other HTTP/SSE tools use transport-level timeouts.
|
||||
- `tool_search.auto_promote_top_k` - Global MCP routing auto-promote breadth. Default `3`, clamped to `1..5`; applies only when `tool_search.enabled=true` and only to deferred MCP tools with `routing.mode="prefer"` and non-empty keywords. For lead agents the deferred catalog is built from the full configured MCP set; auto-promotion never grants authority because an active skill's runtime policy still filters model-visible schemas, `tool_search` results, and execution.
|
||||
- `skills` - Map of skill name → state (enabled)
|
||||
- `middlewares` - Zero-argument `AgentMiddleware` class paths for lead and subagent runtime extension. `config.yaml -> extensions` can override these fields after validation; overrides are replace-per-field, not list concatenation.
|
||||
- `middlewares` - `AgentMiddleware` entries for lead and subagent runtime extension: class-path strings or `{class, kwargs}` objects. `kwargs` values must be JSON types; YAML dates and timestamps are coerced to ISO strings so they match JSON. `config.yaml -> extensions` can override these fields after validation; overrides are replace-per-field, not list concatenation.
|
||||
|
||||
Gateway API endpoints and `DeerFlowClient` methods can modify MCP servers and skill state at runtime; their `extensions_config.json` writes use the shared atomic replacement helper, while `middlewares` remains an operator-controlled config-file extension point.
|
||||
|
||||
@ -3,12 +3,14 @@
|
||||
import errno
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import stat
|
||||
import tempfile
|
||||
import threading
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from datetime import date, datetime, time
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
@ -23,6 +25,7 @@ from deerflow.constants import (
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_JSON_KWARGS_ERROR = "middleware kwargs values must be JSON types (object, array, string, number, boolean, or null)"
|
||||
_non_atomic_fallback_targets: set[Path] = set()
|
||||
_non_atomic_fallback_targets_lock = threading.Lock()
|
||||
|
||||
@ -307,12 +310,76 @@ class SkillStateConfig(BaseModel):
|
||||
enabled: bool = Field(default=True, description="Whether this skill is enabled")
|
||||
|
||||
|
||||
def _coerce_json_kwargs_value(value: Any) -> Any:
|
||||
"""Keep JSON types; stringify YAML timestamps so they match JSON strings."""
|
||||
if value is None or isinstance(value, (str, bool, int)):
|
||||
return value
|
||||
if isinstance(value, float):
|
||||
if not math.isfinite(value):
|
||||
raise ValueError(_JSON_KWARGS_ERROR)
|
||||
return value
|
||||
if isinstance(value, datetime):
|
||||
return value.isoformat()
|
||||
if isinstance(value, date):
|
||||
return value.isoformat()
|
||||
if isinstance(value, time):
|
||||
return value.isoformat()
|
||||
if isinstance(value, dict):
|
||||
if not all(isinstance(key, str) and key.strip() for key in value):
|
||||
raise ValueError("middleware kwargs keys must be non-empty strings")
|
||||
return {key: _coerce_json_kwargs_value(item) for key, item in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [_coerce_json_kwargs_value(item) for item in value]
|
||||
raise ValueError(_JSON_KWARGS_ERROR)
|
||||
|
||||
|
||||
class ConfiguredMiddlewareSpec(BaseModel):
|
||||
"""One config-declared AgentMiddleware with optional constructor arguments."""
|
||||
|
||||
class_path: str = Field(
|
||||
...,
|
||||
alias="class",
|
||||
min_length=1,
|
||||
description="AgentMiddleware class path in 'module.path:ClassName' form.",
|
||||
)
|
||||
kwargs: dict[str, Any] = Field(
|
||||
default_factory=dict,
|
||||
description=("Keyword arguments passed to the middleware constructor. Values must be JSON types (object, array, string, number, boolean, or null); YAML dates and timestamps are coerced to ISO strings so they match JSON."),
|
||||
)
|
||||
model_config = ConfigDict(extra="forbid", populate_by_name=True)
|
||||
|
||||
@field_validator("class_path")
|
||||
@classmethod
|
||||
def _strip_class_path(cls, value: str) -> str:
|
||||
stripped = value.strip()
|
||||
if not stripped:
|
||||
raise ValueError("middleware class path must be a non-empty string")
|
||||
return stripped
|
||||
|
||||
@field_validator("kwargs", mode="before")
|
||||
@classmethod
|
||||
def _kwargs_none_is_empty(cls, value: Any) -> Any:
|
||||
return {} if value is None else value
|
||||
|
||||
@field_validator("kwargs")
|
||||
@classmethod
|
||||
def _kwargs_are_json_object(cls, value: dict[str, Any]) -> dict[str, Any]:
|
||||
coerced = _coerce_json_kwargs_value(value)
|
||||
json.dumps(coerced)
|
||||
return coerced
|
||||
|
||||
|
||||
class ExtensionsConfig(BaseModel):
|
||||
"""Unified configuration for MCP servers and skills."""
|
||||
|
||||
middlewares: list[str] = Field(
|
||||
middlewares: list[str | ConfiguredMiddlewareSpec] = Field(
|
||||
default_factory=list,
|
||||
description="AgentMiddleware class paths loaded into the lead-agent and subagent middleware chains. Each entry uses 'module.path:ClassName'.",
|
||||
description=(
|
||||
"AgentMiddleware entries loaded into the lead-agent and subagent middleware chains. "
|
||||
"Each entry is a 'module.path:ClassName' string or an object with 'class' and optional "
|
||||
"'kwargs'. kwargs values must be JSON types; YAML dates and timestamps are coerced to "
|
||||
"ISO strings."
|
||||
),
|
||||
)
|
||||
mcp_servers: dict[str, McpServerConfig] = Field(
|
||||
default_factory=dict,
|
||||
@ -325,6 +392,20 @@ class ExtensionsConfig(BaseModel):
|
||||
)
|
||||
model_config = ConfigDict(extra="allow", populate_by_name=True)
|
||||
|
||||
@field_validator("middlewares")
|
||||
@classmethod
|
||||
def _normalize_middleware_entries(cls, value: list[str | ConfiguredMiddlewareSpec]) -> list[str | ConfiguredMiddlewareSpec]:
|
||||
normalized: list[str | ConfiguredMiddlewareSpec] = []
|
||||
for entry in value:
|
||||
if isinstance(entry, str):
|
||||
stripped = entry.strip()
|
||||
if not stripped:
|
||||
raise ValueError("middleware class path must be a non-empty string")
|
||||
normalized.append(stripped)
|
||||
continue
|
||||
normalized.append(entry)
|
||||
return normalized
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_task_server_names_fit_storage(self) -> "ExtensionsConfig":
|
||||
for server_name, server in self.mcp_servers.items():
|
||||
|
||||
@ -278,6 +278,37 @@ def test_app_config_loads_extension_middlewares_from_extensions_config(tmp_path,
|
||||
assert config.extensions.middlewares == ["pkg.from_file:FileMiddleware"]
|
||||
|
||||
|
||||
def test_app_config_loads_middleware_kwargs_from_config_yaml(tmp_path, monkeypatch):
|
||||
from deerflow.config.extensions_config import ConfiguredMiddlewareSpec
|
||||
|
||||
config_path = tmp_path / "config.yaml"
|
||||
extensions_path = tmp_path / "extensions_config.json"
|
||||
extensions_path.write_text(
|
||||
json.dumps({"mcpServers": {}, "skills": {}, "middlewares": ["pkg.from_file:FileMiddleware"]}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
_write_config_with_sections(
|
||||
config_path,
|
||||
{
|
||||
"extensions": {
|
||||
"middlewares": [
|
||||
"pkg.from_yaml:PlainMiddleware",
|
||||
{"class": "pkg.from_yaml:KwargsMiddleware", "kwargs": {"max_tool_calls": 4}},
|
||||
],
|
||||
}
|
||||
},
|
||||
)
|
||||
monkeypatch.setenv("DEER_FLOW_EXTENSIONS_CONFIG_PATH", str(extensions_path))
|
||||
|
||||
config = AppConfig.from_file(str(config_path))
|
||||
|
||||
assert config.extensions.middlewares[0] == "pkg.from_yaml:PlainMiddleware"
|
||||
spec = config.extensions.middlewares[1]
|
||||
assert isinstance(spec, ConfiguredMiddlewareSpec)
|
||||
assert spec.class_path == "pkg.from_yaml:KwargsMiddleware"
|
||||
assert spec.kwargs == {"max_tool_calls": 4}
|
||||
|
||||
|
||||
def test_app_config_defaults_empty_database_to_sqlite(tmp_path, monkeypatch):
|
||||
config_path = tmp_path / "config.yaml"
|
||||
extensions_path = tmp_path / "extensions_config.json"
|
||||
|
||||
161
backend/tests/test_configured_extensions.py
Normal file
161
backend/tests/test_configured_extensions.py
Normal file
@ -0,0 +1,161 @@
|
||||
"""Config-declared extension middleware loading, including constructor kwargs."""
|
||||
|
||||
import json
|
||||
from datetime import date, datetime
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from langchain.agents.middleware import AgentMiddleware
|
||||
from pydantic import ValidationError
|
||||
|
||||
from deerflow.agents.middlewares.configured_extensions import load_configured_extension_middlewares
|
||||
from deerflow.config.extensions_config import ConfiguredMiddlewareSpec, ExtensionsConfig
|
||||
|
||||
|
||||
class RecordingMiddleware(AgentMiddleware):
|
||||
"""Test double that records constructor kwargs."""
|
||||
|
||||
def __init__(self, max_tool_calls: int = 10):
|
||||
super().__init__()
|
||||
self.max_tool_calls = max_tool_calls
|
||||
|
||||
|
||||
class ZeroArgMiddleware(AgentMiddleware):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
|
||||
|
||||
def _config(*entries) -> SimpleNamespace:
|
||||
return SimpleNamespace(extensions=SimpleNamespace(middlewares=list(entries)))
|
||||
|
||||
|
||||
def test_string_entry_still_zero_arg():
|
||||
loaded = load_configured_extension_middlewares(_config(f"{__name__}:ZeroArgMiddleware"))
|
||||
|
||||
assert len(loaded) == 1
|
||||
assert isinstance(loaded[0], ZeroArgMiddleware)
|
||||
|
||||
|
||||
def test_dict_entry_passes_constructor_kwargs():
|
||||
entry = ConfiguredMiddlewareSpec.model_validate({"class": f"{__name__}:RecordingMiddleware", "kwargs": {"max_tool_calls": 3}})
|
||||
|
||||
loaded = load_configured_extension_middlewares(_config(entry))
|
||||
|
||||
assert len(loaded) == 1
|
||||
assert isinstance(loaded[0], RecordingMiddleware)
|
||||
assert loaded[0].max_tool_calls == 3
|
||||
|
||||
|
||||
def test_raw_dict_entry_passes_constructor_kwargs():
|
||||
loaded = load_configured_extension_middlewares(_config({"class": f"{__name__}:RecordingMiddleware", "kwargs": {"max_tool_calls": 2}}))
|
||||
|
||||
assert len(loaded) == 1
|
||||
assert isinstance(loaded[0], RecordingMiddleware)
|
||||
assert loaded[0].max_tool_calls == 2
|
||||
|
||||
|
||||
def test_malformed_raw_dict_fails_at_load():
|
||||
with pytest.raises(ValidationError):
|
||||
load_configured_extension_middlewares(_config({"class": f"{__name__}:RecordingMiddleware", "apply_to": "lead"}))
|
||||
|
||||
|
||||
def test_empty_kwargs_matches_zero_arg_constructor():
|
||||
entry = ConfiguredMiddlewareSpec.model_validate({"class": f"{__name__}:RecordingMiddleware"})
|
||||
|
||||
loaded = load_configured_extension_middlewares(_config(entry))
|
||||
|
||||
assert loaded[0].max_tool_calls == 10
|
||||
|
||||
|
||||
def test_unknown_constructor_kwarg_fails_loudly():
|
||||
entry = ConfiguredMiddlewareSpec.model_validate({"class": f"{__name__}:ZeroArgMiddleware", "kwargs": {"not_a_param": 1}})
|
||||
|
||||
with pytest.raises(TypeError):
|
||||
load_configured_extension_middlewares(_config(entry))
|
||||
|
||||
|
||||
def test_extensions_config_keeps_string_entries():
|
||||
config = ExtensionsConfig.model_validate({"middlewares": ["pkg:Middleware"]})
|
||||
|
||||
assert config.middlewares == ["pkg:Middleware"]
|
||||
|
||||
|
||||
def test_extensions_config_parses_class_and_kwargs():
|
||||
config = ExtensionsConfig.model_validate(
|
||||
{
|
||||
"middlewares": [
|
||||
"pkg:Plain",
|
||||
{"class": "pkg:WithArgs", "kwargs": {"max_tool_calls": 5}},
|
||||
]
|
||||
}
|
||||
)
|
||||
|
||||
assert config.middlewares[0] == "pkg:Plain"
|
||||
spec = config.middlewares[1]
|
||||
assert isinstance(spec, ConfiguredMiddlewareSpec)
|
||||
assert spec.class_path == "pkg:WithArgs"
|
||||
assert spec.kwargs == {"max_tool_calls": 5}
|
||||
|
||||
|
||||
def test_extensions_config_rejects_unknown_entry_fields():
|
||||
with pytest.raises(ValidationError):
|
||||
ExtensionsConfig.model_validate({"middlewares": [{"class": "pkg:Middleware", "apply_to": "lead"}]})
|
||||
|
||||
|
||||
def test_extensions_config_rejects_blank_class_path():
|
||||
with pytest.raises(ValidationError):
|
||||
ExtensionsConfig.model_validate({"middlewares": [{"class": " "}]})
|
||||
|
||||
|
||||
def test_extensions_config_rejects_blank_string_entry():
|
||||
with pytest.raises(ValidationError):
|
||||
ExtensionsConfig.model_validate({"middlewares": [" "]})
|
||||
|
||||
|
||||
def test_extensions_config_strips_string_entries():
|
||||
config = ExtensionsConfig.model_validate({"middlewares": [" pkg:Plain "]})
|
||||
|
||||
assert config.middlewares == ["pkg:Plain"]
|
||||
|
||||
|
||||
def test_kwargs_yaml_date_normalizes_to_iso_string():
|
||||
spec = ConfiguredMiddlewareSpec.model_validate({"class": "pkg:Mw", "kwargs": {"cutoff": date(2026, 1, 1)}})
|
||||
|
||||
assert spec.kwargs == {"cutoff": "2026-01-01"}
|
||||
json.dumps(ExtensionsConfig(middlewares=[spec]).to_file_dict())
|
||||
|
||||
|
||||
def test_kwargs_yaml_datetime_normalizes_to_iso_string():
|
||||
spec = ConfiguredMiddlewareSpec.model_validate({"class": "pkg:Mw", "kwargs": {"cutoff": datetime(2026, 1, 1, 12, 0, 0)}})
|
||||
|
||||
assert spec.kwargs == {"cutoff": "2026-01-01T12:00:00"}
|
||||
|
||||
|
||||
def test_kwargs_reject_non_json_values():
|
||||
with pytest.raises(ValidationError, match="JSON types"):
|
||||
ConfiguredMiddlewareSpec.model_validate({"class": "pkg:Mw", "kwargs": {"hook": object()}})
|
||||
|
||||
|
||||
def test_kwargs_reject_nan():
|
||||
with pytest.raises(ValidationError, match="JSON types"):
|
||||
ConfiguredMiddlewareSpec.model_validate({"class": "pkg:Mw", "kwargs": {"n": float("nan")}})
|
||||
|
||||
|
||||
def test_to_file_dict_round_trips_kwargs_entries():
|
||||
config = ExtensionsConfig.model_validate(
|
||||
{
|
||||
"middlewares": [
|
||||
"pkg:Plain",
|
||||
{"class": "pkg:WithArgs", "kwargs": {"max_tool_calls": 5}},
|
||||
]
|
||||
}
|
||||
)
|
||||
|
||||
dumped = config.to_file_dict()
|
||||
restored = ExtensionsConfig.model_validate(dumped)
|
||||
|
||||
assert dumped["middlewares"][0] == "pkg:Plain"
|
||||
assert dumped["middlewares"][1]["class"] == "pkg:WithArgs"
|
||||
assert dumped["middlewares"][1]["kwargs"] == {"max_tool_calls": 5}
|
||||
assert restored.middlewares[1].class_path == "pkg:WithArgs"
|
||||
assert restored.middlewares[1].kwargs == {"max_tool_calls": 5}
|
||||
@ -43,9 +43,13 @@ logging:
|
||||
# Optional AgentMiddleware classes loaded into the lead and subagent runtime
|
||||
# middleware chains after built-in runtime middlewares, but before the
|
||||
# safety/clarification tail. Missing packages, invalid classes, and broken
|
||||
# modules fail loudly at agent creation with an actionable import error.
|
||||
# The same zero-argument class list applies to both lead and subagent runtimes;
|
||||
# lead-only vs subagent-only configuration is not expressible yet. Treat these
|
||||
# modules fail at agent creation with an actionable import error; constructor
|
||||
# errors fail at agent creation with the underlying exception.
|
||||
# The same list applies to both lead and subagent runtimes; lead-only vs
|
||||
# subagent-only configuration is not expressible yet. Each entry is a class
|
||||
# path or `{class, kwargs}` for constructor arguments. kwargs values must be
|
||||
# JSON types (object, array, string, number, boolean, or null); YAML dates
|
||||
# and timestamps are coerced to ISO strings so they match JSON. Treat these
|
||||
# files as trusted operator config because middleware classes execute code.
|
||||
# Uncomment this block to define middlewares in config.yaml. Leaving it commented
|
||||
# lets extensions_config.json remain the source of truth for this fixed-slot
|
||||
@ -53,7 +57,9 @@ logging:
|
||||
# extensions:
|
||||
# middlewares:
|
||||
# - my_company.deerflow_middlewares:DomainGuardMiddleware
|
||||
# - my_company.deerflow_middlewares:LatencyStampingMiddleware
|
||||
# - class: my_company.deerflow_middlewares:LatencyStampingMiddleware
|
||||
# kwargs:
|
||||
# header: X-DeerFlow-Latency
|
||||
|
||||
# ============================================================================
|
||||
# Tracing / Observability (Monocle)
|
||||
|
||||
@ -44,12 +44,15 @@ class AuditMiddleware(AgentMiddleware[AgentState]):
|
||||
Lifecycle hooks can return a dictionary of state updates, which LangChain merges
|
||||
into the agent state; return `None` when observing only.
|
||||
|
||||
For an operator-managed deployment, the class must have a zero-argument constructor and be importable by the Gateway process:
|
||||
For an operator-managed deployment, the class must be importable by the Gateway process. A string entry uses the zero-argument constructor; `{class, kwargs}` passes constructor arguments. `kwargs` values must be JSON types (object, array, string, number, boolean, or null); YAML dates and timestamps are coerced to ISO strings so they match JSON:
|
||||
|
||||
```yaml
|
||||
extensions:
|
||||
middlewares:
|
||||
- my_company.deerflow_middlewares:AuditMiddleware
|
||||
- class: my_company.deerflow_middlewares:AuditMiddleware
|
||||
kwargs:
|
||||
max_tool_calls: 5
|
||||
```
|
||||
|
||||
Configured middleware is loaded after the built-in middleware and optional loop/token guards. On the lead-agent pipeline, it runs before the terminal-response, model-length, safety, and clarification tail; subagents have no terminal-response, model-length, or clarification stage, so configured middleware is followed by the optional safety guard, `DurableContextMiddleware`, optional `SummarizationMiddleware`, then `SubagentDateContextMiddleware` and `SystemMessageCoalescingMiddleware`. Treat these class paths as trusted configuration because loading one executes Python code. Embedded callers can instead use `DeerFlowClient(middlewares=[AuditMiddleware()])`, which builds the full lead-agent chain and places middleware before its terminal-response, model-length, safety, and clarification tail. `create_deerflow_agent(extra_middleware=[AuditMiddleware()])` instead builds a smaller feature-based lead-agent chain; unanchored extras are placed immediately before `ClarificationMiddleware` (anchored extras follow their `@Next`/`@Prev` placement). Neither API forwards middleware to subagents.
|
||||
|
||||
@ -257,12 +257,15 @@ class MyMiddleware(AgentMiddleware[AgentState]):
|
||||
Lifecycle hooks can return a dictionary of state updates, which LangChain merges
|
||||
into the agent state, or `None` when they only observe state.
|
||||
|
||||
For operator-managed deployments, register a zero-argument class by import path:
|
||||
For operator-managed deployments, register a class by import path. A string uses the zero-argument constructor; `{class, kwargs}` passes constructor arguments. `kwargs` values must be JSON types (object, array, string, number, boolean, or null); YAML dates and timestamps are coerced to ISO strings so they match JSON:
|
||||
|
||||
```yaml
|
||||
extensions:
|
||||
middlewares:
|
||||
- my_company.deerflow_middlewares:MyMiddleware
|
||||
- class: my_company.deerflow_middlewares:MyMiddleware
|
||||
kwargs:
|
||||
max_tool_calls: 5
|
||||
```
|
||||
|
||||
Configured middleware runs after the built-in middleware and optional loop/token guards. On the lead-agent pipeline, it runs before the terminal-response, model-length, safety, and clarification tail; subagents have no terminal-response, model-length, or clarification stage, so configured middleware is followed by the optional safety guard, `DurableContextMiddleware`, optional `SummarizationMiddleware`, then `SubagentDateContextMiddleware` and `SystemMessageCoalescingMiddleware`. Treat middleware paths as trusted configuration because loading one executes Python code. Embedded callers can instead use `DeerFlowClient(middlewares=[...])`, which builds the full lead-agent chain and places middleware before its terminal-response, model-length, safety, and clarification tail. `create_deerflow_agent(extra_middleware=[...])` instead builds a smaller feature-based lead-agent chain; unanchored extras are placed immediately before `ClarificationMiddleware` (anchored extras follow their `@Next`/`@Prev` placement). Neither API forwards middleware to subagents.
|
||||
|
||||
@ -43,12 +43,15 @@ class AuditMiddleware(AgentMiddleware[AgentState]):
|
||||
|
||||
生命周期钩子可以返回状态更新字典,LangChain 会将其合并到 Agent 状态中;仅观察时返回 `None`。
|
||||
|
||||
对于运维配置的部署,该类必须提供零参数构造函数,并且 Gateway 进程必须能够导入它:
|
||||
对于运维配置的部署,该类必须能被 Gateway 进程导入。字符串条目使用零参数构造;`{class, kwargs}` 会把 `kwargs` 传给构造函数。`kwargs` 的值必须是 JSON 类型(对象、数组、字符串、数字、布尔或 null);YAML 日期和时间戳会转成 ISO 字符串,与 JSON 保持一致:
|
||||
|
||||
```yaml
|
||||
extensions:
|
||||
middlewares:
|
||||
- my_company.deerflow_middlewares:AuditMiddleware
|
||||
- class: my_company.deerflow_middlewares:AuditMiddleware
|
||||
kwargs:
|
||||
max_tool_calls: 5
|
||||
```
|
||||
|
||||
配置的中间件会在内置中间件及可选的循环/token 保护之后加载。对于主 Agent 链,它位于终态响应、模型长度、安全和澄清尾部之前;子 Agent 链没有终态响应、模型长度或澄清阶段,因此配置中间件之后会继续执行可选的安全防护、`DurableContextMiddleware`、可选的 `SummarizationMiddleware`,随后是 `SubagentDateContextMiddleware` 和 `SystemMessageCoalescingMiddleware`。中间件路径会执行 Python 代码,因此应视为可信配置。嵌入式调用方也可以使用 `DeerFlowClient(middlewares=[AuditMiddleware()])`;它构建完整的主 Agent 链,并将中间件放在其终态响应、模型长度、安全和澄清尾部之前。`create_deerflow_agent(extra_middleware=[AuditMiddleware()])` 则构建较小的按功能组装的主 Agent 链;未锚定的额外中间件会紧接在 `ClarificationMiddleware` 之前放置(锚定中间件遵循其 `@Next`/`@Prev` 位置)。两个 API 均不会将中间件转发给子 Agent。
|
||||
|
||||
@ -240,12 +240,15 @@ class MyMiddleware(AgentMiddleware[AgentState]):
|
||||
|
||||
生命周期钩子可以返回状态更新字典,LangChain 会将其合并到 Agent 状态中;仅观察时返回 `None`。
|
||||
|
||||
对于运维配置的部署,请通过导入路径注册零参数构造的类:
|
||||
对于运维配置的部署,请通过导入路径注册类。字符串条目使用零参数构造;`{class, kwargs}` 会把 `kwargs` 传给构造函数。`kwargs` 的值必须是 JSON 类型(对象、数组、字符串、数字、布尔或 null);YAML 日期和时间戳会转成 ISO 字符串,与 JSON 保持一致:
|
||||
|
||||
```yaml
|
||||
extensions:
|
||||
middlewares:
|
||||
- my_company.deerflow_middlewares:MyMiddleware
|
||||
- class: my_company.deerflow_middlewares:MyMiddleware
|
||||
kwargs:
|
||||
max_tool_calls: 5
|
||||
```
|
||||
|
||||
配置的中间件会在内置中间件及可选的循环/token 保护之后加载。对于主 Agent 链,它位于终态响应、模型长度、安全和澄清尾部之前;子 Agent 链没有终态响应、模型长度或澄清阶段,因此配置中间件之后会继续执行可选的安全防护、`DurableContextMiddleware`、可选的 `SummarizationMiddleware`,随后是 `SubagentDateContextMiddleware` 和 `SystemMessageCoalescingMiddleware`。中间件路径会执行 Python 代码,因此应视为可信配置。嵌入式调用方也可以使用 `DeerFlowClient(middlewares=[...])`;它构建完整的主 Agent 链,并将中间件放在其终态响应、模型长度、安全和澄清尾部之前。`create_deerflow_agent(extra_middleware=[...])` 则构建较小的按功能组装的主 Agent 链;未锚定的额外中间件会紧接在 `ClarificationMiddleware` 之前放置(锚定中间件遵循其 `@Next`/`@Prev` 位置)。两个 API 均不会将中间件转发给子 Agent。
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user