deer-flow/backend/tests/test_extension_api_contracts.py
Nan Gao 1f792d0f4b
feat(extensions): add middleware plugin foundation (#4636)
* feat(extensions): add middleware plugin foundation

* fix(extensions): stop config resolution from masking extension loading

`create_app()` resolved the configured plugin list inside the fail-open
guard around `load_extensions()`. CI has no `config.yaml` (gitignored and
never generated by the workflow), so `get_app_config()` raised
`FileNotFoundError` there and was swallowed as an extension failure --
`load_extensions()` never ran at all, and the four `create_app()` tests in
`test_extension_app_loading.py` passed locally but failed on every runner.

Resolve the plugin list before the guard. Only an absent `config.yaml` is
tolerated, mirroring `_resolve_trace_enabled_for_app_construction()`:
`create_app()` runs at import time, and lifespan still performs strict
config loading before serving. A `config.yaml` that exists but fails to
parse or validate now propagates instead of being reported as an extension
failure -- reporting it as the latter silently dropped a `required: true`
extension rather than failing the boot.

Make the tests config-independent with an autouse `stub_app_config`
fixture, following the existing pattern in `test_gateway_lifespan_shutdown.py`,
and cover both new branches of the config-resolution boundary.

* fix(extensions): bind the run's extension snapshot through subagent delegation

The lead-agent path resolves one immutable loaded-extension snapshot per run
and binds it through task-store allocation and graph construction, but the
subagent path re-read the process-wide singleton at execution time. In
production both are the same object, yet a `set_loaded_extensions()` between
the lead run's start and a subagent's execution (test teardown, a future
hot-reload path) would let one run mix two extension generations — exactly what
the documented invariant exists to prevent.

The graph-build binding is a ContextVar scoped to synchronous construction, so
it has already exited by the time a tool delegates; the snapshot has to travel
through runtime context instead. The run worker publishes it under the
host-internal `EXTENSION_SNAPSHOT_CONTEXT_KEY` (written after the caller merge,
popped when the run has none, so a caller-supplied value is never
authoritative), `task_tool` reads it back through the type-checking
`resolve_run_extensions()`, and `SubagentExecutor` binds it at construction.

Callers outside the Gateway run path — embedded `DeerFlowClient`, standalone
LangGraph Server — install no snapshot and keep the existing
`get_loaded_extensions()` fallback.

* refactor(extensions): defer the ordering table by call, not by a lying tuple

`CORE_ORDERING_CONSTRAINTS` was a `tuple` subclass that overrode only
`__iter__` and resolved into a class-level `_resolved` side channel. A tuple
cannot populate its own storage after construction, so the instance stayed the
empty tuple it was built as: `len()` was 0, `bool()` was False, `in` was always
False, indexing raised, slicing and `reversed()` came back empty, and it
compared unequal to the plain tuples tests substitute for it — all while
iteration yielded the real constraints. Only `assert_ordering` consumed it, and
only by iterating, so the split went unnoticed.

The sibling `_AnchorTable(dict)` uses the same idea soundly because dict is
mutable: `self.update()` fills the real storage, making every inherited
operation correct. That trick does not survive the port to an immutable type.

Replace it with `core_ordering_constraints()`, matching how `stack.py` defers
the same kind of table via `_anchors()`. The deferral is kept — it is about
dependency direction, not just cycles: `extensions/` is the layer the
middleware layer calls into, so a module-scope `agents.middlewares` import here
points the dependency backwards and closes a cycle as soon as any middleware
imports something under `extensions/` at module level. Resolution stays at
`assert_ordering` time, which already runs inside the middleware builder.

Tests pin both halves: the returned value is a plain tuple whose len/bool/
membership/indexing/reversal/equality agree with iteration, and a subprocess
probe asserts importing `extensions.ordering` does not load the middleware
layer while calling the function does.
2026-08-04 22:33:26 +08:00

215 lines
7.5 KiB
Python

"""Tests for the extension contract surface.
The contracts carry two compatibility promises that are easy to break by
accident and impossible to catch at runtime later: every Protocol method has a
default implementation, and every optional dataclass field has a default. Both
are asserted here.
"""
from __future__ import annotations
import dataclasses
import importlib.resources
import inspect
import pytest
from deerflow_extension_api import (
API_VERSION,
AgentBuildContext,
AgentScope,
ExtensionData,
ExtensionInstall,
ExtensionRegistry,
HostPolicySnapshot,
MiddlewareContributor,
MiddlewarePlacement,
Placement,
extension,
)
from deerflow_extension_api.runtime_bridge import (
EXTENSION_TASK_STORE_KEY,
task_store_from_runtime,
)
def test_placement_members_cover_both_axes():
assert Placement.MODEL_LOGICAL.value == "model_logical"
assert Placement.MODEL_PHYSICAL.value == "model_physical"
assert Placement.TOOL_VISIBLE.value == "tool_visible"
assert Placement.TOOL_RAW.value == "tool_raw"
assert Placement.STANDARD.value == "standard"
def test_agent_scope_both_is_union():
assert AgentScope.BOTH == AgentScope.LEAD | AgentScope.SUBAGENT
assert AgentScope.LEAD in AgentScope.BOTH
def test_middleware_placement_defaults():
p = MiddlewarePlacement(middleware=object(), placement=Placement.STANDARD)
assert p.scope is AgentScope.BOTH
assert p.order == 0
@pytest.mark.parametrize(
"cls",
[
HostPolicySnapshot,
AgentBuildContext,
MiddlewarePlacement,
],
)
def test_every_dataclass_is_frozen(cls):
assert dataclasses.is_dataclass(cls)
assert cls.__dataclass_params__.frozen, f"{cls.__name__} must be frozen"
@pytest.mark.parametrize(
"cls",
[HostPolicySnapshot],
)
def test_additive_dataclasses_are_constructible_with_required_fields_only(cls):
"""Fields added later must carry defaults, or old extensions break on upgrade.
HostPolicySnapshot is host-constructed and fully optional.
AgentBuildContext gets its own dedicated test below because its scope is
legitimately required.
"""
assert cls() is not None
def test_agent_build_context_optional_fields_keep_their_defaults():
"""AgentBuildContext has one required field (scope); the rest must default.
Unlike the fully-optional dataclasses above, scope is legitimately
required, so this is not folded into the parametrized test above — it
would misrepresent the required/optional split this suite is meant to
document.
"""
ctx = AgentBuildContext(scope=AgentScope.LEAD)
assert ctx.agent_name is None
assert ctx.model_name is None
assert isinstance(ctx.policy, HostPolicySnapshot)
@pytest.mark.parametrize(
"protocol",
[
ExtensionRegistry,
MiddlewareContributor,
],
)
def test_every_protocol_method_has_a_default_implementation(protocol):
"""Adding a method to a Protocol is only additive when it has a default.
Without this, shipping a new contract method breaks every already-released
extension that does not implement it.
"""
checked = 0
for name, member in vars(protocol).items():
if name.startswith("_") or not inspect.isfunction(member):
continue
checked += 1
body = inspect.getsource(member).split("\n", 1)[1]
assert "return" in body, f"{protocol.__name__}.{name} has no default implementation. Adding a contract method is only additive when it returns a default; otherwise every already-released extension breaks on upgrade."
assert checked > 0, f"{protocol.__name__} declared no methods to check"
def test_contributor_defaults_return_empty():
class _Bare:
pass
bare = _Bare()
assert MiddlewareContributor.contribute_middlewares(bare, ExtensionData("app"), AgentBuildContext(scope=AgentScope.LEAD)) == ()
def test_future_contribution_points_are_not_advertised_before_the_host_supports_them():
"""A merged slice must not silently accept registrations it cannot run."""
import deerflow_extension_api
for name in (
"ExtensionRuntimeDeps",
"ExtensionService",
"SystemModelCallObserver",
"TaskLifecycleContributor",
):
assert name not in deerflow_extension_api.__all__
assert not hasattr(deerflow_extension_api, name)
def test_task_store_from_runtime_reads_the_host_key():
class _Runtime:
def __init__(self, context):
self.context = context
store = ExtensionData("task-1")
assert task_store_from_runtime(_Runtime({EXTENSION_TASK_STORE_KEY: store})) is store
def test_task_store_from_runtime_returns_none_on_missing_or_wrong_shape():
class _Runtime:
def __init__(self, context):
self.context = context
assert task_store_from_runtime(None) is None
assert task_store_from_runtime(_Runtime({})) is None
assert task_store_from_runtime(_Runtime("not-a-mapping")) is None
assert task_store_from_runtime(_Runtime({EXTENSION_TASK_STORE_KEY: "wrong type"})) is None
def test_extension_decorator_stamps_api_requirement():
@extension(api="0.1", name="demo")
def install(registry, config):
return None
assert install.__deerflow_api__ == "0.1"
assert install.__deerflow_name__ == "demo"
def test_registry_and_install_alias_are_part_of_the_public_surface():
"""Independent extensions annotate install(registry, config) against the
contract package alone — importing the host's concrete registry would pin
them to the harness release cadence and advertise host-only machinery."""
import typing
import deerflow_extension_api
assert "ExtensionRegistry" in deerflow_extension_api.__all__
assert "ExtensionInstall" in deerflow_extension_api.__all__
parameters, return_type = typing.get_args(ExtensionInstall)
assert parameters[0] is ExtensionRegistry, "install()'s first argument must be the public registry contract"
def test_distribution_marks_the_contract_package_as_typed():
marker = importlib.resources.files("deerflow_extension_api").joinpath("py.typed")
assert marker.is_file()
def test_harness_pins_the_contract_package_exactly():
"""The version contract (extension-system design): the host pins the
contract package exactly, extensions use ranges. A range here would let an
older harness resolve a newer 1.x contract package — API_VERSION would
then come from the upgraded package and newer extensions would look
supported against a host whose registry/placements/hook pipeline still
implements the older contract. The pin makes pip reject that skew at
install time."""
import tomllib
from importlib.metadata import version
from pathlib import Path
from packaging.requirements import Requirement
pyproject = Path(__file__).parent.parent / "packages" / "harness" / "pyproject.toml"
dependencies = tomllib.loads(pyproject.read_text())["project"]["dependencies"]
requirement = next(Requirement(dep) for dep in dependencies if Requirement(dep).name == "deerflow-extension-api")
expected = f"=={version('deerflow-extension-api')}"
assert str(requirement.specifier) == expected, f"the host must pin deerflow-extension-api exactly ({expected}); a range lets pip resolve a contract newer than the host implements"
def test_runtime_api_version_matches_the_installed_contract_package():
"""Every additive contract slice bumps both gates together."""
from importlib.metadata import version
assert API_VERSION == version("deerflow-extension-api")