mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-22 12:49:20 +00:00
1f792d0f4b
* feat(extensions): add middleware plugin foundation * fix(extensions): stop config resolution from masking extension loading `create_app()` resolved the configured plugin list inside the fail-open guard around `load_extensions()`. CI has no `config.yaml` (gitignored and never generated by the workflow), so `get_app_config()` raised `FileNotFoundError` there and was swallowed as an extension failure -- `load_extensions()` never ran at all, and the four `create_app()` tests in `test_extension_app_loading.py` passed locally but failed on every runner. Resolve the plugin list before the guard. Only an absent `config.yaml` is tolerated, mirroring `_resolve_trace_enabled_for_app_construction()`: `create_app()` runs at import time, and lifespan still performs strict config loading before serving. A `config.yaml` that exists but fails to parse or validate now propagates instead of being reported as an extension failure -- reporting it as the latter silently dropped a `required: true` extension rather than failing the boot. Make the tests config-independent with an autouse `stub_app_config` fixture, following the existing pattern in `test_gateway_lifespan_shutdown.py`, and cover both new branches of the config-resolution boundary. * fix(extensions): bind the run's extension snapshot through subagent delegation The lead-agent path resolves one immutable loaded-extension snapshot per run and binds it through task-store allocation and graph construction, but the subagent path re-read the process-wide singleton at execution time. In production both are the same object, yet a `set_loaded_extensions()` between the lead run's start and a subagent's execution (test teardown, a future hot-reload path) would let one run mix two extension generations — exactly what the documented invariant exists to prevent. The graph-build binding is a ContextVar scoped to synchronous construction, so it has already exited by the time a tool delegates; the snapshot has to travel through runtime context instead. The run worker publishes it under the host-internal `EXTENSION_SNAPSHOT_CONTEXT_KEY` (written after the caller merge, popped when the run has none, so a caller-supplied value is never authoritative), `task_tool` reads it back through the type-checking `resolve_run_extensions()`, and `SubagentExecutor` binds it at construction. Callers outside the Gateway run path — embedded `DeerFlowClient`, standalone LangGraph Server — install no snapshot and keep the existing `get_loaded_extensions()` fallback. * refactor(extensions): defer the ordering table by call, not by a lying tuple `CORE_ORDERING_CONSTRAINTS` was a `tuple` subclass that overrode only `__iter__` and resolved into a class-level `_resolved` side channel. A tuple cannot populate its own storage after construction, so the instance stayed the empty tuple it was built as: `len()` was 0, `bool()` was False, `in` was always False, indexing raised, slicing and `reversed()` came back empty, and it compared unequal to the plain tuples tests substitute for it — all while iteration yielded the real constraints. Only `assert_ordering` consumed it, and only by iterating, so the split went unnoticed. The sibling `_AnchorTable(dict)` uses the same idea soundly because dict is mutable: `self.update()` fills the real storage, making every inherited operation correct. That trick does not survive the port to an immutable type. Replace it with `core_ordering_constraints()`, matching how `stack.py` defers the same kind of table via `_anchors()`. The deferral is kept — it is about dependency direction, not just cycles: `extensions/` is the layer the middleware layer calls into, so a module-scope `agents.middlewares` import here points the dependency backwards and closes a cycle as soon as any middleware imports something under `extensions/` at module level. Resolution stays at `assert_ordering` time, which already runs inside the middleware builder. Tests pin both halves: the returned value is a plain tuple whose len/bool/ membership/indexing/reversal/equality agree with iteration, and a subprocess probe asserts importing `extensions.ordering` does not load the middleware layer while calling the function does.
108 lines
3.9 KiB
Python
108 lines
3.9 KiB
Python
"""Registration-phase registry and its immutable runtime product.
|
|
|
|
Extensions only ever see the write-only public ``ExtensionRegistry`` contract.
|
|
The concrete host type additionally owns attribution, rollback, and immutable
|
|
runtime projection.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from collections.abc import Iterator
|
|
from contextlib import contextmanager
|
|
from dataclasses import dataclass
|
|
from typing import Any
|
|
|
|
from deerflow_extension_api import ExtensionData, MiddlewareContributor
|
|
from deerflow_extension_api import ExtensionRegistry as ExtensionRegistryContract
|
|
|
|
_Entry = tuple[str, Any]
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class LoadedExtensions:
|
|
"""Immutable view consumed at runtime.
|
|
|
|
Every entry carries its source string so diagnostics, provenance and
|
|
ordering errors can name the extension responsible.
|
|
"""
|
|
|
|
app_store: ExtensionData
|
|
middleware_contributors: tuple[tuple[str, MiddlewareContributor], ...] = ()
|
|
|
|
# Precomputed attributes, not methods: hook sites read one attribute to
|
|
# short-circuit, so the zero-extension path constructs nothing.
|
|
has_middleware_contributors: bool = False
|
|
needs_task_store: bool = False
|
|
|
|
|
|
class ExtensionRegistry(ExtensionRegistryContract):
|
|
"""Mutable, registration-phase only.
|
|
|
|
Subclasses the public contract Protocol so the host implementation is
|
|
type-checked against what extensions annotate; the host-only machinery
|
|
below (attribution, discard, mark/rollback_to, build) stays out of the
|
|
contract on purpose.
|
|
"""
|
|
|
|
def __init__(self) -> None:
|
|
self._middlewares: list[_Entry] = []
|
|
self._current_source: str | None = None
|
|
|
|
@contextmanager
|
|
def attributed_to(self, source: str) -> Iterator[None]:
|
|
"""Attribute everything registered inside the block to ``source``."""
|
|
previous = self._current_source
|
|
self._current_source = source
|
|
try:
|
|
yield
|
|
finally:
|
|
self._current_source = previous
|
|
|
|
def _source(self) -> str:
|
|
if self._current_source is None:
|
|
raise RuntimeError("registration must happen inside ExtensionRegistry.attributed_to(...)")
|
|
return self._current_source
|
|
|
|
def middlewares(self, contributor: MiddlewareContributor) -> None:
|
|
self._middlewares.append((self._source(), contributor))
|
|
|
|
def discard(self, source: str) -> None:
|
|
"""Remove every entry registered by ``source``.
|
|
|
|
Called when install() raises partway through. A half-registered
|
|
extension is more dangerous than an absent one because the data it
|
|
produces looks complete.
|
|
|
|
Note: this matches by source string, so it is unsafe when two specs
|
|
share the same ``use`` with different config — it would remove a
|
|
different, successfully-installed instance's entries too. Callers
|
|
that process one install() at a time should prefer
|
|
``mark()``/``rollback_to()`` instead.
|
|
"""
|
|
self._middlewares[:] = [entry for entry in self._middlewares if entry[0] != source]
|
|
|
|
def mark(self) -> int:
|
|
"""Snapshot bucket lengths so one install() can be undone positionally."""
|
|
return len(self._middlewares)
|
|
|
|
def rollback_to(self, mark: int) -> None:
|
|
"""Undo every registration made since ``mark``.
|
|
|
|
Positional rather than source-keyed: two specs may legitimately share
|
|
a ``use`` string with different config, and deleting by source would
|
|
take the other instance's successful registrations with it.
|
|
"""
|
|
del self._middlewares[mark:]
|
|
|
|
def build(self) -> LoadedExtensions:
|
|
return LoadedExtensions(
|
|
app_store=ExtensionData("app"),
|
|
middleware_contributors=tuple(self._middlewares),
|
|
has_middleware_contributors=bool(self._middlewares),
|
|
needs_task_store=bool(self._middlewares),
|
|
)
|
|
|
|
|
|
#: Shared empty instance for hosts that load no extensions.
|
|
EMPTY_EXTENSIONS = ExtensionRegistry().build()
|