diff --git a/AGENTS.md b/AGENTS.md index 8533c8336..d9b3b9260 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,6 +57,7 @@ deer-flow/ ├── extensions_config.example.json # Template → copy to extensions_config.json (gitignored): MCP servers + skills ├── backend/ # Python backend — see backend/AGENTS.md │ ├── Makefile # Per-module backend commands (dev, gateway, test, lint, migrate-rev) +│ ├── packages/extension-api/ # deerflow-extension-api package (import: deerflow_extension_api.*) — public extension contract │ ├── packages/harness/ # deerflow-harness package (import: deerflow.*) — agent framework │ └── app/ # FastAPI Gateway + IM channels (import: app.*) ├── frontend/ # Next.js frontend (pnpm) — see frontend/AGENTS.md @@ -70,6 +71,11 @@ deer-flow/ └── docs/ # Cross-cutting docs, plans, and design notes ``` +Third-party extensions are loaded from a top-level `plugins:` list in `config.yaml` +(operator-controlled on purpose — that list causes code to be imported, so it is deliberately +kept out of the API-writable `extensions_config.json`). See the Extension System section in +[backend/AGENTS.md](backend/AGENTS.md). + Runtime config lives at the **repo root**: copy `config.example.yaml` → `config.yaml` (main app config) and `extensions_config.example.json` → `extensions_config.json` (MCP servers + skills). Both real files are gitignored and may be edited at runtime via the diff --git a/README.md b/README.md index 50379d2ff..927389a9e 100644 --- a/README.md +++ b/README.md @@ -810,6 +810,18 @@ Advanced deployments can enable pluggable authorization with `authorization.enab Advanced deployments can also extend the agent runtime itself by declaring zero-argument `AgentMiddleware` classes under `extensions.middlewares` in `config.yaml` or `extensions_config.json`. DeerFlow loads the same configured class list into the lead-agent and subagent pipelines after their built-in runtime middlewares and loop/token guards, but before the terminal-response/safety/clarification tail, so enterprise forks can add domain guardrails, tool-call governance, or observability hooks without patching the built-in middleware builders. Missing packages, invalid classes, and broken modules fail loudly at agent creation. Treat `config.yaml` and `extensions_config.json` as trusted operator-controlled files: middleware paths are code execution, just like custom tool, model, sandbox, guardrail, MCP server, and MCP interceptor declarations. Gateway skill/MCP toggle endpoints preserve this field but do not expose an API write path for `extensions.middlewares`. Per-context parameterization and separate lead-only/subagent-only middleware lists are not supported yet. +For packaged and configurable middleware integrations, use the top-level `plugins:` list +in `config.yaml`. A plugin exposes `module.path:install`, depends only on the standalone +`deerflow-extension-api` contract package, and can contribute isolated middleware to +semantic lead/subagent model or tool positions without patching DeerFlow's builders. +Plugin order is deterministic, per-plugin configuration is passed to `install()`, and +`required: true` makes load failure abort startup; otherwise failures are reported and +skipped. Plugins load once when the Gateway app is constructed, so changes require a +restart. Because this imports Python code, `plugins:` is intentionally unavailable through +the API-writable `extensions_config.json`. In Docker deployments, install the plugin in the +Gateway image rather than only in the host environment. See `config.example.yaml` for +configuration. + Gateway-generated follow-up suggestions now normalize both plain-string model output and block/list-style rich content before parsing the JSON array response, so provider-specific content wrappers do not silently drop suggestions. The Web UI composer can polish draft input before sending. The rewrite runs as a short Gateway LLM request using the `input_polish` model configuration, keeps slash skill prefixes such as `/data-analysis`, and only replaces the local draft after the user clicks the polish button; it does not create a thread run or persist a message. diff --git a/backend/AGENTS.md b/backend/AGENTS.md index e58a6c984..03ecc3812 100644 --- a/backend/AGENTS.md +++ b/backend/AGENTS.md @@ -29,6 +29,7 @@ deer-flow/ │ ├── Makefile # Backend-only commands (dev, gateway, lint) │ ├── langgraph.json # LangGraph Studio graph configuration │ ├── packages/ +│ │ ├── extension-api/ # public, host-independent extension contracts (import: deerflow_extension_api.*) │ │ └── harness/ # deerflow-harness package (import: deerflow.*) │ │ ├── pyproject.toml │ │ └── deerflow/ @@ -49,6 +50,7 @@ deer-flow/ │ │ ├── tools/builtins/ # Built-in tools (present_files, ask_clarification, view_image, review_skill_package) │ │ ├── mcp/ # MCP integration (tools, cache, client) │ │ ├── integrations/ # Managed first-party integration installers (e.g. Lark CLI skill pack) +│ │ ├── extensions/ # Python plugin loader, registry, placement, and isolation │ │ ├── models/ # Model factory with thinking/vision support │ │ ├── skills/ # Skills discovery, loading, parsing │ │ ├── config/ # Configuration system (app, model, sandbox, tool, etc.) @@ -435,6 +437,59 @@ Before changing a later authorization phase, read the [authorization RFC](../doc 34. **SafetyFinishReasonMiddleware** - *(optional, if `safety_finish_reason.enabled`)* Suppresses tool execution when the provider safety-terminated the response (e.g. `finish_reason=content_filter`); registered after terminal-response/custom/configured middlewares so LangChain's reverse-order `after_model` dispatch runs it first 35. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, writes a readable `ToolMessage.content` fallback plus structured `ToolMessage.artifact.human_input` request payload, and interrupts via `Command(goto=END)` (must be last). Payloads are versioned: legacy modes (`free_text` / `choice_with_other`) keep `version: 1` unchanged, while the v2 `form` mode (from `fields`) carries `version: 2` so older frontends reject the payload and degrade to the plain-text fallback. Field normalization is deterministic and lives in the middleware, not the tool schema — the middleware short-circuits before tool execution, so tool-arg typing alone provides no runtime validation. Validation is atomic: any structurally broken entry (non-dict, bad/duplicate name, a name colliding with a JS `Object.prototype` member like `__proto__`/`constructor`, exceeding the caps of 16 fields / 24 options per field / 200 chars per text, or the whole normalized definition exceeding `MAX_FORM_SERIALIZED_BYTES` = 16KB UTF-8 — the per-item caps alone admit forms whose IM text fallback would blow channel delivery limits and truncate away trailing fields) degrades the whole form to the legacy option/free-text modes, so a card can never render "complete" while silently missing a business field; benign issues keep local degradation (unknown types — including unhashable JSON like `type: []`, which must never raise from the membership probe — and option-less selects become `text`), and options are trimmed/deduped with blanks dropped (both form-level and top-level) because the frontend parser rejects blank option labels. Model-produced XML-to-dict option payloads are recursively flattened from dict/list containers in source order, scalar string/number leaves are retained, and residual XML tags are removed before the same trimming and deduplication. Checkbox fields are booleans that default to an explicit "no"; `required` on a checkbox means must-agree/consent semantics. The response protocol is deliberately unchanged (v1 `text`/`option` only): form cards submit a readable text summary as `response_kind: "text"`, so journal persistence and answered-card recovery need no new allowlist entries. Because this middleware can short-circuit tool execution before LangChain emits `on_tool_end`, `RunJournal` performs a root-run final reconciliation for allowlisted clarification `ToolMessage`s whose `tool_call_id` was produced by the current run, so human-input request cards remain recoverable from `run_events` after checkpoint compaction. Human Input Card replies are submitted as `hide_from_ui` `HumanMessage`s with `additional_kwargs.human_input_response`; `RunJournal` persists only allowlisted hidden response sources (currently `ask_clarification`) as `llm.human.input`, which preserves answered-card state after compaction without exposing generic internal hidden context. +### Python Extension System (Middleware Slice) + +Third-party Python packages can expose an `install(registry, config)` function and be +loaded, in deterministic order, from the startup-only top-level `plugins:` list in +`config.yaml`. Keep this list out of `extensions_config.json`: the latter is writable +through Gateway APIs, while importing Python entry points is an operator-controlled code +execution boundary. A plugin marked `required: true` fails Gateway construction when it +cannot load; optional plugins fail open with attributed diagnostics. + +The public package is `packages/extension-api/` and must never import `deerflow`. In this +slice its registry contract exposes middleware contribution only. Each contribution +declares lead/subagent scope, stable order, and a semantic placement (`MODEL_LOGICAL`, +`MODEL_PHYSICAL`, `TOOL_VISIBLE`, `TOOL_RAW`, or `STANDARD`) rather than a fragile list +index. `extensions/stack.py` is the single final composition point; do not inject inside +the shared base builder because the lead builder appends more middleware afterward. +`extensions/ordering.py` owns host ordering invariants and validates the final composed +stack. Nothing under `extensions/` may import `agents.middlewares` at module scope: the +middleware layer calls into this one, so a module-scope reference points the dependency +backwards and closes a cycle as soon as any middleware imports something under +`extensions/` at module level. Both tables that need middleware classes therefore resolve +on first use — `ordering.py::core_ordering_constraints()` and `stack.py::_anchors()` — +which is `assert_ordering` / composition time, already inside the middleware builder. +Defer by deferring the *call*; do not fake a resolved value with a lazy container +subclass, which reports one answer when iterated and another when measured. + +Contributed middlewares are wrapped by `IsolatedMiddleware`: extension failures emit +diagnostics and fail open without repeating a downstream model/tool side effect. The +wrapper mirrors lifecycle hooks, tools, transformers, and state schema implemented by +the inner middleware. LangChain treats each sync/async model or tool wrapper pair as one +capability, so a single-sided wrapper receives a pass-through counterpart; implement +both sides when the extension must observe both synchronous and asynchronous execution +paths. Lead runs and +subagents allocate an `ExtensionData` task store only when middleware contributors are +present and expose it through `EXTENSION_TASK_STORE_KEY`; extensions retrieve it with +`task_store_from_runtime()`. Each run resolves the immutable loaded-extension snapshot +once and binds that same object through task-store allocation and synchronous agent +construction, so a concurrent singleton replacement cannot mix two extension +generations without changing the LangGraph graph-factory ABI. The graph-build binding is +a ContextVar scoped to synchronous construction, so it has already exited by the time the +lead agent delegates; the run worker therefore also publishes the snapshot on runtime +context under the host-internal `EXTENSION_SNAPSHOT_CONTEXT_KEY`, `task_tool` reads it +back through `resolve_run_extensions()` (type-checked — runtime context is +caller-mergeable), and `SubagentExecutor` binds it at construction. That key is written +after the caller merge and popped when the run has none, so a caller-supplied value is +never authoritative. Absent the key — embedded `DeerFlowClient`, standalone LangGraph +Server — the executor keeps its `get_loaded_extensions()` fallback. + +Gateway `create_app()` loads plugins once, stores the immutable registry on `app.state` +and in the process-wide singleton, and installs one canonical live diagnostics list. +Changing `plugins` requires a restart. Later extension contribution points must be added +to the public contract and host runtime in the same slice; never accept a registration +method that the current host silently ignores. + ### Configuration System **Main Configuration** (`config.yaml`): @@ -447,7 +502,7 @@ Setup: Copy `config.example.yaml` to `config.yaml` in the **project root** direc **Config Hot-Reload Boundary**: Gateway dependencies route through `get_app_config()` on every request, so per-run fields like `models[*].max_tokens`, `summarization.*`, `title.*`, `memory.*`, `subagents.*`, `tools[*]`, and the agent system prompt pick up `config.yaml` edits on the next message. `AppConfig` is intentionally **not** cached on `app.state` — `lifespan()` keeps a local `startup_config` variable for one-shot bootstrap work and passes it to `langgraph_runtime(app, startup_config)`. -Infrastructure fields are **restart-required**. The authoritative list lives in `packages/harness/deerflow/config/reload_boundary.py::STARTUP_ONLY_FIELDS` and is mirrored by the standardised `"startup-only:"` prefix on the corresponding `Field(description=...)` in `AppConfig`, so IDE hover on those fields surfaces the reason inline (no need to context-switch into this table). Currently registered: `database`, `checkpointer`, `run_events`, `stream_bridge`, `sandbox`, `log_level`, `logging`, `channels`, `channel_connections`, `scheduler`, `run_ownership`. Adding a new restart-required field requires updating the registry; drift is pinned by `tests/test_reload_boundary.py`. +Infrastructure fields are **restart-required**. The authoritative list lives in `packages/harness/deerflow/config/reload_boundary.py::STARTUP_ONLY_FIELDS` and is mirrored by the standardised `"startup-only:"` prefix on the corresponding `Field(description=...)` in `AppConfig`, so IDE hover on those fields surfaces the reason inline (no need to context-switch into this table). Currently registered: `plugins`, `database`, `checkpointer`, `run_events`, `stream_bridge`, `sandbox`, `log_level`, `logging`, `channels`, `channel_connections`, `scheduler`, `run_ownership`. Adding a new restart-required field requires updating the registry; drift is pinned by `tests/test_reload_boundary.py`. **Persistence backend resolution**: the unified `database` section selects the Gateway's LangGraph checkpointer, LangGraph Store, and DeerFlow SQL repositories. diff --git a/backend/app/gateway/app.py b/backend/app/gateway/app.py index 3ee88a29b..2ccfe63e5 100644 --- a/backend/app/gateway/app.py +++ b/backend/app/gateway/app.py @@ -569,6 +569,44 @@ This gateway provides runtime endpoints for agent runs plus custom endpoints for # configure_logging() at lifespan startup) out of sync with the middleware. app.add_middleware(TraceMiddleware, enabled=_resolve_trace_enabled_for_app_construction()) + # Python extensions load once while the Gateway app is constructed. Agent + # middleware builders consume the same immutable set through the process + # singleton; app.state exposes it to the Gateway runtime. + from deerflow.extensions import ( + EMPTY_EXTENSIONS, + ExtensionLoadError, + initialize_runtime_diagnostics, + load_extensions, + set_loaded_extensions, + ) + + # Resolving the configured plugin list is deliberately outside the + # fail-open guard below: a config.yaml that exists but cannot be parsed or + # validated is a configuration failure, not an extension failure. Reporting + # it as the latter would silently drop a `required: true` extension instead + # of failing the boot. Only an absent config.yaml is tolerated, mirroring + # _resolve_trace_enabled_for_app_construction() — create_app() runs at + # import time, and lifespan still performs strict config loading before + # serving. + try: + configured_plugins = get_app_config().plugins + except FileNotFoundError: + logger.debug("config.yaml not found while constructing Gateway app; loading no extensions for this app instance") + configured_plugins = [] + + try: + loaded_extensions, extension_diagnostics = load_extensions(configured_plugins) + except ExtensionLoadError: + # `required: true` makes the extension part of the startup contract. + # Booting without it would silently change configured behaviour. + raise + except Exception: + logger.exception("Extension loading failed; continuing with no extensions") + loaded_extensions, extension_diagnostics = EMPTY_EXTENSIONS, [] + set_loaded_extensions(loaded_extensions) + app.state.extensions = loaded_extensions + app.state.extension_diagnostics = initialize_runtime_diagnostics(extension_diagnostics) + # Include routers # Models API is mounted at /api/models app.include_router(models.router) diff --git a/backend/app/gateway/deps.py b/backend/app/gateway/deps.py index 9c5540b80..1f0c7662f 100644 --- a/backend/app/gateway/deps.py +++ b/backend/app/gateway/deps.py @@ -596,6 +596,7 @@ def get_run_context(request: Request) -> RunContext: checkpoint_snapshot_frequency=getattr(request.app.state, "checkpoint_snapshot_frequency", None), thread_store=get_thread_store(request), app_config=get_config(), + extensions=getattr(request.app.state, "extensions", None), on_run_completed=getattr(request.app.state, "scheduled_task_service", None).handle_run_completion if getattr(request.app.state, "scheduled_task_service", None) is not None else None, ) diff --git a/backend/extension_test_fixtures/__init__.py b/backend/extension_test_fixtures/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/backend/extension_test_fixtures/demo_extensions.py b/backend/extension_test_fixtures/demo_extensions.py new file mode 100644 index 000000000..79cdb315e --- /dev/null +++ b/backend/extension_test_fixtures/demo_extensions.py @@ -0,0 +1,75 @@ +"""Install functions used by the extension loader tests.""" + +from __future__ import annotations + +from collections.abc import Mapping +from typing import Any + +from deerflow_extension_api import ExtensionRegistry, extension + +INSTALLED: list[str] = [] + + +class _Contributor: + def __init__(self, tag: str) -> None: + self.tag = tag + + +def install_ok(registry: ExtensionRegistry, config: Mapping[str, Any]) -> None: + INSTALLED.append("ok") + registry.middlewares(_Contributor("ok")) + + +@extension(api="0.1", name="stamped") +def install_stamped(registry: ExtensionRegistry, config: Mapping[str, Any]) -> None: + INSTALLED.append("stamped") + registry.middlewares(_Contributor("stamped")) + + +@extension(api="99.0", name="future") +def install_future_api(registry: ExtensionRegistry, config: Mapping[str, Any]) -> None: + INSTALLED.append("future") + registry.middlewares(_Contributor("future")) + + +@extension(api="0.2", name="newer-minor") +def install_newer_minor_api(registry: ExtensionRegistry, config: Mapping[str, Any]) -> None: + """Written against a newer 0.x minor than the host provides: before 1.0, + minors carry no compatibility promise in either direction.""" + INSTALLED.append("newer-minor") + registry.middlewares(_Contributor("newer-minor")) + + +def install_partial_then_raise(registry: ExtensionRegistry, config: Mapping[str, Any]) -> None: + """Registers two contributors, then fails — exercises rollback.""" + registry.middlewares(_Contributor("partial-a")) + registry.middlewares(_Contributor("partial-b")) + raise ValueError("boom") + + +def install_reads_config(registry: ExtensionRegistry, config: Mapping[str, Any]) -> None: + INSTALLED.append(f"config:{config.get('mode')}") + + +def install_disabled(registry: ExtensionRegistry, config: Mapping[str, Any]) -> None: + """Registers nothing when disabled — the zero-cost path.""" + if not config.get("enabled", False): + return + registry.middlewares(_Contributor("enabled")) + + +def install_shared_use(registry: ExtensionRegistry, config: Mapping[str, Any]) -> None: + """Registers a middleware contributor; raises afterward if configured to. + + Two ExtensionSpecs may legitimately share the same `use` with different + config (e.g. the same extension mounted twice with different settings). + This exists to exercise that rollback must be positional, not keyed by + `use` — a failing instance must not erase an earlier, successful + instance's registrations just because they share a source string. + """ + registry.middlewares(_Contributor(f"shared:{config.get('label', '')}")) + if config.get("fail"): + raise ValueError("boom-shared") + + +NOT_CALLABLE = "i am not a function" diff --git a/backend/packages/extension-api/deerflow_extension_api/__init__.py b/backend/packages/extension-api/deerflow_extension_api/__init__.py new file mode 100644 index 000000000..ec2778e0a --- /dev/null +++ b/backend/packages/extension-api/deerflow_extension_api/__init__.py @@ -0,0 +1,49 @@ +"""Public contracts for DeerFlow extensions. + +This package MUST NOT import `deerflow`. Everything an extension needs to +integrate lives here, so an extension depends on this package alone and can +be released independently of the host. +""" + +from __future__ import annotations + +from deerflow_extension_api.contracts import ( + ExtensionInstall, + ExtensionRegistry, + HostPolicySnapshot, + MiddlewareContributor, + extension, +) +from deerflow_extension_api.placement import ( + AgentBuildContext, + AgentScope, + MiddlewarePlacement, + Placement, +) +from deerflow_extension_api.runtime_bridge import ( + EXTENSION_TASK_STORE_KEY, + task_store_from_runtime, +) +from deerflow_extension_api.state import ExtensionData + +#: Contract version. Pre-1.0: the contract surface is observational only +#: (contributors and observers), so minors may break and only patches promise +#: to be additive. From 1.0 on, bump the major on any breaking change; see the +#: spec's evolution rules for what counts as additive. +API_VERSION = "0.1.0" + +__all__ = [ + "API_VERSION", + "EXTENSION_TASK_STORE_KEY", + "AgentBuildContext", + "AgentScope", + "ExtensionData", + "ExtensionInstall", + "ExtensionRegistry", + "HostPolicySnapshot", + "MiddlewareContributor", + "MiddlewarePlacement", + "Placement", + "extension", + "task_store_from_runtime", +] diff --git a/backend/packages/extension-api/deerflow_extension_api/contracts.py b/backend/packages/extension-api/deerflow_extension_api/contracts.py new file mode 100644 index 000000000..d92cd6e50 --- /dev/null +++ b/backend/packages/extension-api/deerflow_extension_api/contracts.py @@ -0,0 +1,96 @@ +"""The extension contracts and their data types. + +Compatibility rules enforced throughout this module: + * every Protocol method carries a default implementation, so adding a method + later stays additive for already-released extensions; + * every optional dataclass field carries a default, so adding a field stays + additive. +""" + +from __future__ import annotations + +from collections.abc import Callable, Mapping, Sequence +from dataclasses import dataclass +from typing import TYPE_CHECKING, Any, Protocol, TypeVar, runtime_checkable + +from deerflow_extension_api.state import ExtensionData + +if TYPE_CHECKING: # pragma: no cover - typing only + from deerflow_extension_api.placement import AgentBuildContext, MiddlewarePlacement + +F = TypeVar("F", bound=Callable[..., Any]) + + +# --- Host projections ------------------------------------------------------- + + +@dataclass(frozen=True) +class HostPolicySnapshot: + """The limits the host actually enforces, projected for extensions. + + A narrow projection instead of the host's AppConfig: exposing AppConfig + would pin every extension to the harness release cadence. Every field has + a default so widening this stays additive. + """ + + token_budget_enabled: bool = False + max_input_tokens: int | None = None + max_output_tokens: int | None = None + max_total_tokens: int | None = None + budget_warn_fraction: float | None = None + budget_hard_fraction: float | None = None + max_subagents_per_run: int | None = None + + +# --- Middleware ------------------------------------------------------------- + + +class MiddlewareContributor(Protocol): + def contribute_middlewares( + self, + app_store: ExtensionData, + ctx: AgentBuildContext, + ) -> Sequence[MiddlewarePlacement]: + return () + + +# --- Registration surface --------------------------------------------------- + + +@runtime_checkable +class ExtensionRegistry(Protocol): + """The write-only registration surface handed to ``install()``. + + Structural and minimal on purpose. This first capability slice exposes + middleware contribution only; later slices can add defaulted registration + methods without breaking existing implementations. The host's concrete + registry additionally carries host-only machinery (attribution, positional + rollback, build) that is deliberately absent here. + """ + + def middlewares(self, contributor: MiddlewareContributor) -> None: + return None + + +#: The install() entry point signature every extension exposes. +ExtensionInstall = Callable[[ExtensionRegistry, Mapping[str, Any]], None] + + +# --- Declaration decorator -------------------------------------------------- + + +def extension(*, api: str, name: str | None = None) -> Callable[[F], F]: + """Stamp an install function with the API version it was written against. + + Optional. pip's dependency resolution is the primary compatibility + mechanism; this covers `--no-deps` installs and editable monorepo checkouts + where versions can skew, and turns a deep AttributeError into an + actionable startup diagnostic. + """ + + def _decorate(func: F) -> F: + func.__deerflow_api__ = api # type: ignore[attr-defined] + func.__deerflow_name__ = name # type: ignore[attr-defined] + return func + + return _decorate diff --git a/backend/packages/extension-api/deerflow_extension_api/placement.py b/backend/packages/extension-api/deerflow_extension_api/placement.py new file mode 100644 index 000000000..ff534d1f2 --- /dev/null +++ b/backend/packages/extension-api/deerflow_extension_api/placement.py @@ -0,0 +1,70 @@ +"""Where an extension's middleware sits in the host's middleware stack. + +Placement is declared as a *semantic guarantee* ("I need to observe the raw +tool return") rather than as a structural position ("put me in layer 3"). A +middleware occupies one index in the list, but that index only has meaning on +the hook chain it actually implements — so "outermost" means different things +on the model axis and the tool axis. Declaring by axis-and-end removes that +ambiguity and keeps the host free to restructure its stack. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from enum import Flag, StrEnum, auto +from typing import Any + +from deerflow_extension_api.contracts import HostPolicySnapshot + + +class Placement(StrEnum): + MODEL_LOGICAL = "model_logical" + """Model axis, outer end. Guarantee: outer of retry and error handling. + Fires once per logical decision regardless of how many times the host + retries underneath.""" + + MODEL_PHYSICAL = "model_physical" + """Model axis, inner end. Guarantee: inner of every request-transforming + middleware. Fires once per physical provider call; retries re-enter it.""" + + TOOL_VISIBLE = "tool_visible" + """Tool axis, outer end. Guarantee: outer of truncation, sanitization and + error wrapping. Observes what the model finally sees.""" + + TOOL_RAW = "tool_raw" + """Tool axis, inner end. Guarantee: adjacent to the real callable + boundary. Observes the tool's raw return before any processing.""" + + STANDARD = "standard" + """No before/after-processing requirement. Relative order against other + STANDARD contributors is not guaranteed.""" + + +class AgentScope(Flag): + LEAD = auto() + SUBAGENT = auto() + BOTH = LEAD | SUBAGENT + + +@dataclass(frozen=True) +class AgentBuildContext: + """What an extension may know while deciding what to contribute.""" + + scope: AgentScope + agent_name: str | None = None + model_name: str | None = None + policy: HostPolicySnapshot = field(default_factory=HostPolicySnapshot) + + +@dataclass(frozen=True) +class MiddlewarePlacement: + """One middleware plus where it needs to sit. + + ``middleware`` is typed ``Any`` rather than ``AgentMiddleware`` so this + module stays import-light; the host validates the type at injection time. + """ + + middleware: Any + placement: Placement + scope: AgentScope = AgentScope.BOTH + order: int = 0 diff --git a/backend/packages/extension-api/deerflow_extension_api/py.typed b/backend/packages/extension-api/deerflow_extension_api/py.typed new file mode 100644 index 000000000..8b1378917 --- /dev/null +++ b/backend/packages/extension-api/deerflow_extension_api/py.typed @@ -0,0 +1 @@ + diff --git a/backend/packages/extension-api/deerflow_extension_api/runtime_bridge.py b/backend/packages/extension-api/deerflow_extension_api/runtime_bridge.py new file mode 100644 index 000000000..a83902808 --- /dev/null +++ b/backend/packages/extension-api/deerflow_extension_api/runtime_bridge.py @@ -0,0 +1,25 @@ +"""Bridge between LangGraph's runtime context and the task-scoped store. + +Middlewares run inside the agent graph and can only reach host state through +``request.runtime``. The host installs the task store under a host-owned key; +extensions read it through this helper and keep their own objects *inside* the +store, so two extensions cannot collide on a runtime-context key. +""" + +from __future__ import annotations + +from collections.abc import Mapping + +from deerflow_extension_api.state import ExtensionData + +#: Host-owned key. Extensions must not write to the runtime context directly. +EXTENSION_TASK_STORE_KEY = "__deerflow_extension_task_store" + + +def task_store_from_runtime(runtime: object) -> ExtensionData | None: + """Return the task-scoped store, or None when there is no live task.""" + context = getattr(runtime, "context", None) + if not isinstance(context, Mapping): + return None + store = context.get(EXTENSION_TASK_STORE_KEY) + return store if isinstance(store, ExtensionData) else None diff --git a/backend/packages/extension-api/deerflow_extension_api/state.py b/backend/packages/extension-api/deerflow_extension_api/state.py new file mode 100644 index 000000000..f95a09978 --- /dev/null +++ b/backend/packages/extension-api/deerflow_extension_api/state.py @@ -0,0 +1,57 @@ +"""Per-scope typed storage handed to extensions.""" + +from __future__ import annotations + +from collections.abc import Callable +from threading import RLock +from typing import Any + + +class ExtensionData: + """Extension-private state attached to one host-owned scope. + + Keyed by type rather than by string so independent extensions cannot + collide on a key. The host creates one instance per scope (app, task) and + drops it when that scope ends, which is why extensions never need a + stale-handle check: they are handed the store for the current scope on + every callback instead of capturing one. + """ + + __slots__ = ("_scope_id", "_entries", "_lock") + + def __init__(self, scope_id: str) -> None: + self._scope_id = scope_id + self._entries: dict[type, Any] = {} + self._lock = RLock() + + @property + def scope_id(self) -> str: + """Host identity of the scope this store is attached to.""" + return self._scope_id + + def get[T](self, typ: type[T]) -> T | None: + with self._lock: + return self._entries.get(typ) + + def get_or_init[T](self, typ: type[T], init: Callable[[], T]) -> T: + """Return the stored value, creating it from ``init`` when absent. + + ``init`` runs while the store is locked. It may compose other state in + this store, but heavyweight lazy work belongs inside the stored value + itself. + """ + with self._lock: + existing = self._entries.get(typ) + if existing is not None: + return existing + created = init() + self._entries[typ] = created + return created + + def set[T](self, value: T) -> None: + with self._lock: + self._entries[type(value)] = value + + def remove[T](self, typ: type[T]) -> T | None: + with self._lock: + return self._entries.pop(typ, None) diff --git a/backend/packages/extension-api/pyproject.toml b/backend/packages/extension-api/pyproject.toml new file mode 100644 index 000000000..70a84a157 --- /dev/null +++ b/backend/packages/extension-api/pyproject.toml @@ -0,0 +1,15 @@ +[project] +name = "deerflow-extension-api" +version = "0.1.0" +description = "Public contracts for DeerFlow extensions" +requires-python = ">=3.12" +# Keep the contract package import-light and independent from the host. Public +# values are stdlib-based; concrete middleware types are validated by DeerFlow. +dependencies = [] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.hatch.build.targets.wheel] +packages = ["deerflow_extension_api"] diff --git a/backend/packages/harness/deerflow/agents/lead_agent/agent.py b/backend/packages/harness/deerflow/agents/lead_agent/agent.py index 1b3fb8eae..688a52f40 100644 --- a/backend/packages/harness/deerflow/agents/lead_agent/agent.py +++ b/backend/packages/harness/deerflow/agents/lead_agent/agent.py @@ -382,6 +382,7 @@ def build_middlewares( mcp_routing_middleware: AgentMiddleware | None = None, user_id: str | None = None, authorization_provider=None, + extensions=None, ): """Build the lead-agent middleware chain based on runtime configuration. @@ -404,6 +405,8 @@ def build_middlewares( to ``SkillActivationMiddleware`` so it can resolve per-user custom skills. authorization_provider: Provider already resolved for assembly-time filtering. Reused by the execution-time authorization middleware. + extensions: Loaded extensions whose middleware contributions are merged + into the final stack. Defaults to the process-wide set. Returns: List of middleware instances. @@ -528,9 +531,11 @@ def build_middlewares( # Add SubagentLimitMiddleware to truncate excess parallel task calls subagent_enabled = cfg.get("subagent_enabled", False) + effective_max_subagents_per_run: int | None = None if subagent_enabled: max_concurrent_subagents = cfg.get("max_concurrent_subagents", 3) max_total_subagents = cfg.get("max_total_subagents", _default_max_total_subagents(resolved_app_config)) + effective_max_subagents_per_run = max_total_subagents middlewares.append(SubagentLimitMiddleware(max_concurrent=max_concurrent_subagents, max_total=max_total_subagents)) # LoopDetectionMiddleware — detect and break repetitive tool call loops @@ -575,7 +580,39 @@ def build_middlewares( # ClarificationMiddleware should always be last middlewares.append(ClarificationMiddleware()) - return middlewares + + # Extension contributions are merged only here, once the full stack exists. + # Doing it inside build_lead_runtime_middlewares() would place + # MODEL_PHYSICAL contributions above the lead-specific middlewares appended + # above, changing what "the final request" means for observers. + from deerflow_extension_api import AgentScope + + from deerflow.extensions import get_agent_build_extensions + from deerflow.extensions.stack import compose_with_extensions + + resolved_extensions = extensions if extensions is not None else get_agent_build_extensions() + if not resolved_extensions.has_middleware_contributors: + return compose_with_extensions(middlewares, AgentScope.LEAD, None, resolved_extensions) + + from deerflow_extension_api import AgentBuildContext + + from deerflow.extensions.policy import project_host_policy + + return compose_with_extensions( + middlewares, + AgentScope.LEAD, + AgentBuildContext( + scope=AgentScope.LEAD, + agent_name=agent_name, + model_name=model_name, + policy=project_host_policy( + resolved_app_config, + token_budget_config=token_budget_config, + max_subagents_per_run=effective_max_subagents_per_run, + ), + ), + resolved_extensions, + ) def _available_skill_names(agent_config, is_bootstrap: bool) -> set[str] | None: diff --git a/backend/packages/harness/deerflow/agents/middlewares/tool_error_handling_middleware.py b/backend/packages/harness/deerflow/agents/middlewares/tool_error_handling_middleware.py index eaae4ddc2..92674698f 100644 --- a/backend/packages/harness/deerflow/agents/middlewares/tool_error_handling_middleware.py +++ b/backend/packages/harness/deerflow/agents/middlewares/tool_error_handling_middleware.py @@ -270,26 +270,19 @@ def _build_runtime_middlewares( # on every result before ToolProgressMiddleware reads it in _update_state_from_result. # Framework rule: first in list = outermost (types.py: "compose with first in list as outermost layer"). tool_progress_config = app_config.tool_progress - _ToolProgressMiddleware = None if tool_progress_config.enabled: - from deerflow.agents.middlewares.tool_progress_middleware import ToolProgressMiddleware as _ToolProgressMiddleware + from deerflow.agents.middlewares.tool_progress_middleware import ToolProgressMiddleware - tail.append(_ToolProgressMiddleware.from_config(tool_progress_config)) + tail.append(ToolProgressMiddleware.from_config(tool_progress_config)) tail.append(ToolErrorHandlingMiddleware(app_config=app_config)) middlewares = [*outer_wrappers, *thread_hooks, *tail] - # Guard: ToolProgressMiddleware (outer) must appear before ToolErrorHandlingMiddleware (inner) - # so that its wrap_tool_call chain encloses the stamping step. Fail loudly at build time - # rather than silently no-oping at runtime if a future insertion reverses the order. - # Uses isinstance (not type().__name__) so subclasses and renames are covered. - if _ToolProgressMiddleware is not None: - _progress_idx = next((i for i, m in enumerate(middlewares) if isinstance(m, _ToolProgressMiddleware)), None) - _error_idx = next((i for i, m in enumerate(middlewares) if isinstance(m, ToolErrorHandlingMiddleware)), None) - if _progress_idx is not None and _error_idx is not None and _progress_idx > _error_idx: - raise RuntimeError(f"ToolProgressMiddleware must be outer (index {_progress_idx}) of ToolErrorHandlingMiddleware (index {_error_idx}) — check middleware append order") - + # Ordering invariants are declared in deerflow.extensions.ordering and + # validated once at the end of the composing builder, after extension + # contributions are merged in — otherwise a contribution could silently + # reverse an invariant this builder had already checked. return middlewares @@ -322,6 +315,7 @@ def build_subagent_runtime_middlewares( available_skills: set[str] | None = None, user_id: str | None = None, authorization_provider=None, + extensions=None, ) -> list[AgentMiddleware]: """Middlewares shared by subagent runtime before subagent-only middlewares.""" if app_config is None: @@ -528,4 +522,31 @@ def build_subagent_runtime_middlewares( middlewares.append(SystemMessageCoalescingMiddleware()) - return middlewares + from deerflow_extension_api import AgentScope + + from deerflow.extensions import get_agent_build_extensions + from deerflow.extensions.stack import compose_with_extensions + + resolved_extensions = extensions if extensions is not None else get_agent_build_extensions() + if not resolved_extensions.has_middleware_contributors: + return compose_with_extensions(middlewares, AgentScope.SUBAGENT, None, resolved_extensions) + + from deerflow_extension_api import AgentBuildContext + + from deerflow.extensions.policy import project_host_policy + + return compose_with_extensions( + middlewares, + AgentScope.SUBAGENT, + AgentBuildContext( + scope=AgentScope.SUBAGENT, + agent_name=agent_name, + model_name=model_name, + policy=project_host_policy( + app_config, + token_budget_config=token_budget_config, + max_subagents_per_run=None, + ), + ), + resolved_extensions, + ) diff --git a/backend/packages/harness/deerflow/config/app_config.py b/backend/packages/harness/deerflow/config/app_config.py index ae11a3f95..ce16689ca 100644 --- a/backend/packages/harness/deerflow/config/app_config.py +++ b/backend/packages/harness/deerflow/config/app_config.py @@ -48,6 +48,7 @@ from deerflow.config.tool_config import ToolConfig, ToolGroupConfig from deerflow.config.tool_output_config import ToolOutputConfig from deerflow.config.tool_progress_config import ToolProgressConfig from deerflow.config.tool_search_config import ToolSearchConfig, load_tool_search_config_from_dict +from deerflow.extensions.loader import ExtensionSpec load_dotenv() @@ -203,6 +204,19 @@ class AppConfig(BaseModel): ) token_usage: TokenUsageConfig = Field(default_factory=TokenUsageConfig, description="Token usage tracking configuration") token_budget: TokenBudgetConfig = Field(default_factory=TokenBudgetConfig, description="Token Budget tracking and limits configuration.") + plugins: list[ExtensionSpec] = Field( + default_factory=list, + description=format_field_description( + "plugins", + field_doc=( + "Extension packages to load at startup, in order. Each entry names an install " + "entry point as 'module.path:install' and carries its own private config block. " + "Distinct from the `extensions` field above, which configures MCP servers, skills " + "and config-declared middlewares and is backed by the HTTP-writable " + "extensions_config.json." + ), + ), + ) max_recursion_limit: int = Field( default=1000, ge=1, diff --git a/backend/packages/harness/deerflow/config/reload_boundary.py b/backend/packages/harness/deerflow/config/reload_boundary.py index 6f6f87d62..759bdae33 100644 --- a/backend/packages/harness/deerflow/config/reload_boundary.py +++ b/backend/packages/harness/deerflow/config/reload_boundary.py @@ -43,6 +43,7 @@ STARTUP_ONLY_PREFIX = "startup-only:" #: field is restart-required — so an operator changing the value knows #: which subsystem to restart. STARTUP_ONLY_FIELDS: dict[str, str] = { + "plugins": ("load_extensions() runs once during create_app() and the process-wide middleware registry is not rebuilt on config.yaml edits; adding, removing or reconfiguring a plugin requires a restart."), "database": ("init_engine_from_config() runs once during langgraph_runtime() startup; the SQLAlchemy engine holds the connection pool and is not rebuilt on config.yaml edits."), "checkpointer": ("make_checkpointer() binds the persistent checkpointer once at startup, including SQLite WAL / busy_timeout settings."), "run_events": ("make_run_event_store() picks the memory- vs SQL-backed implementation at startup and is frozen onto app.state.run_events_config to stay paired with the underlying event store."), diff --git a/backend/packages/harness/deerflow/extensions/__init__.py b/backend/packages/harness/deerflow/extensions/__init__.py new file mode 100644 index 000000000..efbbb7a99 --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/__init__.py @@ -0,0 +1,158 @@ +"""DeerFlow's extension mechanism (host side). + +The public contracts live in the separate `deerflow-extension-api` package; +this module implements loading, registration, middleware injection and the +hook-site plumbing. +""" + +from __future__ import annotations + +import threading +from collections.abc import Iterator, Mapping +from contextlib import contextmanager +from contextvars import ContextVar +from typing import Any + +from deerflow.extensions.loader import ( + Diagnostic, + ExtensionLoadError, + ExtensionSpec, + load_extensions, +) +from deerflow.extensions.registry import EMPTY_EXTENSIONS, ExtensionRegistry, LoadedExtensions + +#: Runtime-context key carrying the run's immutable extension snapshot. +#: +#: The graph-build binding below is a ContextVar scoped to synchronous agent +#: construction, so it is long gone by the time a tool delegates work. Runtime +#: context is how the run reaches that later code. The double-underscore prefix +#: marks it as host-internal: the Gateway strips caller-supplied ``__`` keys, +#: and this snapshot is never part of the public extension contract. +EXTENSION_SNAPSHOT_CONTEXT_KEY = "__deerflow_extension_snapshot" + +_loaded: LoadedExtensions = EMPTY_EXTENSIONS +_agent_build_extensions: ContextVar[LoadedExtensions | None] = ContextVar( + "deerflow_agent_build_extensions", + default=None, +) + + +def get_loaded_extensions() -> LoadedExtensions: + """Return the process-wide loaded extensions. + + Mirrors the existing `get_app_config()` convention so call sites can take + an explicit override parameter and fall back to this. + """ + return _loaded + + +def get_agent_build_extensions() -> LoadedExtensions: + """Return the run-bound snapshot while an agent graph is being built.""" + return _agent_build_extensions.get() or get_loaded_extensions() + + +@contextmanager +def bind_agent_build_extensions(loaded: LoadedExtensions) -> Iterator[None]: + """Bind one immutable extension snapshot to synchronous graph assembly.""" + token = _agent_build_extensions.set(loaded) + try: + yield + finally: + _agent_build_extensions.reset(token) + + +def resolve_run_extensions(context: Any | None) -> LoadedExtensions | None: + """Return the run's extension snapshot from *context*, or ``None``. + + Runtime context is caller-mergeable, so the value is type-checked rather + than trusted. ``None`` means "this caller installed no snapshot" (embedded + client, standalone LangGraph Server) and leaves consumers on their existing + ``get_loaded_extensions()`` fallback. + """ + if not isinstance(context, Mapping): + return None + snapshot = context.get(EXTENSION_SNAPSHOT_CONTEXT_KEY) + return snapshot if isinstance(snapshot, LoadedExtensions) else None + + +def set_loaded_extensions(loaded: LoadedExtensions) -> None: + global _loaded + _loaded = loaded + + +def reset_loaded_extensions() -> None: + """Reset to a FRESH empty set. Used by tests to prevent singleton leaks. + + Builds a new instance rather than reusing EMPTY_EXTENSIONS: that singleton + owns a mutable ExtensionData app_store, so resetting to it would carry any + write made while "empty" across every later reset and across the process. + """ + global _loaded + _loaded = ExtensionRegistry().build() + + +_runtime_diagnostics: list[Diagnostic] = [] +_runtime_diagnostics_lock = threading.RLock() +_MAX_RUNTIME_DIAGNOSTICS = 1000 + + +def _trim_runtime_diagnostics() -> None: + overflow = len(_runtime_diagnostics) - _MAX_RUNTIME_DIAGNOSTICS + if overflow > 0: + del _runtime_diagnostics[:overflow] + + +def initialize_runtime_diagnostics(diagnostics: list[Diagnostic]) -> list[Diagnostic]: + """Install and return the live diagnostic list for the current host.""" + with _runtime_diagnostics_lock: + _runtime_diagnostics.clear() + _runtime_diagnostics.extend(diagnostics) + _trim_runtime_diagnostics() + return _runtime_diagnostics + + +def record_runtime_diagnostic(diagnostic: Diagnostic) -> None: + """Collect one diagnostic in the canonical process sink.""" + with _runtime_diagnostics_lock: + _runtime_diagnostics.append(diagnostic) + _trim_runtime_diagnostics() + + +def record_runtime_diagnostics(diagnostics: list[Diagnostic]) -> None: + """Collect a diagnostic batch in the canonical process sink.""" + with _runtime_diagnostics_lock: + _runtime_diagnostics.extend(diagnostics) + _trim_runtime_diagnostics() + + +def get_runtime_diagnostics() -> list[Diagnostic]: + with _runtime_diagnostics_lock: + return list(_runtime_diagnostics) + + +def reset_runtime_diagnostics() -> None: + with _runtime_diagnostics_lock: + _runtime_diagnostics.clear() + + +__all__ = [ + "EMPTY_EXTENSIONS", + "EXTENSION_SNAPSHOT_CONTEXT_KEY", + "Diagnostic", + "ExtensionLoadError", + "ExtensionRegistry", + "ExtensionSpec", + "LoadedExtensions", + "bind_agent_build_extensions", + "get_agent_build_extensions", + "get_loaded_extensions", + "get_runtime_diagnostics", + "initialize_runtime_diagnostics", + "load_extensions", + "record_runtime_diagnostic", + "record_runtime_diagnostics", + "reset_loaded_extensions", + "reset_runtime_diagnostics", + "resolve_run_extensions", + "set_loaded_extensions", +] diff --git a/backend/packages/harness/deerflow/extensions/anchors.py b/backend/packages/harness/deerflow/extensions/anchors.py new file mode 100644 index 000000000..9f3b57fd7 --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/anchors.py @@ -0,0 +1,120 @@ +"""Translating semantic placements into concrete stack indices. + +This is the only module that knows the shape of DeerFlow's middleware stack. +Restructuring the stack means updating the anchor table here; extensions, +which declare only what they need to observe, stay untouched. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from dataclasses import dataclass +from typing import Literal + +_Side = Literal[ + "outer", + "inner", + "outer_last", + "inner_last", + "inner_last_after", + "start", + "end", +] + + +@dataclass(frozen=True) +class AnchorRule: + """One attempt at locating an insertion index. + + ``side`` "outer"/"inner" position relative to the first middleware whose + type is in ``types``; "outer_last"/"inner_last" use the last matching + middleware; "inner_last_after" additionally requires that match to follow + the last middleware in ``after_types``. "start"/"end" are the absolute + ends of the stack and ignore ``types``. + """ + + side: _Side + types: tuple[type, ...] = () + after_types: tuple[type, ...] = () + + def resolve(self, middlewares: Sequence[object]) -> int | None: + if self.side == "start": + return 0 + if self.side == "end": + return len(middlewares) + if self.side in {"outer_last", "inner_last"}: + for index in range(len(middlewares) - 1, -1, -1): + if isinstance(middlewares[index], self.types): + return index if self.side == "outer_last" else index + 1 + return None + if self.side == "inner_last_after": + boundary = next( + (index for index in range(len(middlewares) - 1, -1, -1) if isinstance(middlewares[index], self.after_types)), + None, + ) + if boundary is None: + return None + for index in range(len(middlewares) - 1, boundary, -1): + if isinstance(middlewares[index], self.types): + return index + 1 + return None + for index, middleware in enumerate(middlewares): + if isinstance(middleware, self.types): + return index if self.side == "outer" else index + 1 + return None + + +@dataclass(frozen=True) +class PlacementAnchor: + """An ordered fallback chain of anchor rules.""" + + chain: tuple[AnchorRule, ...] + + @classmethod + def of(cls, *anchors: PlacementAnchor) -> PlacementAnchor: + """Concatenate anchors into one fallback chain.""" + rules: list[AnchorRule] = [] + for anchor in anchors: + rules.extend(anchor.chain) + return cls(tuple(rules)) + + def resolve(self, middlewares: Sequence[object]) -> tuple[int, bool]: + """Return (index, used_primary_rule). + + ``used_primary_rule`` is False when the first rule did not match, which + the caller reports as a diagnostic — a silently degraded placement + changes what the extension observes with no signal. + """ + for position, rule in enumerate(self.chain): + index = rule.resolve(middlewares) + if index is not None: + return index, position == 0 + return len(middlewares), False + + +def outer_of(*types: type) -> PlacementAnchor: + return PlacementAnchor((AnchorRule("outer", types),)) + + +def inner_of(*types: type) -> PlacementAnchor: + return PlacementAnchor((AnchorRule("inner", types),)) + + +def inner_of_last(*types: type) -> PlacementAnchor: + return PlacementAnchor((AnchorRule("inner_last", types),)) + + +def inner_of_last_after(*types: type, after: tuple[type, ...]) -> PlacementAnchor: + return PlacementAnchor((AnchorRule("inner_last_after", types, after),)) + + +def outer_of_last(*types: type) -> PlacementAnchor: + return PlacementAnchor((AnchorRule("outer_last", types),)) + + +def outermost() -> PlacementAnchor: + return PlacementAnchor((AnchorRule("start"),)) + + +def innermost() -> PlacementAnchor: + return PlacementAnchor((AnchorRule("end"),)) diff --git a/backend/packages/harness/deerflow/extensions/injection.py b/backend/packages/harness/deerflow/extensions/injection.py new file mode 100644 index 000000000..d3358534e --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/injection.py @@ -0,0 +1,139 @@ +"""Merging extension-contributed middlewares into the host stack.""" + +from __future__ import annotations + +import logging +from collections.abc import Callable, Mapping, Sequence + +from deerflow_extension_api import AgentBuildContext, AgentScope, MiddlewarePlacement, Placement +from langchain.agents.middleware import AgentMiddleware + +from deerflow.extensions.anchors import PlacementAnchor +from deerflow.extensions.isolation import IsolatedMiddleware, graph_safe_middleware_name +from deerflow.extensions.loader import Diagnostic +from deerflow.extensions.registry import LoadedExtensions + +logger = logging.getLogger(__name__) + + +def inject_middlewares( + middlewares: Sequence[object], + anchors: Mapping[Placement, PlacementAnchor], + scope: AgentScope, + ctx: AgentBuildContext, + extensions: LoadedExtensions, + *, + isolation_diagnostic_sink: Callable[[Diagnostic], None] | None = None, +) -> tuple[list[object], dict[int, str], list[Diagnostic]]: + """Insert contributed middlewares at their semantic positions. + + Returns the merged stack, a provenance map from final index to extension + source (core middlewares are absent from it), and construction diagnostics. + Later isolation failures go to ``isolation_diagnostic_sink``; when omitted, + they append to the returned diagnostic list for standalone callers. + """ + result = list(middlewares) + diagnostics: list[Diagnostic] = [] + + if not extensions.has_middleware_contributors: + return result, {}, diagnostics + + collected: list[tuple[str, MiddlewarePlacement]] = [] + for source, contributor in extensions.middleware_contributors: + try: + contributions = tuple(contributor.contribute_middlewares(extensions.app_store, ctx) or ()) + except Exception as exc: + message = f"contribute_middlewares() failed: {exc}" + diagnostics.append(Diagnostic.error(source, message)) + logger.exception("Extension %s: contribute_middlewares() failed", source) + continue + for index, placement in enumerate(contributions): + if not isinstance(placement, MiddlewarePlacement): + message = f"contribution {index} must be a MiddlewarePlacement, got {type(placement).__name__}" + diagnostics.append(Diagnostic.error(source, message)) + logger.error("Extension %s: %s", source, message) + continue + if not isinstance(placement.scope, AgentScope): + message = f"contribution {index} has invalid scope {placement.scope!r}" + diagnostics.append(Diagnostic.error(source, message)) + logger.error("Extension %s: %s", source, message) + continue + if not isinstance(placement.placement, Placement): + message = f"contribution {index} has invalid placement {placement.placement!r}" + diagnostics.append(Diagnostic.error(source, message)) + logger.error("Extension %s: %s", source, message) + continue + if not isinstance(placement.order, int) or isinstance(placement.order, bool): + message = f"contribution {index} has invalid order {placement.order!r}; expected int" + diagnostics.append(Diagnostic.error(source, message)) + logger.error("Extension %s: %s", source, message) + continue + if not isinstance(placement.middleware, AgentMiddleware): + message = f"contribution {index} middleware must be an AgentMiddleware, got {type(placement.middleware).__name__}" + diagnostics.append(Diagnostic.error(source, message)) + logger.error("Extension %s: %s", source, message) + continue + if not (placement.scope & scope): + continue + collected.append((source, placement)) + + if not collected: + return result, {}, diagnostics + + # Sort by declared order, then by registration order, so the outcome is + # reproducible regardless of dict iteration details. + ordered = sorted(enumerate(collected), key=lambda item: (item[1][1].order, item[0])) + + # Insert inner-most positions first: each insertion shifts the indices of + # everything after it, so working from the back keeps earlier anchors valid. + # + # `priority` records each contribution's position in `ordered` (already + # sorted by declared order, then registration order). It breaks ties when + # two contributions resolve to the *same* target index: inserting always + # pushes the previous occupant of that index outward, so to make the + # higher-priority (earlier in `ordered`) contribution end up outermost, it + # must be the *last* one inserted at that index. Sorting by (index, + # priority) descending achieves that: lower-priority items are processed + # — and therefore inserted, and therefore displaced outward — first. + resolved: list[tuple[int, int, str, object]] = [] + for priority, (_, (source, placement)) in enumerate(ordered): + anchor = anchors.get(placement.placement) + if anchor is None: + diagnostics.append(Diagnostic.error(source, f"no anchor configured for placement {placement.placement.name}")) + continue + index, used_primary = anchor.resolve(result) + if not used_primary: + message = f"placement {placement.placement.name} fell back to a secondary anchor (primary anchor middleware is absent from this stack); the observation semantics of this placement may differ from its documented guarantee" + diagnostics.append(Diagnostic.warning(source, message)) + logger.warning("Extension %s: %s", source, message) + resolved.append((index, priority, source, placement.middleware)) + + # LangChain requires names to be unique across the complete stack and uses + # them as trace identities and, for before/after hooks, LangGraph node IDs. + used_names = {getattr(middleware, "name", type(middleware).__name__) for middleware in result} + runtime_diagnostic_sink = isolation_diagnostic_sink if isolation_diagnostic_sink is not None else diagnostics.append + for index, priority, source, middleware in sorted(resolved, key=lambda item: (item[0], item[1]), reverse=True): + try: + inner_name = getattr(middleware, "name", type(middleware).__name__) + base_name = graph_safe_middleware_name(f"extension:{source}:{inner_name}:{priority}") + name = base_name + suffix = 2 + while name in used_names: + name = f"{base_name}_{suffix}" + suffix += 1 + wrapped = IsolatedMiddleware( + middleware, + source, + runtime_diagnostic_sink, + name=name, + ) + except Exception as exc: + message = f"middleware construction failed: {exc}" + diagnostics.append(Diagnostic.error(source, message)) + logger.exception("Extension %s: %s", source, message) + continue + used_names.add(name) + result.insert(index, wrapped) + + provenance = {index: middleware.source for index, middleware in enumerate(result) if isinstance(middleware, IsolatedMiddleware)} + return result, provenance, diagnostics diff --git a/backend/packages/harness/deerflow/extensions/isolation.py b/backend/packages/harness/deerflow/extensions/isolation.py new file mode 100644 index 000000000..5e883158b --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/isolation.py @@ -0,0 +1,364 @@ +"""Isolating extension middleware failures from the user's run. + +Extension middlewares execute inside LangChain's call chain, so an unhandled +exception would abort the user's run. Every contributed middleware is wrapped +so an observation failure degrades to a diagnostic and the call passes through. +The downstream handler is tracked so isolation recovery never adds another +model request or tool side effect: pre-handler extension failures invoke it +once, post-handler failures return its captured result, and handler failures +remain owned by the graph's error policy. + +The wrapper must mirror the inner middleware's full interface, not just the +four wrap-call hooks: LangChain discovers capabilities by inspecting the +wrapper — hook participation via class-level identity checks +(`m.__class__.before_model is not AgentMiddleware.before_model`), tools, +state_schema and transformers via instance attributes. Lifecycle mirroring is +exact in both directions. LangChain deliberately treats each sync/async wrap +pair as one capability and wires both execution paths when either side exists, +so the wrapper supplies a silent pass-through counterpart when the inner +implements only one side; otherwise the base class raises +``NotImplementedError`` before isolation can fail open. + +All first-version contributions are observational, hence fail-open. A future +intercepting (decision-making) contribution would need to fail closed and must +opt out of this wrapper explicitly. +""" + +from __future__ import annotations + +import logging +import re +import threading +from collections.abc import Awaitable, Callable +from types import TracebackType +from typing import Any + +from langchain.agents.middleware import AgentMiddleware +from langgraph.errors import GraphBubbleUp + +from deerflow.extensions.loader import Diagnostic + +logger = logging.getLogger(__name__) + +_UNSAFE_GRAPH_NAME = re.compile(r"[^A-Za-z0-9_.-]+") + + +def graph_safe_middleware_name(value: str) -> str: + """Normalize a middleware identity for LangGraph node names.""" + return _UNSAFE_GRAPH_NAME.sub("_", value) + + +_WRAP_HOOKS = ("wrap_model_call", "awrap_model_call", "wrap_tool_call", "awrap_tool_call") +_WRAP_HOOK_PAIRS = ( + ("wrap_model_call", "awrap_model_call"), + ("wrap_tool_call", "awrap_tool_call"), +) +_LIFECYCLE_HOOKS = ( + "before_agent", + "abefore_agent", + "before_model", + "abefore_model", + "after_model", + "aafter_model", + "after_agent", + "aafter_agent", +) + + +def _implemented_hooks(inner: AgentMiddleware) -> frozenset[str]: + """The hooks ``inner`` actually overrides, by LangChain's own class-level + identity check — instance-level attributes are invisible to the factory, + so they are invisible here too.""" + return frozenset(hook for hook in (*_WRAP_HOOKS, *_LIFECYCLE_HOOKS) if getattr(type(inner), hook, None) is not getattr(AgentMiddleware, hook, None)) + + +def _make_sync_wrap_delegate(hook: str): + def delegate(self: IsolatedMiddleware, request: Any, handler: Callable[[Any], Any]) -> Any: + return self._invoke_sync(hook, getattr(self._inner, hook), request, handler) + + return delegate + + +def _make_async_wrap_delegate(hook: str): + async def delegate(self: IsolatedMiddleware, request: Any, handler: Callable[[Any], Awaitable[Any]]) -> Any: + return await self._invoke_async(hook, getattr(self._inner, hook), request, handler) + + return delegate + + +def _make_sync_wrap_passthrough(): + def delegate(self: IsolatedMiddleware, request: Any, handler: Callable[[Any], Any]) -> Any: + return handler(request) + + return delegate + + +def _make_async_wrap_passthrough(): + async def delegate(self: IsolatedMiddleware, request: Any, handler: Callable[[Any], Awaitable[Any]]) -> Any: + return await handler(request) + + return delegate + + +def _make_sync_lifecycle_delegate(hook: str): + def delegate(self: IsolatedMiddleware, state: Any, runtime: Any) -> Any: + return self._invoke_lifecycle_sync(hook, state, runtime) + + return delegate + + +def _make_async_lifecycle_delegate(hook: str): + async def delegate(self: IsolatedMiddleware, state: Any, runtime: Any) -> Any: + return await self._invoke_lifecycle_async(hook, state, runtime) + + return delegate + + +# Async variants are named explicitly: startswith("a") would also catch the +# sync after_model/after_agent. +_ASYNC_HOOKS = frozenset(hook for hook in (*_WRAP_HOOKS, *_LIFECYCLE_HOOKS) if hook[1:].startswith(("wrap", "before", "after"))) + + +def _delegate_for(hook: str): + if hook in _WRAP_HOOKS: + return _make_async_wrap_delegate(hook) if hook in _ASYNC_HOOKS else _make_sync_wrap_delegate(hook) + return _make_async_lifecycle_delegate(hook) if hook in _ASYNC_HOOKS else _make_sync_lifecycle_delegate(hook) + + +_subclass_cache: dict[frozenset[str], type[IsolatedMiddleware]] = {} +_subclass_cache_lock = threading.Lock() + + +def _wrapper_subclass(hooks: frozenset[str]) -> type[IsolatedMiddleware]: + """A cached IsolatedMiddleware subclass defining ``hooks`` and required + wrap-hook pass-through counterparts. + + Per hook set, not per middleware: every inner middleware with the same + implemented-hook combination shares one subclass. + """ + with _subclass_cache_lock: + subclass = _subclass_cache.get(hooks) + if subclass is None: + namespace = {hook: _delegate_for(hook) for hook in hooks} + for sync_hook, async_hook in _WRAP_HOOK_PAIRS: + if sync_hook in hooks and async_hook not in hooks: + namespace[async_hook] = _make_async_wrap_passthrough() + elif async_hook in hooks and sync_hook not in hooks: + namespace[sync_hook] = _make_sync_wrap_passthrough() + subclass = type(IsolatedMiddleware.__name__, (IsolatedMiddleware,), namespace) + _subclass_cache[hooks] = subclass + return subclass + + +class IsolatedMiddleware(AgentMiddleware): + """Wrap one extension middleware so its failures cannot break the run. + + Instantiation returns a cached subclass that defines exactly the hooks the + inner middleware implements, so LangChain's class-level capability checks + see the same interface on the wrapper as on the inner middleware itself. + """ + + def __new__(cls, inner: AgentMiddleware, source: str, on_error: Callable[[Diagnostic], None], *, name: str | None = None): + if cls is IsolatedMiddleware: + cls = _wrapper_subclass(_implemented_hooks(inner)) + return super().__new__(cls) + + def __init__( + self, + inner: AgentMiddleware, + source: str, + on_error: Callable[[Diagnostic], None], + *, + name: str | None = None, + ) -> None: + super().__init__() + self._inner = inner + self._source = source + self._on_error = on_error + if name is None: + inner_name = getattr(inner, "name", type(inner).__name__) + name = f"extension:{source}:{inner_name}" + self._name = graph_safe_middleware_name(name) + # Mirror the declared-contribution attributes LangChain reads off the + # middleware instance (factory.py: m.tools, m.state_schema, + # m.transformers). state_schema is a class attribute on the base but + # must be per-instance here: cached subclasses are shared across + # middlewares whose schemas differ. + self.tools = getattr(inner, "tools", []) + self.transformers = getattr(inner, "transformers", ()) + self.state_schema = getattr(inner, "state_schema", AgentMiddleware.state_schema) + + @property + def name(self) -> str: + """Stable graph and trace identity for this isolated contribution.""" + return self._name + + @property + def inner(self) -> AgentMiddleware: + """The wrapped middleware. Used by ordering checks and tests.""" + return self._inner + + @property + def source(self) -> str: + """Extension this middleware came from. Read by the provenance map.""" + return self._source + + def _report(self, hook: str, exc: Exception) -> None: + message = f"{type(self._inner).__name__}.{hook} failed and was skipped: {exc}" + logger.exception("Extension %s: %s", self._source, message) + try: + self._on_error(Diagnostic.error(self._source, message)) + except Exception: # pragma: no cover - reporting must never raise + logger.exception("Extension %s: diagnostic reporting failed", self._source) + + def _invoke_sync( + self, + hook: str, + inner_hook: Callable[[Any, Callable[[Any], Any]], Any], + request: Any, + handler: Callable[[Any], Any], + ) -> Any: + handler_called = False + handler_succeeded = False + handler_result: Any = None + handler_error: BaseException | None = None + handler_error_traceback: TracebackType | None = None + duplicate_call_error: RuntimeError | None = None + + def tracked_handler(inner_request: Any) -> Any: + nonlocal handler_called, duplicate_call_error + nonlocal handler_error, handler_error_traceback + nonlocal handler_result, handler_succeeded + if handler_called: + duplicate_call_error = RuntimeError(f"{type(self._inner).__name__}.{hook} called the downstream handler more than once") + raise duplicate_call_error + handler_called = True + handler_error = None + handler_error_traceback = None + handler_succeeded = False + try: + # The first contract slice is observational: a contributed + # wrapper may inspect the request but cannot substitute a new + # one after the host's policy/authorization layers have run. + handler_result = handler(request) + except BaseException as exc: + handler_error = exc + handler_error_traceback = exc.__traceback__ + raise + else: + handler_succeeded = True + return handler_result + + try: + inner_hook(request, tracked_handler) + if handler_error is not None: + raise handler_error.with_traceback(handler_error_traceback) + if duplicate_call_error is not None: + raise duplicate_call_error + if not handler_called: + raise RuntimeError(f"{type(self._inner).__name__}.{hook} did not call the downstream handler") + return handler_result + except GraphBubbleUp as exc: + if handler_error is not None: + if handler_error is exc: + raise + raise handler_error.with_traceback(handler_error_traceback) from None + if handler_succeeded: + self._report(hook, duplicate_call_error or exc) + return handler_result + raise + except Exception as exc: + if handler_error is not None: + if handler_error is exc: + raise + raise handler_error.with_traceback(handler_error_traceback) from None + self._report(hook, exc) + if handler_succeeded: + return handler_result + return handler(request) + + async def _invoke_async( + self, + hook: str, + inner_hook: Callable[ + [Any, Callable[[Any], Awaitable[Any]]], + Awaitable[Any], + ], + request: Any, + handler: Callable[[Any], Awaitable[Any]], + ) -> Any: + handler_called = False + handler_succeeded = False + handler_result: Any = None + handler_error: BaseException | None = None + handler_error_traceback: TracebackType | None = None + duplicate_call_error: RuntimeError | None = None + + async def tracked_handler(inner_request: Any) -> Any: + nonlocal handler_called, duplicate_call_error + nonlocal handler_error, handler_error_traceback + nonlocal handler_result, handler_succeeded + if handler_called: + duplicate_call_error = RuntimeError(f"{type(self._inner).__name__}.{hook} called the downstream handler more than once") + raise duplicate_call_error + handler_called = True + handler_error = None + handler_error_traceback = None + handler_succeeded = False + try: + handler_result = await handler(request) + except BaseException as exc: + handler_error = exc + handler_error_traceback = exc.__traceback__ + raise + else: + handler_succeeded = True + return handler_result + + try: + await inner_hook(request, tracked_handler) + if handler_error is not None: + raise handler_error.with_traceback(handler_error_traceback) + if duplicate_call_error is not None: + raise duplicate_call_error + if not handler_called: + raise RuntimeError(f"{type(self._inner).__name__}.{hook} did not call the downstream handler") + return handler_result + except GraphBubbleUp as exc: + if handler_error is not None: + if handler_error is exc: + raise + raise handler_error.with_traceback(handler_error_traceback) from None + if handler_succeeded: + self._report(hook, duplicate_call_error or exc) + return handler_result + raise + except Exception as exc: + if handler_error is not None: + if handler_error is exc: + raise + raise handler_error.with_traceback(handler_error_traceback) from None + self._report(hook, exc) + if handler_succeeded: + return handler_result + return await handler(request) + + def _invoke_lifecycle_sync(self, hook: str, state: Any, runtime: Any) -> Any: + """Lifecycle hooks have no handler to fall through to: the fail-open + degradation for a failed observation is applying no state update.""" + try: + return getattr(self._inner, hook)(state, runtime) + except GraphBubbleUp: + raise + except Exception as exc: + self._report(hook, exc) + return None + + async def _invoke_lifecycle_async(self, hook: str, state: Any, runtime: Any) -> Any: + try: + return await getattr(self._inner, hook)(state, runtime) + except GraphBubbleUp: + raise + except Exception as exc: + self._report(hook, exc) + return None diff --git a/backend/packages/harness/deerflow/extensions/loader.py b/backend/packages/harness/deerflow/extensions/loader.py new file mode 100644 index 000000000..6ddf5745b --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/loader.py @@ -0,0 +1,225 @@ +"""Config-driven extension loading. + +Entry points are named as `module.path:install`, resolved through the same +`resolve_variable` helper the guardrails provider already uses. Load order is +the config list order — explicit and reproducible, which matters because the +middleware stack is position-sensitive. +""" + +from __future__ import annotations + +import logging +from collections.abc import Mapping, Sequence +from dataclasses import dataclass +from typing import Any, Literal + +from deerflow_extension_api import API_VERSION +from pydantic import BaseModel, ConfigDict, Field + +from deerflow.extensions.registry import ExtensionRegistry, LoadedExtensions +from deerflow.reflection import resolve_variable + +logger = logging.getLogger(__name__) + +DiagnosticLevel = Literal["debug", "info", "warning", "error"] + + +class ExtensionSpec(BaseModel): + """One entry of the `plugins:` list in config.yaml.""" + + model_config = ConfigDict(extra="forbid") + + use: str = Field(description="Entry point path, e.g. 'my_extension:install'") + config: dict[str, Any] = Field( + default_factory=dict, + description="Extension-private configuration, passed to install() verbatim", + ) + required: bool = Field( + default=False, + description="When true, a load failure aborts startup instead of being skipped", + ) + + +@dataclass(frozen=True) +class Diagnostic: + """A load- or run-time problem attributed to a specific extension. + + The repository has no structured diagnostics channel today; this is a + deliberately minimal one whose only job is keeping failures attributable. + """ + + level: DiagnosticLevel + source: str + message: str + + @classmethod + def error(cls, source: str, message: str) -> Diagnostic: + return cls("error", source, message) + + @classmethod + def warning(cls, source: str, message: str) -> Diagnostic: + return cls("warning", source, message) + + @classmethod + def info(cls, source: str, message: str) -> Diagnostic: + return cls("info", source, message) + + @classmethod + def debug(cls, source: str, message: str) -> Diagnostic: + return cls("debug", source, message) + + +class ExtensionLoadError(RuntimeError): + """Raised when an extension marked `required: true` fails to load.""" + + +def _parse_version(version: object) -> tuple[int, ...] | None: + if not isinstance(version, str): + return None + try: + return tuple(int(part) for part in str.split(version, ".")) + except ValueError: + return None + + +def _compatible(declared: str, current: str) -> bool: + """One-directional, with the semver window for the contract's life stage. + + Pre-1.0 the contract surface is observational only and minors may break, + so the window is same major.minor with patches additive: host >= declared. + From 1.0 on contracts only grow within a major, so a newer host stays + compatible with older extensions while an extension written against a + newer minor is refused — it would reach for contract additions the host + does not implement. Unparseable versions are refused, not waved through.""" + declared_parts = _parse_version(declared) + current_parts = _parse_version(current) + if not declared_parts or not current_parts: + return False + width = max(len(declared_parts), len(current_parts), 2) + declared_padded = declared_parts + (0,) * (width - len(declared_parts)) + current_padded = current_parts + (0,) * (width - len(current_parts)) + if declared_padded[0] != current_padded[0]: + return False + if declared_padded[0] == 0 and declared_padded[1] != current_padded[1]: + return False + return current_padded >= declared_padded + + +def _range_for(declared: str) -> str: + """The pip window matching ``_compatible``'s rules, for the actionable + refusal message. Falls back to an exact request when the declared version + is unparseable — the message must survive the version that caused it.""" + parts = _parse_version(declared) + if not parts: + return f"=={declared}" + if parts[0] == 0: + minor = parts[1] if len(parts) > 1 else 0 + return f">={declared},<0.{minor + 1}" + return f">={declared},<{parts[0] + 1}.0" + + +def load_extensions(specs: Sequence[ExtensionSpec]) -> tuple[LoadedExtensions, list[Diagnostic]]: + """Resolve and install every configured extension. + + Fail-open by default: a broken extension is skipped with a diagnostic so + the Gateway still starts. `required: true` flips that to fail-closed for + extensions whose absence changes behaviour rather than just observability. + """ + registry = ExtensionRegistry() + diagnostics: list[Diagnostic] = [] + loaded_sources: list[str] = [] + + for spec in specs: + try: + install = resolve_variable(spec.use) + except Exception as exc: + message = f"could not resolve extension entry point: {exc}" + diagnostics.append(Diagnostic.error(spec.use, message)) + logger.error("Extension %s: %s", spec.use, message) + if spec.required: + raise ExtensionLoadError(f"required extension {spec.use} failed to load") from exc + continue + + if not callable(install): + message = f"extension entry point is not callable: {type(install).__name__}" + diagnostics.append(Diagnostic.error(spec.use, message)) + logger.error("Extension %s: %s", spec.use, message) + if spec.required: + raise ExtensionLoadError(f"required extension {spec.use} is not callable") + continue + + try: + declared = getattr(install, "__deerflow_api__", None) + except Exception as exc: + message = f"could not inspect extension-api version marker: {type(exc).__name__}" + diagnostics.append(Diagnostic.error(spec.use, message)) + logger.error("Extension %s: %s", spec.use, message) + if spec.required: + raise ExtensionLoadError(f"required extension {spec.use} could not inspect api marker") from exc + continue + if declared is not None and _parse_version(declared) is None: + message = f"extension declares invalid extension-api version marker of type {type(declared).__name__}; expected a dotted numeric string such as '0.1'" + diagnostics.append(Diagnostic.error(spec.use, message)) + logger.error("Extension %s: %s", spec.use, message) + if spec.required: + raise ExtensionLoadError(f"required extension {spec.use} declares invalid api marker") + continue + if declared is not None: + # ``isinstance(..., str)`` also accepts subclasses whose + # ``__str__``/``__format__`` methods can execute plugin code while + # we build an incompatibility diagnostic. Normalize with the base + # implementation before compatibility checks and rendering. + declared = str.__str__(declared) + if declared is not None and not _compatible(declared, API_VERSION): + message = f"extension requires extension-api {declared}, host provides {API_VERSION}. Install a matching version: pip install 'deerflow-extension-api{_range_for(declared)}'" + diagnostics.append(Diagnostic.error(spec.use, message)) + logger.error("Extension %s: %s", spec.use, message) + if spec.required: + raise ExtensionLoadError(f"required extension {spec.use} declares incompatible api {declared}") + continue + + # Positional rollback, not registry.discard(spec.use): two specs may + # legitimately share the same `use` with different config, and + # discard-by-source would also erase an earlier, successfully + # installed instance that happens to share this spec's `use`. + mark = registry.mark() + try: + with registry.attributed_to(spec.use): + install(registry, _frozen_config(spec.config)) + except Exception as exc: + registry.rollback_to(mark) + message = f"install() failed: {exc}" + diagnostics.append(Diagnostic.error(spec.use, message)) + logger.exception("Extension %s: install() failed", spec.use) + if spec.required: + raise ExtensionLoadError(f"required extension {spec.use} failed to install") from exc + continue + + loaded_sources.append(spec.use) + + # Loading third-party code is exactly the event an operator needs positive + # confirmation of, and every other branch here is failure-only — so without + # this line a fully successful load is indistinguishable from a `plugins:` + # block the host never read. The x/y count names the difference between + # "all loaded" and "some were skipped" without repeating the per-failure + # errors already logged above. + if specs: + logger.info("Extensions loaded: %d/%d (%s)", len(loaded_sources), len(specs), ", ".join(loaded_sources) or "none") + else: + # Debug, not info: no configured plugins is the default state for almost + # every deployment, and an unconditional line would be pure boot noise. + logger.debug("No extensions configured") + + return registry.build(), diagnostics + + +def _frozen_config(config: dict[str, Any]) -> Mapping[str, Any]: + """Hand extensions a shallow copy of their config block. + + This is a shallow copy: it stops an extension from reassigning + top-level keys on another extension's (or the caller's) config dict, but + nested structures (lists, dicts) are still shared by reference and can be + mutated in place. Use plain, top-level config values if this guarantee + matters to you. + """ + return dict(config) diff --git a/backend/packages/harness/deerflow/extensions/ordering.py b/backend/packages/harness/deerflow/extensions/ordering.py new file mode 100644 index 000000000..bfcfbe03e --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/ordering.py @@ -0,0 +1,88 @@ +"""Declarative ordering invariants for the middleware stack. + +Replaces hand-written index comparisons. Extension-contributed middlewares are +merged before validation runs, so a contribution cannot slip past an invariant, +and the failure names the extension responsible. + +A broken invariant is the one hard failure in this system: unlike a missing +observation, it produces wrong behaviour without an error. +""" + +from __future__ import annotations + +from collections.abc import Mapping, Sequence +from dataclasses import dataclass +from functools import cache + +from deerflow.extensions.isolation import IsolatedMiddleware + + +@dataclass(frozen=True) +class OrderingConstraint: + outer: type + inner: type + reason: str + + +def _indices_of(middlewares: Sequence[object], target: type) -> list[int]: + indices: list[int] = [] + for index, middleware in enumerate(middlewares): + candidate = middleware.inner if isinstance(middleware, IsolatedMiddleware) else middleware + if isinstance(candidate, target): + indices.append(index) + return indices + + +def assert_ordering( + middlewares: Sequence[object], + provenance: Mapping[int, str], + constraints: Sequence[OrderingConstraint] | None = None, +) -> None: + """Raise when a constraint is violated. No-op when both sides are absent.""" + for constraint in constraints if constraints is not None else core_ordering_constraints(): + outer_indices = _indices_of(middlewares, constraint.outer) + inner_indices = _indices_of(middlewares, constraint.inner) + if not outer_indices or not inner_indices: + continue + if max(outer_indices) < min(inner_indices): + continue + violating_indices = [index for index in outer_indices if index >= min(inner_indices)] + [index for index in inner_indices if index <= max(outer_indices)] + culprits = sorted({source for index in violating_indices if (source := provenance.get(index)) is not None}) + blame = ", ".join(culprits) if culprits else "core middleware order" + raise RuntimeError( + f"Middleware ordering constraint violated: {constraint.outer.__name__} must be outer " + f"(lower index) of every {constraint.inner.__name__}, but found outer indices " + f"{outer_indices} vs inner indices {inner_indices}. Reason: {constraint.reason}. " + f"Contributed by: {blame}." + ) + + +@cache +def core_ordering_constraints() -> tuple[OrderingConstraint, ...]: + """The host's ordering invariants, resolved on first use. + + Deferred deliberately, and the deferral is about dependency *direction*, + not just cycles: ``extensions/`` is the layer the middleware layer calls + into, so importing ``agents.middlewares`` at module scope here would point + the dependency backwards and close a cycle the moment any middleware + imports something under ``extensions/`` at module level. Resolution instead + happens at ``assert_ordering`` time, which already runs inside the + middleware builder — a forward reference within one layer. + + Returns a plain tuple. The predecessor deferred by way of a ``tuple`` + subclass overriding only ``__iter__``; because a tuple cannot populate its + own storage after construction, every operation reading that storage + (``len``, ``bool``, ``in``, indexing, slicing, ``reversed``, ``==``) + reported an empty sequence while iteration yielded the real constraints. + Deferring the call instead of faking the value keeps one answer. + """ + from deerflow.agents.middlewares.tool_error_handling_middleware import ToolErrorHandlingMiddleware + from deerflow.agents.middlewares.tool_progress_middleware import ToolProgressMiddleware + + return ( + OrderingConstraint( + outer=ToolProgressMiddleware, + inner=ToolErrorHandlingMiddleware, + reason=("ToolProgressMiddleware reads deerflow_tool_meta in _update_state_from_result, so its wrap_tool_call chain must enclose the ToolErrorHandlingMiddleware step that stamps it"), + ), + ) diff --git a/backend/packages/harness/deerflow/extensions/policy.py b/backend/packages/harness/deerflow/extensions/policy.py new file mode 100644 index 000000000..f204fd950 --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/policy.py @@ -0,0 +1,39 @@ +"""Extension-facing projection of host policy. + +This module is intentionally independent of Gateway router and service +plumbing: lead and subagent builders need the projection even when the +Gateway-specific contribution points are not installed. +""" + +from __future__ import annotations + +from typing import Any + +from deerflow_extension_api import HostPolicySnapshot + +_UNSET = object() + + +def project_host_policy( + app_config: Any, + *, + token_budget_config: Any | None = None, + max_subagents_per_run: int | None | object = _UNSET, +) -> HostPolicySnapshot: + """Project the host's enforced limits into the public extension contract.""" + token_budget = token_budget_config if token_budget_config is not None else getattr(app_config, "token_budget", None) + token_budget_enabled = bool(getattr(token_budget, "enabled", False)) + if max_subagents_per_run is _UNSET: + subagents = getattr(app_config, "subagents", None) + effective_max_subagents = getattr(subagents, "max_total_per_run", None) + else: + effective_max_subagents = max_subagents_per_run + return HostPolicySnapshot( + token_budget_enabled=token_budget_enabled, + max_input_tokens=getattr(token_budget, "max_input_tokens", None) if token_budget_enabled else None, + max_output_tokens=getattr(token_budget, "max_output_tokens", None) if token_budget_enabled else None, + max_total_tokens=getattr(token_budget, "max_tokens", None) if token_budget_enabled else None, + budget_warn_fraction=getattr(token_budget, "warn_threshold", None) if token_budget_enabled else None, + budget_hard_fraction=getattr(token_budget, "hard_stop_threshold", None) if token_budget_enabled else None, + max_subagents_per_run=effective_max_subagents if isinstance(effective_max_subagents, int) else None, + ) diff --git a/backend/packages/harness/deerflow/extensions/registry.py b/backend/packages/harness/deerflow/extensions/registry.py new file mode 100644 index 000000000..af7179cd5 --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/registry.py @@ -0,0 +1,107 @@ +"""Registration-phase registry and its immutable runtime product. + +Extensions only ever see the write-only public ``ExtensionRegistry`` contract. +The concrete host type additionally owns attribution, rollback, and immutable +runtime projection. +""" + +from __future__ import annotations + +from collections.abc import Iterator +from contextlib import contextmanager +from dataclasses import dataclass +from typing import Any + +from deerflow_extension_api import ExtensionData, MiddlewareContributor +from deerflow_extension_api import ExtensionRegistry as ExtensionRegistryContract + +_Entry = tuple[str, Any] + + +@dataclass(frozen=True) +class LoadedExtensions: + """Immutable view consumed at runtime. + + Every entry carries its source string so diagnostics, provenance and + ordering errors can name the extension responsible. + """ + + app_store: ExtensionData + middleware_contributors: tuple[tuple[str, MiddlewareContributor], ...] = () + + # Precomputed attributes, not methods: hook sites read one attribute to + # short-circuit, so the zero-extension path constructs nothing. + has_middleware_contributors: bool = False + needs_task_store: bool = False + + +class ExtensionRegistry(ExtensionRegistryContract): + """Mutable, registration-phase only. + + Subclasses the public contract Protocol so the host implementation is + type-checked against what extensions annotate; the host-only machinery + below (attribution, discard, mark/rollback_to, build) stays out of the + contract on purpose. + """ + + def __init__(self) -> None: + self._middlewares: list[_Entry] = [] + self._current_source: str | None = None + + @contextmanager + def attributed_to(self, source: str) -> Iterator[None]: + """Attribute everything registered inside the block to ``source``.""" + previous = self._current_source + self._current_source = source + try: + yield + finally: + self._current_source = previous + + def _source(self) -> str: + if self._current_source is None: + raise RuntimeError("registration must happen inside ExtensionRegistry.attributed_to(...)") + return self._current_source + + def middlewares(self, contributor: MiddlewareContributor) -> None: + self._middlewares.append((self._source(), contributor)) + + def discard(self, source: str) -> None: + """Remove every entry registered by ``source``. + + Called when install() raises partway through. A half-registered + extension is more dangerous than an absent one because the data it + produces looks complete. + + Note: this matches by source string, so it is unsafe when two specs + share the same ``use`` with different config — it would remove a + different, successfully-installed instance's entries too. Callers + that process one install() at a time should prefer + ``mark()``/``rollback_to()`` instead. + """ + self._middlewares[:] = [entry for entry in self._middlewares if entry[0] != source] + + def mark(self) -> int: + """Snapshot bucket lengths so one install() can be undone positionally.""" + return len(self._middlewares) + + def rollback_to(self, mark: int) -> None: + """Undo every registration made since ``mark``. + + Positional rather than source-keyed: two specs may legitimately share + a ``use`` string with different config, and deleting by source would + take the other instance's successful registrations with it. + """ + del self._middlewares[mark:] + + def build(self) -> LoadedExtensions: + return LoadedExtensions( + app_store=ExtensionData("app"), + middleware_contributors=tuple(self._middlewares), + has_middleware_contributors=bool(self._middlewares), + needs_task_store=bool(self._middlewares), + ) + + +#: Shared empty instance for hosts that load no extensions. +EMPTY_EXTENSIONS = ExtensionRegistry().build() diff --git a/backend/packages/harness/deerflow/extensions/stack.py b/backend/packages/harness/deerflow/extensions/stack.py new file mode 100644 index 000000000..0e9e7e070 --- /dev/null +++ b/backend/packages/harness/deerflow/extensions/stack.py @@ -0,0 +1,191 @@ +"""The anchor table and the single composition entry point. + +This is where DeerFlow's stack shape is encoded. Two structural facts drive it: + +* The stack is built at two nested points — `build_lead_runtime_middlewares()` + produces the base, then `build_middlewares()` appends ~18 lead-specific + middlewares that are all *inner* of it. MODEL_PHYSICAL lands in the second + group, so extension injection must happen after the final list is assembled, + never inside the base builder. +* First item in the list is the outermost wrapper (LangChain composition rule). +""" + +from __future__ import annotations + +from collections.abc import Sequence + +from deerflow_extension_api import AgentBuildContext, AgentScope, Placement + +from deerflow.extensions.anchors import ( + PlacementAnchor, + inner_of_last, + inner_of_last_after, + innermost, + outer_of, + outer_of_last, + outermost, +) +from deerflow.extensions.injection import inject_middlewares +from deerflow.extensions.ordering import assert_ordering +from deerflow.extensions.registry import LoadedExtensions + + +def _anchors() -> dict[Placement, PlacementAnchor]: + from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware + from deerflow.agents.middlewares.llm_error_handling_middleware import LLMErrorHandlingMiddleware + from deerflow.agents.middlewares.safety_finish_reason_middleware import SafetyFinishReasonMiddleware + from deerflow.agents.middlewares.terminal_response_middleware import TerminalResponseMiddleware + + return { + # Outer of the retry loop, so one logical decision stays one event even + # when LLMErrorHandlingMiddleware retries underneath. + Placement.MODEL_LOGICAL: outer_of(LLMErrorHandlingMiddleware), + # Inner of every lead-agent request transform. Deliberately NOT + # innermost(): ClarificationMiddleware sits inner of this point today, + # and moving the anchor past it would change what "the final request" + # means. + Placement.MODEL_PHYSICAL: PlacementAnchor.of( + inner_of_last_after( + SafetyFinishReasonMiddleware, + after=(TerminalResponseMiddleware,), + ), + inner_of_last(TerminalResponseMiddleware), + outer_of_last(ClarificationMiddleware), + innermost(), + ), + Placement.TOOL_VISIBLE: outermost(), + # As close to the tool callable as the chain allows. Deliberately NOT + # inner_of(ToolErrorHandlingMiddleware): SkillToolPolicyMiddleware and + # ClarificationMiddleware are appended later and also wrap tool calls, + # so anchoring there left two wrappers inner of "raw" and the placement + # silently stopped meaning what it says. + # + # ClarificationMiddleware remains the one carve-out, the same shape as + # MODEL_PHYSICAL's above: it must stay last (it short-circuits the tool + # loop with Command(goto=END)), and it only ever intercepts + # ask_clarification — it does not transform the result of any tool that + # actually executes, so TOOL_RAW still sees raw results. + Placement.TOOL_RAW: PlacementAnchor.of( + outer_of_last(ClarificationMiddleware), + innermost(), + ), + Placement.STANDARD: PlacementAnchor.of( + outer_of(LLMErrorHandlingMiddleware), + innermost(), + ), + } + + +class _AnchorTable(dict): + """Resolve the table lazily so importing this module stays cheap and + free of middleware import cycles.""" + + _loaded = False + + def _ensure(self) -> None: + if not _AnchorTable._loaded: + self.update(_anchors()) + _AnchorTable._loaded = True + + def __getitem__(self, key): + self._ensure() + return dict.__getitem__(self, key) + + def get(self, key, default=None): + self._ensure() + return dict.get(self, key, default) + + def __iter__(self): + self._ensure() + return dict.__iter__(self) + + def __len__(self): + self._ensure() + return dict.__len__(self) + + def snapshot(self) -> dict[Placement, PlacementAnchor]: + """Return a populated plain-dict copy. + + CPython's ``dict(subclass)`` fast path can copy the underlying storage + without calling this class's lazy ``__iter__`` or ``__len__`` hooks. + Callers that need a copy must therefore force resolution explicitly. + """ + self._ensure() + return dict(self) + + +PLACEMENT_ANCHORS = _AnchorTable() + + +def _placement_anchors_for_scope(scope: AgentScope) -> dict[Placement, PlacementAnchor]: + if scope != AgentScope.SUBAGENT: + return PLACEMENT_ANCHORS + + from deerflow.agents.middlewares.system_message_coalescing_middleware import SystemMessageCoalescingMiddleware + + anchors = PLACEMENT_ANCHORS.snapshot() + anchors[Placement.MODEL_PHYSICAL] = PlacementAnchor.of( + inner_of_last(SystemMessageCoalescingMiddleware), + PLACEMENT_ANCHORS[Placement.MODEL_PHYSICAL], + ) + return anchors + + +def compose_with_extensions( + middlewares: Sequence[object], + scope: AgentScope, + ctx: AgentBuildContext | None, + extensions: LoadedExtensions | None = None, +) -> list[object]: + """Merge extension contributions into a fully-assembled stack and validate. + + Call this once, at the end of the outermost builder. Calling it inside the + base builder would place MODEL_PHYSICAL contributions above the ~18 + lead-specific middlewares appended afterwards. + """ + from deerflow.extensions import get_agent_build_extensions, record_runtime_diagnostic + + resolved = extensions if extensions is not None else get_agent_build_extensions() + + if not resolved.has_middleware_contributors: + assert_ordering(middlewares, {}) + return middlewares if isinstance(middlewares, list) else list(middlewares) + + if ctx is None: + raise ValueError("AgentBuildContext is required when middleware extensions are loaded") + + result = list(middlewares) + + result, provenance, diagnostics = inject_middlewares( + result, + _placement_anchors_for_scope(scope), + scope, + ctx, + resolved, + isolation_diagnostic_sink=record_runtime_diagnostic, + ) + _record_diagnostics(diagnostics) + assert_ordering(result, provenance) + return result + + +def _record_diagnostics(diagnostics) -> None: + """Diagnostics raised while building a stack are logged by their producers; + this hook exists so the Gateway can also surface them on app.state.""" + from deerflow.extensions import record_runtime_diagnostics + + record_runtime_diagnostics(diagnostics) + + +def middleware_implements(middleware: object, hook_name: str) -> bool: + """Whether ``middleware`` actually overrides ``hook_name``. + + Placement guarantees are per hook chain, not per list index: a middleware's + position only means something on the chains it participates in. This is how + the guarantee tests tell participation from mere presence. + """ + from langchain.agents.middleware import AgentMiddleware + + own = getattr(type(middleware), hook_name, None) + base = getattr(AgentMiddleware, hook_name, None) + return own is not None and own is not base diff --git a/backend/packages/harness/deerflow/runtime/runs/worker.py b/backend/packages/harness/deerflow/runtime/runs/worker.py index 8d2a6f1b5..b1b0c1e07 100644 --- a/backend/packages/harness/deerflow/runtime/runs/worker.py +++ b/backend/packages/harness/deerflow/runtime/runs/worker.py @@ -349,6 +349,8 @@ def _build_runtime_context( run_id: str, caller_context: Any | None, app_config: AppConfig | None = None, + task_store: Any | None = None, + extensions: Any | None = None, ) -> dict[str, Any]: """Build the dict that becomes ``ToolRuntime.context`` for the run. @@ -370,6 +372,21 @@ def _build_runtime_context( runtime_ctx.setdefault(key, value) if app_config is not None: runtime_ctx["app_config"] = app_config + if task_store is not None: + from deerflow_extension_api import EXTENSION_TASK_STORE_KEY + + runtime_ctx[EXTENSION_TASK_STORE_KEY] = task_store + # Publish the run's extension snapshot so work dispatched during graph + # execution (task delegation) binds the same generation the lead agent was + # built with, instead of re-reading a singleton that may have been replaced + # mid-run. Written after the caller merge and popped when absent, because a + # caller-supplied value for this host-internal key is never authoritative. + from deerflow.extensions import EXTENSION_SNAPSHOT_CONTEXT_KEY + + if extensions is not None: + runtime_ctx[EXTENSION_SNAPSHOT_CONTEXT_KEY] = extensions + else: + runtime_ctx.pop(EXTENSION_SNAPSHOT_CONTEXT_KEY, None) return runtime_ctx @@ -388,6 +405,7 @@ class RunContext: run_events_config: Any | None = field(default=None) thread_store: Any | None = field(default=None) app_config: AppConfig | None = field(default=None) + extensions: Any | None = field(default=None) checkpoint_channel_mode: CheckpointChannelMode = "full" # Delta snapshot cadence frozen at startup; ``None`` means "not frozen in # this process" (embedded/tests) and resolves to the config default. @@ -519,6 +537,13 @@ async def run_agent( run_id = record.run_id thread_id = record.thread_id + + from deerflow_extension_api import ExtensionData + + from deerflow.extensions import get_loaded_extensions + + extensions = ctx.extensions if ctx.extensions is not None else get_loaded_extensions() + task_store: ExtensionData | None = None pre_run_checkpoint_id: str | None = None pre_run_workspace_snapshot: WorkspaceSnapshot | None = None workspace_changes_user_id: str | None = None @@ -631,6 +656,9 @@ async def run_agent( return started = True + if extensions.needs_task_store: + task_store = ExtensionData(run_id) + if not record.ownership_lost and thread_store is not None: try: await thread_store.update_status(thread_id, "running") @@ -698,7 +726,14 @@ async def run_agent( # access thread-level data. langgraph-cli does this automatically; we must do it # manually here because we drive the graph through ``agent.astream(config=...)`` # without passing the official ``context=`` parameter. - runtime_ctx = _build_runtime_context(thread_id, run_id, config.get("context"), ctx.app_config) + runtime_ctx = _build_runtime_context( + thread_id, + run_id, + config.get("context"), + ctx.app_config, + task_store, + extensions, + ) incoming_metadata = config.get("metadata") if isinstance(config.get("metadata"), dict) else {} deerflow_trace_id = resolve_deerflow_trace_id(incoming_metadata.get(DEERFLOW_TRACE_METADATA_KEY)) if deerflow_trace_id: @@ -750,10 +785,13 @@ async def run_agent( continuation_config["configurable"] = configurable return RunnableConfig(**continuation_config) + agent_factory_kwargs: dict[str, Any] = {"config": initial_runnable_config} if ctx.app_config is not None and _agent_factory_supports_app_config(agent_factory): - agent = agent_factory(config=initial_runnable_config, app_config=ctx.app_config) - else: - agent = agent_factory(config=initial_runnable_config) + agent_factory_kwargs["app_config"] = ctx.app_config + from deerflow.extensions import bind_agent_build_extensions + + with bind_agent_build_extensions(extensions): + agent = agent_factory(**agent_factory_kwargs) accessor = CheckpointStateAccessor.bind( agent, diff --git a/backend/packages/harness/deerflow/subagents/executor.py b/backend/packages/harness/deerflow/subagents/executor.py index 43b71d2e0..0e4b70cc2 100644 --- a/backend/packages/harness/deerflow/subagents/executor.py +++ b/backend/packages/harness/deerflow/subagents/executor.py @@ -452,6 +452,7 @@ class SubagentExecutor: is_internal: bool = False, authz_attributes: Mapping[str, Any] | None = None, deerflow_trace_id: str | None = None, + extensions: Any | None = None, ): """Initialize the executor. @@ -476,6 +477,10 @@ class SubagentExecutor: the same run as the lead agent. deerflow_trace_id: DeerFlow request-level correlation id propagated from the parent run for Langfuse metadata correlation. + extensions: The parent run's immutable ``LoadedExtensions`` snapshot, + captured at ``task_tool`` dispatch. When None (embedded client, + standalone LangGraph Server), ``_aexecute`` falls back to the + process-wide singleton. """ self.config = config self.app_config = app_config @@ -508,6 +513,12 @@ class SubagentExecutor: self.is_internal = is_internal self.authz_attributes = normalize_authz_attributes(authz_attributes) self.deerflow_trace_id = deerflow_trace_id + # Parent run's extension snapshot. Binding it here (rather than reading + # the singleton at execution time) is what keeps one run on a single + # extension generation: a concurrent ``set_loaded_extensions()`` between + # the lead run's start and this subagent's execution must not swap the + # generation underneath the delegated work. + self.extensions = extensions self._base_tools = _filter_tools( tools, @@ -530,7 +541,13 @@ class SubagentExecutor: logger.info(f"[trace={self.trace_id}] SubagentExecutor initialized: {config.name} with {len(self.tools)} tools") - def _create_agent(self, tools: list[BaseTool] | None = None, *, deferred_setup: "DeferredToolSetup | None" = None): + def _create_agent( + self, + tools: list[BaseTool] | None = None, + *, + deferred_setup: "DeferredToolSetup | None" = None, + extensions=None, + ): """Create the agent instance. ``deferred_setup`` (assembled in ``_build_initial_state``) carries the @@ -564,6 +581,8 @@ class SubagentExecutor: "available_skills": self._available_skill_names, "user_id": self.user_id or DEFAULT_USER_ID, } + if extensions is not None: + middleware_kwargs["extensions"] = extensions authz_provider = getattr(self, "_authz_provider", None) if authz_provider is not None: middleware_kwargs["authorization_provider"] = authz_provider @@ -788,6 +807,14 @@ class SubagentExecutor: status=SubagentStatus.RUNNING, started_at=datetime.now(), ) + from deerflow.extensions import get_loaded_extensions + + loaded_extensions = self.extensions if self.extensions is not None else get_loaded_extensions() + task_store = None + if loaded_extensions.needs_task_store: + from deerflow_extension_api import ExtensionData + + task_store = ExtensionData(result.task_id) ai_messages = result.ai_messages if ai_messages is None: ai_messages = [] @@ -805,7 +832,11 @@ class SubagentExecutor: collector: SubagentTokenCollector | None = None try: state, final_tools, deferred_setup = await self._build_initial_state(task) - agent = self._create_agent(final_tools, deferred_setup=deferred_setup) + agent = self._create_agent( + final_tools, + deferred_setup=deferred_setup, + extensions=loaded_extensions, + ) # Token collector for subagent LLM calls collector_caller = f"subagent:{self.config.name}" @@ -866,6 +897,10 @@ class SubagentExecutor: context["oauth_provider"] = self.oauth_provider context["oauth_id"] = self.oauth_id context["run_id"] = self.run_id + if task_store is not None: + from deerflow_extension_api import EXTENSION_TASK_STORE_KEY + + context[EXTENSION_TASK_STORE_KEY] = task_store if self.channel_user_id: context["channel_user_id"] = self.channel_user_id # Authorization identity: is_internal written unconditionally diff --git a/backend/packages/harness/deerflow/tools/builtins/task_tool.py b/backend/packages/harness/deerflow/tools/builtins/task_tool.py index a9f9fdce6..b7a9cd2c9 100644 --- a/backend/packages/harness/deerflow/tools/builtins/task_tool.py +++ b/backend/packages/harness/deerflow/tools/builtins/task_tool.py @@ -14,6 +14,7 @@ from langgraph.types import Command from deerflow.authz.principal import normalize_authz_attributes from deerflow.config import get_app_config +from deerflow.extensions import resolve_run_extensions from deerflow.runtime.user_context import resolve_runtime_user_id from deerflow.sandbox.security import LOCAL_BASH_SUBAGENT_DISABLED_MESSAGE, is_host_bash_allowed from deerflow.subagents import SubagentExecutor, get_available_subagent_names, get_subagent_config @@ -360,6 +361,10 @@ async def task_tool( # server-side provenance as user_role/oauth — see inject_authenticated_user_context. is_internal = parent_context.get("is_internal") is True authz_attributes = normalize_authz_attributes(parent_context.get("authz_attributes")) + # The run's immutable extension snapshot, published by the run worker. Stays + # None outside that path (embedded client, standalone LangGraph Server), where + # the executor keeps its process-singleton fallback. + run_extensions = resolve_run_extensions(parent_context) deerflow_trace_id = normalize_trace_id(parent_context.get(DEERFLOW_TRACE_METADATA_KEY)) or normalize_trace_id(metadata.get(DEERFLOW_TRACE_METADATA_KEY)) or get_current_trace_id() parent_available_skills = metadata.get("available_skills") @@ -415,6 +420,8 @@ async def task_tool( } if resolved_app_config is not None: executor_kwargs["app_config"] = resolved_app_config + if run_extensions is not None: + executor_kwargs["extensions"] = run_extensions executor = SubagentExecutor(**executor_kwargs) # Start background execution (always async to prevent blocking) diff --git a/backend/packages/harness/pyproject.toml b/backend/packages/harness/pyproject.toml index 5dafec0ad..fe6e23e7a 100644 --- a/backend/packages/harness/pyproject.toml +++ b/backend/packages/harness/pyproject.toml @@ -7,6 +7,11 @@ dependencies = [ "agent-client-protocol>=0.4.0", "agent-sandbox>=0.0.30", "croniter>=6.0.0", + # Exact pin by design (extension-system version contract): the host pins + # the contract version it implements, extensions declare ranges. A range + # here would let pip resolve a newer contract package than this harness + # implements, making newer extensions look supported at runtime. + "deerflow-extension-api==0.1.0", "dotenv>=0.9.9", "exa-py>=1.0.0", "httpx>=0.28.0", diff --git a/backend/pyproject.toml b/backend/pyproject.toml index 1963b5d22..e47f41813 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -77,7 +77,8 @@ index-url = "https://pypi.org/simple" override-dependencies = ["websockets==16.0"] [tool.uv.workspace] -members = ["packages/harness"] +members = ["packages/harness", "packages/extension-api"] [tool.uv.sources] deerflow-harness = { workspace = true } +deerflow-extension-api = { workspace = true } diff --git a/backend/tests/test_extension_api_contracts.py b/backend/tests/test_extension_api_contracts.py new file mode 100644 index 000000000..d0681a15c --- /dev/null +++ b/backend/tests/test_extension_api_contracts.py @@ -0,0 +1,214 @@ +"""Tests for the extension contract surface. + +The contracts carry two compatibility promises that are easy to break by +accident and impossible to catch at runtime later: every Protocol method has a +default implementation, and every optional dataclass field has a default. Both +are asserted here. +""" + +from __future__ import annotations + +import dataclasses +import importlib.resources +import inspect + +import pytest +from deerflow_extension_api import ( + API_VERSION, + AgentBuildContext, + AgentScope, + ExtensionData, + ExtensionInstall, + ExtensionRegistry, + HostPolicySnapshot, + MiddlewareContributor, + MiddlewarePlacement, + Placement, + extension, +) +from deerflow_extension_api.runtime_bridge import ( + EXTENSION_TASK_STORE_KEY, + task_store_from_runtime, +) + + +def test_placement_members_cover_both_axes(): + assert Placement.MODEL_LOGICAL.value == "model_logical" + assert Placement.MODEL_PHYSICAL.value == "model_physical" + assert Placement.TOOL_VISIBLE.value == "tool_visible" + assert Placement.TOOL_RAW.value == "tool_raw" + assert Placement.STANDARD.value == "standard" + + +def test_agent_scope_both_is_union(): + assert AgentScope.BOTH == AgentScope.LEAD | AgentScope.SUBAGENT + assert AgentScope.LEAD in AgentScope.BOTH + + +def test_middleware_placement_defaults(): + p = MiddlewarePlacement(middleware=object(), placement=Placement.STANDARD) + assert p.scope is AgentScope.BOTH + assert p.order == 0 + + +@pytest.mark.parametrize( + "cls", + [ + HostPolicySnapshot, + AgentBuildContext, + MiddlewarePlacement, + ], +) +def test_every_dataclass_is_frozen(cls): + assert dataclasses.is_dataclass(cls) + assert cls.__dataclass_params__.frozen, f"{cls.__name__} must be frozen" + + +@pytest.mark.parametrize( + "cls", + [HostPolicySnapshot], +) +def test_additive_dataclasses_are_constructible_with_required_fields_only(cls): + """Fields added later must carry defaults, or old extensions break on upgrade. + + HostPolicySnapshot is host-constructed and fully optional. + AgentBuildContext gets its own dedicated test below because its scope is + legitimately required. + """ + assert cls() is not None + + +def test_agent_build_context_optional_fields_keep_their_defaults(): + """AgentBuildContext has one required field (scope); the rest must default. + + Unlike the fully-optional dataclasses above, scope is legitimately + required, so this is not folded into the parametrized test above — it + would misrepresent the required/optional split this suite is meant to + document. + """ + ctx = AgentBuildContext(scope=AgentScope.LEAD) + assert ctx.agent_name is None + assert ctx.model_name is None + assert isinstance(ctx.policy, HostPolicySnapshot) + + +@pytest.mark.parametrize( + "protocol", + [ + ExtensionRegistry, + MiddlewareContributor, + ], +) +def test_every_protocol_method_has_a_default_implementation(protocol): + """Adding a method to a Protocol is only additive when it has a default. + + Without this, shipping a new contract method breaks every already-released + extension that does not implement it. + """ + checked = 0 + for name, member in vars(protocol).items(): + if name.startswith("_") or not inspect.isfunction(member): + continue + checked += 1 + body = inspect.getsource(member).split("\n", 1)[1] + assert "return" in body, f"{protocol.__name__}.{name} has no default implementation. Adding a contract method is only additive when it returns a default; otherwise every already-released extension breaks on upgrade." + assert checked > 0, f"{protocol.__name__} declared no methods to check" + + +def test_contributor_defaults_return_empty(): + class _Bare: + pass + + bare = _Bare() + assert MiddlewareContributor.contribute_middlewares(bare, ExtensionData("app"), AgentBuildContext(scope=AgentScope.LEAD)) == () + + +def test_future_contribution_points_are_not_advertised_before_the_host_supports_them(): + """A merged slice must not silently accept registrations it cannot run.""" + import deerflow_extension_api + + for name in ( + "ExtensionRuntimeDeps", + "ExtensionService", + "SystemModelCallObserver", + "TaskLifecycleContributor", + ): + assert name not in deerflow_extension_api.__all__ + assert not hasattr(deerflow_extension_api, name) + + +def test_task_store_from_runtime_reads_the_host_key(): + class _Runtime: + def __init__(self, context): + self.context = context + + store = ExtensionData("task-1") + assert task_store_from_runtime(_Runtime({EXTENSION_TASK_STORE_KEY: store})) is store + + +def test_task_store_from_runtime_returns_none_on_missing_or_wrong_shape(): + class _Runtime: + def __init__(self, context): + self.context = context + + assert task_store_from_runtime(None) is None + assert task_store_from_runtime(_Runtime({})) is None + assert task_store_from_runtime(_Runtime("not-a-mapping")) is None + assert task_store_from_runtime(_Runtime({EXTENSION_TASK_STORE_KEY: "wrong type"})) is None + + +def test_extension_decorator_stamps_api_requirement(): + @extension(api="0.1", name="demo") + def install(registry, config): + return None + + assert install.__deerflow_api__ == "0.1" + assert install.__deerflow_name__ == "demo" + + +def test_registry_and_install_alias_are_part_of_the_public_surface(): + """Independent extensions annotate install(registry, config) against the + contract package alone — importing the host's concrete registry would pin + them to the harness release cadence and advertise host-only machinery.""" + import typing + + import deerflow_extension_api + + assert "ExtensionRegistry" in deerflow_extension_api.__all__ + assert "ExtensionInstall" in deerflow_extension_api.__all__ + parameters, return_type = typing.get_args(ExtensionInstall) + assert parameters[0] is ExtensionRegistry, "install()'s first argument must be the public registry contract" + + +def test_distribution_marks_the_contract_package_as_typed(): + marker = importlib.resources.files("deerflow_extension_api").joinpath("py.typed") + assert marker.is_file() + + +def test_harness_pins_the_contract_package_exactly(): + """The version contract (extension-system design): the host pins the + contract package exactly, extensions use ranges. A range here would let an + older harness resolve a newer 1.x contract package — API_VERSION would + then come from the upgraded package and newer extensions would look + supported against a host whose registry/placements/hook pipeline still + implements the older contract. The pin makes pip reject that skew at + install time.""" + import tomllib + from importlib.metadata import version + from pathlib import Path + + from packaging.requirements import Requirement + + pyproject = Path(__file__).parent.parent / "packages" / "harness" / "pyproject.toml" + dependencies = tomllib.loads(pyproject.read_text())["project"]["dependencies"] + requirement = next(Requirement(dep) for dep in dependencies if Requirement(dep).name == "deerflow-extension-api") + + expected = f"=={version('deerflow-extension-api')}" + assert str(requirement.specifier) == expected, f"the host must pin deerflow-extension-api exactly ({expected}); a range lets pip resolve a contract newer than the host implements" + + +def test_runtime_api_version_matches_the_installed_contract_package(): + """Every additive contract slice bumps both gates together.""" + from importlib.metadata import version + + assert API_VERSION == version("deerflow-extension-api") diff --git a/backend/tests/test_extension_api_state.py b/backend/tests/test_extension_api_state.py new file mode 100644 index 000000000..f1cf7487e --- /dev/null +++ b/backend/tests/test_extension_api_state.py @@ -0,0 +1,113 @@ +"""Tests for ExtensionData, the per-scope typed store handed to extensions.""" + +from __future__ import annotations + +from dataclasses import dataclass +from threading import Thread + +from deerflow_extension_api import ExtensionData + + +@dataclass +class _Counter: + value: int = 0 + + +@dataclass +class _Other: + name: str = "" + + +def test_get_returns_none_when_absent(): + store = ExtensionData("task-1") + assert store.get(_Counter) is None + + +def test_set_then_get_roundtrips(): + store = ExtensionData("task-1") + store.set(_Counter(value=7)) + got = store.get(_Counter) + assert got is not None + assert got.value == 7 + + +def test_get_or_init_creates_once(): + store = ExtensionData("task-1") + calls = [] + + def _init() -> _Counter: + calls.append(1) + return _Counter(value=1) + + first = store.get_or_init(_Counter, _init) + second = store.get_or_init(_Counter, _init) + assert first is second + assert calls == [1] + + +def test_get_or_init_allows_initializer_to_use_the_same_store(): + """Extension initializers may compose other extension-local state.""" + store = ExtensionData("task-1") + completed: list[_Counter] = [] + + def _init_counter() -> _Counter: + store.set(_Other(name="nested")) + return _Counter(value=2) + + def _initialize() -> None: + completed.append(store.get_or_init(_Counter, _init_counter)) + + thread = Thread(target=_initialize, daemon=True) + thread.start() + thread.join(timeout=0.5) + + assert not thread.is_alive(), "nested store access deadlocked" + assert completed == [_Counter(value=2)] + assert store.get(_Other) == _Other(name="nested") + + +def test_types_are_isolated(): + store = ExtensionData("task-1") + store.set(_Counter(value=1)) + store.set(_Other(name="x")) + assert store.get(_Counter).value == 1 + assert store.get(_Other).name == "x" + + +def test_remove_returns_and_clears(): + store = ExtensionData("task-1") + store.set(_Counter(value=3)) + removed = store.remove(_Counter) + assert removed.value == 3 + assert store.get(_Counter) is None + + +def test_scope_id_is_exposed(): + store = ExtensionData("run-42") + assert store.scope_id == "run-42" + + +def test_stores_are_independent(): + a = ExtensionData("task-a") + b = ExtensionData("task-b") + a.set(_Counter(value=1)) + assert b.get(_Counter) is None + + +def test_api_package_does_not_import_deerflow(): + """The API package must stay independent of the host so extensions can + depend on it alone. A `deerflow` import here would silently couple every + extension to the harness release cadence.""" + import pathlib + + import deerflow_extension_api + + root = pathlib.Path(deerflow_extension_api.__file__).parent + offenders = [] + for path in root.rglob("*.py"): + text = path.read_text(encoding="utf-8") + for lineno, line in enumerate(text.splitlines(), start=1): + stripped = line.strip() + if stripped.startswith(("import deerflow", "from deerflow")) and not stripped.startswith(("import deerflow_extension_api", "from deerflow_extension_api")): + offenders.append(f"{path.name}:{lineno}: {stripped}") + assert offenders == [], "deerflow-extension-api must not import deerflow: " + "; ".join(offenders) diff --git a/backend/tests/test_extension_app_loading.py b/backend/tests/test_extension_app_loading.py new file mode 100644 index 000000000..a2b041c7b --- /dev/null +++ b/backend/tests/test_extension_app_loading.py @@ -0,0 +1,202 @@ +"""Gateway app-construction wiring for configured Python extensions.""" + +from __future__ import annotations + +import pytest + +from deerflow.extensions import reset_loaded_extensions, reset_runtime_diagnostics +from deerflow.extensions.loader import ExtensionLoadError, ExtensionSpec +from deerflow.extensions.registry import ExtensionRegistry + + +@pytest.fixture(autouse=True) +def _reset_extension_process_state(): + reset_loaded_extensions() + reset_runtime_diagnostics() + yield + reset_runtime_diagnostics() + reset_loaded_extensions() + + +@pytest.fixture(autouse=True) +def stub_app_config(monkeypatch): + """Keep ``create_app()`` independent of a real ``config.yaml``. + + The repo-root ``config.yaml`` is gitignored and absent on CI runners, so + reading it here would make these tests pass locally and fail on every run + in CI. Tests that need a specific plugin list copy this config instead of + loading one from disk. + """ + import app.gateway.app as app_module + from deerflow.config.app_config import AppConfig + from deerflow.config.sandbox_config import SandboxConfig + + config = AppConfig(sandbox=SandboxConfig(use="test")) + monkeypatch.setattr(app_module, "get_app_config", lambda: config) + return config + + +def test_create_app_exposes_loaded_extensions_on_app_state_and_process_singleton(monkeypatch): + import deerflow.extensions as extensions_module + + loaded = ExtensionRegistry().build() + monkeypatch.setattr( + extensions_module, + "load_extensions", + lambda plugins: (loaded, []), + ) + + from app.gateway.app import create_app + + app = create_app() + + assert app.state.extensions is loaded + assert extensions_module.get_loaded_extensions() is loaded + + +def test_create_app_exposes_one_canonical_live_diagnostics_list(monkeypatch): + import deerflow.extensions as extensions_module + + loaded = ExtensionRegistry().build() + load_diagnostic = extensions_module.Diagnostic.warning( + "demo:install", + "optional extension was skipped", + ) + monkeypatch.setattr( + extensions_module, + "load_extensions", + lambda plugins: (loaded, [load_diagnostic]), + ) + + from app.gateway.app import create_app + + first_app = create_app() + second_app = create_app() + runtime_diagnostic = extensions_module.Diagnostic.error( + "demo:install", + "middleware observation failed", + ) + extensions_module.record_runtime_diagnostic(runtime_diagnostic) + + assert first_app.state.extension_diagnostics is second_app.state.extension_diagnostics + assert first_app.state.extension_diagnostics == [ + load_diagnostic, + runtime_diagnostic, + ] + + +def test_create_app_fails_open_when_extension_loading_raises_unexpectedly(monkeypatch): + import deerflow.extensions as extensions_module + + def _raise_unexpectedly(plugins): + raise RuntimeError("malformed plugins configuration") + + monkeypatch.setattr(extensions_module, "load_extensions", _raise_unexpectedly) + + from app.gateway.app import create_app + + app = create_app() + + assert app.state.extensions is extensions_module.EMPTY_EXTENSIONS + assert extensions_module.get_loaded_extensions() is extensions_module.EMPTY_EXTENSIONS + assert app.state.extension_diagnostics == [] + + +def test_create_app_fails_closed_when_a_required_extension_cannot_load(monkeypatch): + import deerflow.extensions as extensions_module + from app.gateway.app import create_app + + def _raise_required(plugins): + raise extensions_module.ExtensionLoadError("required extension acme_policy:install failed to install") + + monkeypatch.setattr(extensions_module, "load_extensions", _raise_required) + + with pytest.raises(extensions_module.ExtensionLoadError): + create_app() + + +def test_create_app_tolerates_a_missing_config_file_and_loads_no_extensions(monkeypatch): + """``create_app()`` runs at import time, so an absent config.yaml must not break it. + + Mirrors ``_resolve_trace_enabled_for_app_construction()``: lifespan still + performs strict config loading before the Gateway serves traffic. + """ + import app.gateway.app as app_module + import deerflow.extensions as extensions_module + + def _missing_config(): + raise FileNotFoundError("`config.yaml` file not found in the project root or legacy backend/repository root locations") + + monkeypatch.setattr(app_module, "get_app_config", _missing_config) + + observed_plugins = [] + loaded = ExtensionRegistry().build() + + def _record(plugins): + observed_plugins.append(plugins) + return loaded, [] + + monkeypatch.setattr(extensions_module, "load_extensions", _record) + + app = app_module.create_app() + + assert observed_plugins == [[]] + assert app.state.extensions is loaded + + +def test_create_app_propagates_config_failures_instead_of_blaming_extension_loading(monkeypatch): + """A parseable-but-broken config.yaml must not be swallowed by the fail-open guard. + + Extension loading is fail-open for unexpected errors, but resolving the + plugin list is not part of it: degrading to zero extensions there would + drop a ``required: true`` extension without failing the boot. + """ + import app.gateway.app as app_module + import deerflow.extensions as extensions_module + + def _broken_config(): + raise ValueError("config.yaml failed validation") + + monkeypatch.setattr(app_module, "get_app_config", _broken_config) + + def _must_not_run(plugins): + raise AssertionError("load_extensions must not run when the plugin list cannot be resolved") + + monkeypatch.setattr(extensions_module, "load_extensions", _must_not_run) + + with pytest.raises(ValueError, match="config.yaml failed validation"): + app_module.create_app() + + +def test_create_app_fails_closed_for_required_extension_with_malformed_api_marker(monkeypatch, stub_app_config): + import app.gateway.app as app_module + from extension_test_fixtures import demo_extensions + + class _ExplodingAPIMarker: + def split(self, separator: str) -> list[str]: + raise RuntimeError("API marker split exploded") + + def __str__(self) -> str: + return "exploding non-string marker" + + monkeypatch.setattr( + demo_extensions.install_ok, + "__deerflow_api__", + _ExplodingAPIMarker(), + raising=False, + ) + + config = stub_app_config.model_copy( + update={ + "plugins": [ + ExtensionSpec( + use="extension_test_fixtures.demo_extensions:install_ok", + required=True, + ) + ] + } + ) + monkeypatch.setattr(app_module, "get_app_config", lambda: config) + + with pytest.raises(ExtensionLoadError, match="declares invalid api marker"): + app_module.create_app() diff --git a/backend/tests/test_extension_config.py b/backend/tests/test_extension_config.py new file mode 100644 index 000000000..236daa8e5 --- /dev/null +++ b/backend/tests/test_extension_config.py @@ -0,0 +1,142 @@ +"""Tests for extension configuration and the process-wide singleton.""" + +from __future__ import annotations + +import pytest +from pydantic import ValidationError + +from deerflow.config.app_config import AppConfig +from deerflow.config.reload_boundary import STARTUP_ONLY_FIELDS +from deerflow.extensions import ( + EMPTY_EXTENSIONS, + ExtensionRegistry, + get_loaded_extensions, + reset_loaded_extensions, + set_loaded_extensions, +) + + +@pytest.fixture(autouse=True) +def _reset_singleton(): + reset_loaded_extensions() + yield + reset_loaded_extensions() + + +class _Marker: + """Sentinel written into an app_store to detect state leaking across resets.""" + + def __init__(self, tag: str = "") -> None: + self.tag = tag + + +# AppConfig.sandbox has no default (see app_config.py's +# `_drop_null_config_sections`: "Required sections without a default +# (sandbox) intentionally still error when null"), so every AppConfig +# construction below supplies it, matching the pattern already used in +# test_app_config_reload.py. +_SANDBOX = {"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"}} + + +def test_app_config_defaults_to_no_plugins(): + assert AppConfig.model_validate(_SANDBOX).plugins == [] + + +def test_app_config_parses_plugin_entries(): + config = AppConfig.model_validate( + { + **_SANDBOX, + "plugins": [ + {"use": "acme_observability:install", "config": {"enabled": True}}, + {"use": "acme_policy:install", "required": True}, + ], + } + ) + assert [e.use for e in config.plugins] == ["acme_observability:install", "acme_policy:install"] + assert config.plugins[0].config == {"enabled": True} + assert config.plugins[0].required is False + assert config.plugins[1].required is True + + +def test_plugin_entries_reject_unknown_fields_instead_of_weakening_required(): + with pytest.raises(ValidationError, match="require"): + AppConfig.model_validate( + { + **_SANDBOX, + "plugins": [ + { + "use": "acme_policy:install", + "require": True, + } + ], + } + ) + + +def test_new_field_does_not_disturb_the_existing_extensions_field(): + """AppConfig.extensions is a pre-existing, unrelated field (MCP servers, + skills, config-declared middlewares) backed by extensions_config.json. + The plugin list is deliberately a separate top-level key: that file is + writable through an HTTP endpoint, and a code-loading list must not be.""" + config = AppConfig.model_validate({**_SANDBOX, "plugins": [{"use": "a:install"}]}) + assert config.plugins[0].use == "a:install" + assert hasattr(config.extensions, "mcp_servers") + assert hasattr(config.extensions, "middlewares") + + +def test_plugins_is_registered_as_startup_only(): + """Plugins load once in create_app(); a config.yaml edit needs a restart. + Registering here is what surfaces that to operators.""" + assert "plugins" in STARTUP_ONLY_FIELDS + assert "restart" in STARTUP_ONLY_FIELDS["plugins"].lower() + + +def test_singleton_defaults_to_empty(): + loaded = get_loaded_extensions() + assert loaded.has_middleware_contributors is False + assert loaded.needs_task_store is False + + +def test_singleton_roundtrips(): + loaded = ExtensionRegistry().build() + set_loaded_extensions(loaded) + assert get_loaded_extensions() is loaded + + +def test_reset_gives_a_fresh_instance(): + """Reset must not hand back a shared object. EMPTY_EXTENSIONS owns a + mutable app_store, so resetting to it would carry writes forward.""" + populated = ExtensionRegistry().build() + set_loaded_extensions(populated) + reset_loaded_extensions() + after = get_loaded_extensions() + assert after is not populated + assert after is not EMPTY_EXTENSIONS + assert after.has_middleware_contributors is False + + +def test_reset_does_not_leak_app_store_writes(): + """The regression the fresh-build reset exists to prevent.""" + reset_loaded_extensions() + get_loaded_extensions().app_store.set(_Marker("dirty")) + reset_loaded_extensions() + assert get_loaded_extensions().app_store.get(_Marker) is None + + +def test_runtime_diagnostics_are_bounded_without_replacing_the_live_list(monkeypatch): + import deerflow.extensions as extensions_module + + monkeypatch.setattr(extensions_module, "_MAX_RUNTIME_DIAGNOSTICS", 3) + extensions_module.reset_runtime_diagnostics() + live = extensions_module.initialize_runtime_diagnostics([]) + try: + for index in range(5): + extensions_module.record_runtime_diagnostic(extensions_module.Diagnostic.error("demo:install", f"error-{index}")) + + assert [diagnostic.message for diagnostic in live] == [ + "error-2", + "error-3", + "error-4", + ] + finally: + extensions_module.reset_runtime_diagnostics() diff --git a/backend/tests/test_extension_injection.py b/backend/tests/test_extension_injection.py new file mode 100644 index 000000000..1b6c02e76 --- /dev/null +++ b/backend/tests/test_extension_injection.py @@ -0,0 +1,401 @@ +"""Tests for resolving semantic placements into real stack positions.""" + +from __future__ import annotations + +from deerflow_extension_api import ( + AgentBuildContext, + AgentScope, + MiddlewarePlacement, + Placement, +) +from langchain.agents import create_agent +from langchain.agents.middleware import AgentMiddleware +from langchain_core.language_models.fake_chat_models import FakeMessagesListChatModel +from langchain_core.messages import AIMessage + +from deerflow.extensions.anchors import PlacementAnchor, inner_of, innermost, outer_of, outermost +from deerflow.extensions.injection import inject_middlewares +from deerflow.extensions.registry import ExtensionRegistry + + +class _Core: + """Stand-in for a core middleware; only its type matters for anchoring.""" + + +class _Retry(_Core): + pass + + +class _Transform(_Core): + pass + + +class _ToolError(_Core): + pass + + +class _Last(_Core): + pass + + +class _Probe(AgentMiddleware): + def __init__(self, tag: str) -> None: + super().__init__() + self.tag = tag + + +class _NamedCoreMiddleware(AgentMiddleware): + def __init__(self, name: str) -> None: + super().__init__() + self._name = name + + @property + def name(self) -> str: + return self._name + + +class _FakeModel(FakeMessagesListChatModel): + def bind_tools(self, tools, **kwargs): # type: ignore[override] + return self + + +_ANCHORS = { + Placement.MODEL_LOGICAL: outer_of(_Retry), + Placement.MODEL_PHYSICAL: inner_of(_Transform), + Placement.TOOL_VISIBLE: outermost(), + Placement.TOOL_RAW: inner_of(_ToolError), + Placement.STANDARD: outer_of(_Last), +} + + +def _stack() -> list[object]: + return [_Retry(), _Transform(), _ToolError(), _Last()] + + +def _contributor(*placements: MiddlewarePlacement): + class _C: + def contribute_middlewares(self, app_store, ctx): + return placements + + return _C() + + +def _extensions(*placements: MiddlewarePlacement): + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_contributor(*placements)) + return registry.build() + + +def _ctx() -> AgentBuildContext: + return AgentBuildContext(scope=AgentScope.LEAD) + + +def _tags(stack: list[object]) -> list[str]: + from deerflow.extensions.isolation import IsolatedMiddleware + + out = [] + for m in stack: + target = m.inner if isinstance(m, IsolatedMiddleware) else m + out.append(target.tag if isinstance(target, _Probe) else type(target).__name__) + return out + + +def test_outermost_lands_at_index_zero(): + probe = _Probe("visible") + result, _, _ = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions(MiddlewarePlacement(probe, Placement.TOOL_VISIBLE)), + ) + assert _tags(result)[0] == "visible" + + +def test_outer_of_lands_immediately_before_the_anchor(): + probe = _Probe("decision") + result, _, _ = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions(MiddlewarePlacement(probe, Placement.MODEL_LOGICAL)), + ) + assert _tags(result) == ["decision", "_Retry", "_Transform", "_ToolError", "_Last"] + + +def test_inner_of_lands_immediately_after_the_anchor(): + probe = _Probe("attempt") + result, _, _ = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions(MiddlewarePlacement(probe, Placement.MODEL_PHYSICAL)), + ) + assert _tags(result) == ["_Retry", "_Transform", "attempt", "_ToolError", "_Last"] + + +def test_all_four_placements_nest_correctly(): + result, _, _ = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions( + MiddlewarePlacement(_Probe("visible"), Placement.TOOL_VISIBLE), + MiddlewarePlacement(_Probe("decision"), Placement.MODEL_LOGICAL), + MiddlewarePlacement(_Probe("attempt"), Placement.MODEL_PHYSICAL), + MiddlewarePlacement(_Probe("raw"), Placement.TOOL_RAW), + ), + ) + assert _tags(result) == [ + "visible", + "decision", + "_Retry", + "_Transform", + "attempt", + "_ToolError", + "raw", + "_Last", + ] + + +def test_multiple_extension_middlewares_can_compile_into_one_agent(): + result, _, _ = inject_middlewares( + [], + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions( + MiddlewarePlacement(_Probe("first"), Placement.TOOL_VISIBLE), + MiddlewarePlacement(_Probe("second"), Placement.TOOL_VISIBLE), + ), + ) + + agent = create_agent( + _FakeModel(responses=[AIMessage(content="ok")]), + middleware=result, + ) + + assert agent is not None + + +def test_extension_middleware_names_avoid_langgraph_reserved_characters(): + result, _, _ = inject_middlewares( + [], + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions(MiddlewarePlacement(_Probe("probe"), Placement.TOOL_VISIBLE)), + ) + + names = [middleware.name for middleware in result] + assert all(":" not in name and "|" not in name for name in names) + + +def test_extension_middleware_names_are_stable_across_equivalent_builds(): + def build_names() -> list[str]: + result, _, _ = inject_middlewares( + [], + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions( + MiddlewarePlacement(_Probe("first"), Placement.TOOL_VISIBLE), + MiddlewarePlacement(_Probe("second"), Placement.TOOL_VISIBLE), + ), + ) + return [middleware.name for middleware in result] + + assert build_names() == build_names() + + +def test_extension_middleware_name_does_not_collide_with_the_core_stack(): + placement = MiddlewarePlacement(_Probe("probe"), Placement.TOOL_VISIBLE) + first_result, _, _ = inject_middlewares( + [], + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions(placement), + ) + core = _NamedCoreMiddleware(first_result[0].name) + + result, _, _ = inject_middlewares( + [core], + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions(placement), + ) + + agent = create_agent( + _FakeModel(responses=[AIMessage(content="ok")]), + middleware=result, + ) + assert agent is not None + + +def test_scope_filters_out_non_matching_contributions(): + result, _, _ = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.SUBAGENT, + _ctx(), + _extensions(MiddlewarePlacement(_Probe("lead-only"), Placement.STANDARD, scope=AgentScope.LEAD)), + ) + assert "lead-only" not in _tags(result) + + +def test_scope_both_applies_everywhere(): + for scope in (AgentScope.LEAD, AgentScope.SUBAGENT): + result, _, _ = inject_middlewares( + _stack(), + _ANCHORS, + scope, + _ctx(), + _extensions(MiddlewarePlacement(_Probe("everywhere"), Placement.STANDARD)), + ) + assert "everywhere" in _tags(result) + + +def test_order_field_breaks_ties_within_a_placement(): + result, _, _ = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions( + MiddlewarePlacement(_Probe("second"), Placement.TOOL_VISIBLE, order=10), + MiddlewarePlacement(_Probe("first"), Placement.TOOL_VISIBLE, order=1), + ), + ) + assert _tags(result)[:2] == ["first", "second"] + + +def test_provenance_maps_index_to_source(): + probe = _Probe("visible") + result, provenance, _ = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions(MiddlewarePlacement(probe, Placement.TOOL_VISIBLE)), + ) + assert provenance[0] == "demo:install" + assert 1 not in provenance, "core middlewares carry no extension provenance" + + +def test_missing_anchor_falls_back_and_warns(): + """A conditionally-built stack may lack the anchor. Degrading silently + would change what the extension observes with no signal.""" + anchors = {Placement.MODEL_PHYSICAL: PlacementAnchor.of(inner_of(_Transform), innermost())} + stack = [_Retry(), _ToolError()] # no _Transform + result, _, diagnostics = inject_middlewares( + stack, + anchors, + AgentScope.LEAD, + _ctx(), + _extensions(MiddlewarePlacement(_Probe("attempt"), Placement.MODEL_PHYSICAL)), + ) + assert _tags(result)[-1] == "attempt" + assert [d.level for d in diagnostics] == ["warning"] + assert "MODEL_PHYSICAL" in diagnostics[0].message + + +def test_no_contributors_returns_the_stack_untouched(): + stack = _stack() + result, provenance, diagnostics = inject_middlewares(stack, _ANCHORS, AgentScope.LEAD, _ctx(), ExtensionRegistry().build()) + assert result == stack + assert provenance == {} + assert diagnostics == [] + + +def test_contributor_failure_is_isolated_to_that_extension(): + class _Boom: + def contribute_middlewares(self, app_store, ctx): + raise ValueError("contributor exploded") + + registry = ExtensionRegistry() + with registry.attributed_to("bad:install"): + registry.middlewares(_Boom()) + with registry.attributed_to("good:install"): + registry.middlewares(_contributor(MiddlewarePlacement(_Probe("ok"), Placement.TOOL_VISIBLE))) + + result, _, diagnostics = inject_middlewares(_stack(), _ANCHORS, AgentScope.LEAD, _ctx(), registry.build()) + assert "ok" in _tags(result) + assert any(d.level == "error" and d.source == "bad:install" for d in diagnostics) + + +def test_contributor_iterable_failure_is_isolated_to_that_extension(): + class _ExplodingIterable: + def __iter__(self): + raise ValueError("iteration exploded") + + class _Boom: + def contribute_middlewares(self, app_store, ctx): + return _ExplodingIterable() + + registry = ExtensionRegistry() + with registry.attributed_to("bad:install"): + registry.middlewares(_Boom()) + with registry.attributed_to("good:install"): + registry.middlewares(_contributor(MiddlewarePlacement(_Probe("ok"), Placement.TOOL_VISIBLE))) + + result, _, diagnostics = inject_middlewares(_stack(), _ANCHORS, AgentScope.LEAD, _ctx(), registry.build()) + + assert "ok" in _tags(result) + assert any(d.source == "bad:install" and "iteration exploded" in d.message for d in diagnostics) + + +def test_malformed_placement_is_skipped_without_losing_other_extensions(): + class _Malformed: + def contribute_middlewares(self, app_store, ctx): + return (object(),) + + registry = ExtensionRegistry() + with registry.attributed_to("bad:install"): + registry.middlewares(_Malformed()) + with registry.attributed_to("good:install"): + registry.middlewares(_contributor(MiddlewarePlacement(_Probe("ok"), Placement.TOOL_VISIBLE))) + + result, _, diagnostics = inject_middlewares(_stack(), _ANCHORS, AgentScope.LEAD, _ctx(), registry.build()) + + assert "ok" in _tags(result) + assert any(d.source == "bad:install" and "MiddlewarePlacement" in d.message for d in diagnostics) + + +def test_non_middleware_contribution_is_skipped(): + result, _, diagnostics = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions(MiddlewarePlacement(object(), Placement.TOOL_VISIBLE)), + ) + + assert len(result) == len(_stack()) + assert any("AgentMiddleware" in diagnostic.message for diagnostic in diagnostics) + + +def test_wrapper_construction_failure_is_isolated_to_one_contribution(): + class _BadName(AgentMiddleware): + @property + def name(self): + raise ValueError("name exploded") + + result, _, diagnostics = inject_middlewares( + _stack(), + _ANCHORS, + AgentScope.LEAD, + _ctx(), + _extensions( + MiddlewarePlacement(_BadName(), Placement.TOOL_VISIBLE), + MiddlewarePlacement(_Probe("ok"), Placement.TOOL_VISIBLE), + ), + ) + + assert "ok" in _tags(result) + assert any("name exploded" in diagnostic.message for diagnostic in diagnostics) diff --git a/backend/tests/test_extension_isolation.py b/backend/tests/test_extension_isolation.py new file mode 100644 index 000000000..e4a4432ef --- /dev/null +++ b/backend/tests/test_extension_isolation.py @@ -0,0 +1,730 @@ +"""Tests for isolating extension middleware failures from the user's run.""" + +from __future__ import annotations + +import asyncio + +import pytest +from langchain.agents.middleware import AgentMiddleware +from langgraph.errors import GraphBubbleUp + +from deerflow.extensions.isolation import IsolatedMiddleware + + +class _Boom(AgentMiddleware): + def wrap_model_call(self, request, handler): + raise ValueError("observation exploded") + + async def awrap_model_call(self, request, handler): + raise ValueError("observation exploded") + + def wrap_tool_call(self, request, handler): + raise ValueError("observation exploded") + + async def awrap_tool_call(self, request, handler): + raise ValueError("observation exploded") + + +class _Bubble(AgentMiddleware): + def wrap_tool_call(self, request, handler): + raise GraphBubbleUp() + + async def awrap_tool_call(self, request, handler): + raise GraphBubbleUp() + + +class _Passthrough(AgentMiddleware): + def __init__(self) -> None: + super().__init__() + self.seen = 0 + + def wrap_tool_call(self, request, handler): + self.seen += 1 + return handler(request) + + +def _handler(request): + return "core-result" + + +async def _ahandler(request): + return "core-result" + + +def test_failing_middleware_falls_through_to_the_handler(): + errors = [] + wrapped = IsolatedMiddleware(_Boom(), "bad:install", errors.append) + assert wrapped.wrap_tool_call("req", _handler) == "core-result" + assert wrapped.wrap_model_call("req", _handler) == "core-result" + assert len(errors) == 2 + assert errors[0].source == "bad:install" + assert errors[0].level == "error" + + +def test_failing_async_middleware_falls_through(): + errors = [] + wrapped = IsolatedMiddleware(_Boom(), "bad:install", errors.append) + assert asyncio.run(wrapped.awrap_tool_call("req", _ahandler)) == "core-result" + assert asyncio.run(wrapped.awrap_model_call("req", _ahandler)) == "core-result" + assert len(errors) == 2 + + +def test_graph_bubble_up_propagates_unchanged(): + """GraphBubbleUp carries LangGraph's interrupt/pause/resume control flow. + Swallowing it would break the graph, not just the observation.""" + wrapped = IsolatedMiddleware(_Bubble(), "ext:install", lambda d: None) + with pytest.raises(GraphBubbleUp): + wrapped.wrap_tool_call("req", _handler) + with pytest.raises(GraphBubbleUp): + asyncio.run(wrapped.awrap_tool_call("req", _ahandler)) + + +def test_working_middleware_is_not_disturbed(): + inner = _Passthrough() + wrapped = IsolatedMiddleware(inner, "ok:install", lambda d: None) + assert wrapped.wrap_tool_call("req", _handler) == "core-result" + assert inner.seen == 1 + + +def test_sync_only_wrap_hook_falls_through_on_async_execution_path(): + inner = _Passthrough() + wrapped = IsolatedMiddleware(inner, "ok:install", lambda d: None) + + assert asyncio.run(wrapped.awrap_tool_call("req", _ahandler)) == "core-result" + assert inner.seen == 0, "the unavailable sync observer must not run on the async path" + + +def test_async_only_wrap_hook_falls_through_on_sync_execution_path(): + class _AsyncOnly(AgentMiddleware): + async def awrap_model_call(self, request, handler): + raise AssertionError("the unavailable async observer must not run on the sync path") + + wrapped = IsolatedMiddleware(_AsyncOnly(), "ok:install", lambda d: None) + + assert wrapped.wrap_model_call("req", _handler) == "core-result" + + +def test_observer_cannot_replace_the_downstream_request(): + class _RewritesRequest(AgentMiddleware): + def wrap_tool_call(self, request, handler): + return handler("mutated") + + seen = [] + + def handler(request): + seen.append(request) + return "core-result" + + wrapped = IsolatedMiddleware(_RewritesRequest(), "observer:install", lambda d: None) + + assert wrapped.wrap_tool_call("original", handler) == "core-result" + assert seen == ["original"] + + +def test_async_observer_cannot_replace_the_downstream_request(): + class _RewritesRequest(AgentMiddleware): + async def awrap_model_call(self, request, handler): + return await handler("mutated") + + seen = [] + + async def handler(request): + seen.append(request) + return "core-result" + + wrapped = IsolatedMiddleware(_RewritesRequest(), "observer:install", lambda d: None) + + assert asyncio.run(wrapped.awrap_model_call("original", handler)) == "core-result" + assert seen == ["original"] + + +def test_post_handler_failure_does_not_replay_tool_handler(): + """A post-call observer failure must not repeat a tool's side effects.""" + + class _FailsAfterHandler(AgentMiddleware): + def wrap_tool_call(self, request, handler): + handler(request) + raise ValueError("post-call observation exploded") + + calls: list[str] = [] + + def side_effecting_handler(request): + calls.append(request) + return "core-result" + + errors = [] + wrapped = IsolatedMiddleware( + _FailsAfterHandler(), + "bad:install", + errors.append, + ) + + assert wrapped.wrap_tool_call("req", side_effecting_handler) == "core-result" + assert calls == ["req"] + assert len(errors) == 1 + + +def test_tool_handler_failure_propagates_without_replay_or_diagnostic(): + """A real tool failure belongs to the graph, not extension isolation.""" + failure = RuntimeError("tool exploded") + calls: list[str] = [] + + def failing_handler(request): + calls.append(request) + raise failure + + errors = [] + wrapped = IsolatedMiddleware( + _Passthrough(), + "observer:install", + errors.append, + ) + + with pytest.raises(RuntimeError) as exc_info: + wrapped.wrap_tool_call("req", failing_handler) + + assert exc_info.value is failure + assert calls == ["req"] + assert errors == [] + + +def test_post_handler_failure_does_not_replay_model_handler(): + """A post-call observer failure must not duplicate provider cost.""" + + class _FailsAfterHandler(AgentMiddleware): + def wrap_model_call(self, request, handler): + handler(request) + raise ValueError("post-call observation exploded") + + calls: list[str] = [] + + def counted_handler(request): + calls.append(request) + return "model-result" + + errors = [] + wrapped = IsolatedMiddleware( + _FailsAfterHandler(), + "bad:install", + errors.append, + ) + + assert wrapped.wrap_model_call("req", counted_handler) == "model-result" + assert calls == ["req"] + assert len(errors) == 1 + + +def test_async_post_handler_failure_does_not_replay_tool_handler(): + """Isolation must not add another async tool side effect.""" + + class _FailsAfterHandler(AgentMiddleware): + async def awrap_tool_call(self, request, handler): + await handler(request) + raise ValueError("post-call observation exploded") + + calls: list[str] = [] + + async def side_effecting_handler(request): + calls.append(request) + return "core-result" + + errors = [] + wrapped = IsolatedMiddleware( + _FailsAfterHandler(), + "bad:install", + errors.append, + ) + + result = asyncio.run(wrapped.awrap_tool_call("req", side_effecting_handler)) + + assert result == "core-result" + assert calls == ["req"] + assert len(errors) == 1 + + +def test_async_tool_handler_failure_propagates_without_replay_or_diagnostic(): + """Async graph failures remain owned by the graph's error policy.""" + + class _AsyncPassthrough(AgentMiddleware): + async def awrap_tool_call(self, request, handler): + return await handler(request) + + failure = RuntimeError("tool exploded") + calls: list[str] = [] + + async def failing_handler(request): + calls.append(request) + raise failure + + errors = [] + wrapped = IsolatedMiddleware( + _AsyncPassthrough(), + "observer:install", + errors.append, + ) + + with pytest.raises(RuntimeError) as exc_info: + asyncio.run(wrapped.awrap_tool_call("req", failing_handler)) + + assert exc_info.value is failure + assert calls == ["req"] + assert errors == [] + + +def test_async_handler_cancellation_propagates_without_replay_or_diagnostic(): + """Cancellation is control flow and must never enter fail-open recovery.""" + + class _AsyncPassthrough(AgentMiddleware): + async def awrap_tool_call(self, request, handler): + return await handler(request) + + calls: list[str] = [] + + async def cancelled_handler(request): + calls.append(request) + raise asyncio.CancelledError + + errors = [] + wrapped = IsolatedMiddleware( + _AsyncPassthrough(), + "observer:install", + errors.append, + ) + + with pytest.raises(asyncio.CancelledError): + asyncio.run(wrapped.awrap_tool_call("req", cancelled_handler)) + + assert calls == ["req"] + assert errors == [] + + +def test_async_post_handler_failure_does_not_replay_model_handler(): + """Async provider calls also retain their first successful result.""" + + class _FailsAfterHandler(AgentMiddleware): + async def awrap_model_call(self, request, handler): + await handler(request) + raise ValueError("post-call observation exploded") + + calls: list[str] = [] + + async def counted_handler(request): + calls.append(request) + return "model-result" + + errors = [] + wrapped = IsolatedMiddleware( + _FailsAfterHandler(), + "bad:install", + errors.append, + ) + + result = asyncio.run(wrapped.awrap_model_call("req", counted_handler)) + + assert result == "model-result" + assert calls == ["req"] + assert len(errors) == 1 + + +def test_middleware_cannot_replace_a_handler_failure_with_its_own_error(): + """The graph keeps ownership even if an observer masks its exception.""" + + class _MasksHandlerFailure(AgentMiddleware): + def wrap_tool_call(self, request, handler): + try: + return handler(request) + except RuntimeError: + raise ValueError("observer cleanup exploded") from None + + failure = RuntimeError("tool exploded") + calls: list[str] = [] + + def failing_handler(request): + calls.append(request) + raise failure + + errors = [] + wrapped = IsolatedMiddleware( + _MasksHandlerFailure(), + "observer:install", + errors.append, + ) + + with pytest.raises(RuntimeError) as exc_info: + wrapped.wrap_tool_call("req", failing_handler) + + assert exc_info.value is failure + assert calls == ["req"] + assert errors == [] + + +def test_middleware_cannot_replace_a_handler_failure_with_graph_bubble_up(): + class _MasksHandlerFailure(AgentMiddleware): + def wrap_tool_call(self, request, handler): + try: + return handler(request) + except RuntimeError: + raise GraphBubbleUp() from None + + failure = RuntimeError("tool exploded") + + def failing_handler(request): + raise failure + + wrapped = IsolatedMiddleware(_MasksHandlerFailure(), "observer:install", lambda d: None) + + with pytest.raises(RuntimeError) as exc_info: + wrapped.wrap_tool_call("req", failing_handler) + + assert exc_info.value is failure + + +def test_post_handler_graph_bubble_up_cannot_discard_a_successful_result(): + class _InterruptsAfterHandler(AgentMiddleware): + def wrap_tool_call(self, request, handler): + handler(request) + raise GraphBubbleUp() + + errors = [] + wrapped = IsolatedMiddleware(_InterruptsAfterHandler(), "observer:install", errors.append) + + assert wrapped.wrap_tool_call("req", _handler) == "core-result" + assert len(errors) == 1 + + +def test_middleware_cannot_swallow_a_handler_failure_with_a_fallback(): + class _SwallowsHandlerFailure(AgentMiddleware): + def wrap_tool_call(self, request, handler): + try: + handler(request) + except RuntimeError: + return "extension-fallback" + + failure = RuntimeError("tool exploded") + calls: list[str] = [] + + def failing_handler(request): + calls.append(request) + raise failure + + errors = [] + wrapped = IsolatedMiddleware(_SwallowsHandlerFailure(), "observer:install", errors.append) + + with pytest.raises(RuntimeError) as exc_info: + wrapped.wrap_tool_call("req", failing_handler) + + assert exc_info.value is failure + assert calls == ["req"] + assert errors == [] + + +def test_middleware_cannot_call_a_side_effecting_handler_twice(): + class _CallsTwice(AgentMiddleware): + def wrap_tool_call(self, request, handler): + first = handler(request) + try: + handler(request) + except RuntimeError: + return first + + calls: list[str] = [] + + def side_effecting_handler(request): + calls.append(request) + return "core-result" + + errors = [] + wrapped = IsolatedMiddleware(_CallsTwice(), "observer:install", errors.append) + + assert wrapped.wrap_tool_call("req", side_effecting_handler) == "core-result" + assert calls == ["req"] + assert len(errors) == 1 + assert "more than once" in errors[0].message + + +def test_middleware_cannot_skip_the_handler_or_replace_its_result(): + class _SkipsHandler(AgentMiddleware): + def wrap_model_call(self, request, handler): + return "extension-result" + + calls: list[str] = [] + + def handler(request): + calls.append(request) + return "core-result" + + errors = [] + wrapped = IsolatedMiddleware(_SkipsHandler(), "observer:install", errors.append) + + assert wrapped.wrap_model_call("req", handler) == "core-result" + assert calls == ["req"] + assert len(errors) == 1 + assert "did not call" in errors[0].message + + +def test_async_middleware_cannot_swallow_or_repeat_handler_calls(): + class _SwallowsAndRepeats(AgentMiddleware): + async def awrap_tool_call(self, request, handler): + try: + await handler(request) + except RuntimeError: + try: + await handler(request) + except RuntimeError: + return "extension-fallback" + + failure = RuntimeError("tool exploded") + calls: list[str] = [] + + async def failing_handler(request): + calls.append(request) + raise failure + + errors = [] + wrapped = IsolatedMiddleware(_SwallowsAndRepeats(), "observer:install", errors.append) + + with pytest.raises(RuntimeError) as exc_info: + asyncio.run(wrapped.awrap_tool_call("req", failing_handler)) + + assert exc_info.value is failure + assert calls == ["req"] + assert errors == [] + + +def test_error_message_identifies_the_hook(): + errors = [] + wrapped = IsolatedMiddleware(_Boom(), "bad:install", errors.append) + wrapped.wrap_tool_call("req", _handler) + assert "wrap_tool_call" in errors[0].message + + +# --- interface preservation ------------------------------------------------- +# +# LangChain discovers middleware capabilities by inspecting the *wrapper*: hook +# participation is a class-level identity check (`m.__class__.before_model is +# not AgentMiddleware.before_model`, see langchain/agents/factory.py), and +# tools/state_schema/transformers are read off the middleware instance. The +# wrapper must mirror the inner middleware's full interface, not just the four +# wrap-call hooks — otherwise lifecycle hooks silently never enter the graph +# and contributed tools/state never register. + +_LIFECYCLE_HOOKS = ( + "before_agent", + "abefore_agent", + "before_model", + "abefore_model", + "after_model", + "aafter_model", + "after_agent", + "aafter_agent", +) + + +def _langchain_detects(middleware: AgentMiddleware, hook_name: str) -> bool: + """The exact check langchain.agents.factory uses to decide whether a hook + node is added to the graph.""" + return getattr(type(middleware), hook_name) is not getattr(AgentMiddleware, hook_name) + + +class _LifecycleObserver(AgentMiddleware): + """Implements every lifecycle hook, sync and async, none of the wrap-calls.""" + + def __init__(self) -> None: + super().__init__() + self.calls: list[str] = [] + + def before_agent(self, state, runtime): + self.calls.append("before_agent") + return {"seen": "before_agent"} + + async def abefore_agent(self, state, runtime): + self.calls.append("abefore_agent") + return {"seen": "abefore_agent"} + + def before_model(self, state, runtime): + self.calls.append("before_model") + return {"seen": "before_model"} + + async def abefore_model(self, state, runtime): + self.calls.append("abefore_model") + return {"seen": "abefore_model"} + + def after_model(self, state, runtime): + self.calls.append("after_model") + return {"seen": "after_model"} + + async def aafter_model(self, state, runtime): + self.calls.append("aafter_model") + return {"seen": "aafter_model"} + + def after_agent(self, state, runtime): + self.calls.append("after_agent") + return {"seen": "after_agent"} + + async def aafter_agent(self, state, runtime): + self.calls.append("aafter_agent") + return {"seen": "aafter_agent"} + + +def test_wrapper_advertises_the_lifecycle_hooks_the_inner_implements(): + """A wrapped lifecycle observer must be *seen* by LangChain: without a + class-level override the factory never adds the hook node to the graph and + the inner hook silently never runs.""" + wrapped = IsolatedMiddleware(_LifecycleObserver(), "obs:install", lambda d: None) + missing = [hook for hook in _LIFECYCLE_HOOKS if not _langchain_detects(wrapped, hook)] + assert missing == [], f"LangChain cannot see these wrapped hooks: {missing}" + + +def test_wrapper_does_not_fabricate_hooks_the_inner_lacks(): + """The mirror must be exact: fabricating hooks would bolt no-op nodes onto + every graph and corrupt middleware_implements-based placement checks.""" + wrapped = IsolatedMiddleware(_Passthrough(), "ok:install", lambda d: None) + fabricated = [hook for hook in _LIFECYCLE_HOOKS if _langchain_detects(wrapped, hook)] + assert fabricated == [], f"the wrapper invented hooks the inner lacks: {fabricated}" + + +def test_wrapper_mirrors_sync_and_async_hooks_independently(): + """LangChain wires sync and async variants separately; an async-only inner + must not cause a sync no-op node (and vice versa).""" + + class _AsyncOnly(AgentMiddleware): + async def abefore_model(self, state, runtime): + return None + + wrapped = IsolatedMiddleware(_AsyncOnly(), "obs:install", lambda d: None) + assert _langchain_detects(wrapped, "abefore_model") + assert not _langchain_detects(wrapped, "before_model") + for hook in _LIFECYCLE_HOOKS: + if hook != "abefore_model": + assert not _langchain_detects(wrapped, hook), hook + + +def test_wrapper_preserves_tools_state_schema_and_transformers(): + """factory.py reads m.tools / m.state_schema / m.transformers off the + wrapper — dropping them unregisters the middleware's contributions.""" + from langchain_core.tools import tool + + @tool + def ext_echo(text: str) -> str: + """Echo the text back.""" + return f"echo:{text}" + + import typing + + class _State(typing.TypedDict, total=False): + seen: str + + def _transformer(scope): + return None + + class _Contributing(AgentMiddleware): + state_schema = _State + transformers = (_transformer,) + + def __init__(self) -> None: + super().__init__() + self.tools = [ext_echo] + + inner = _Contributing() + wrapped = IsolatedMiddleware(inner, "contrib:install", lambda d: None) + assert list(wrapped.tools) == [ext_echo] + assert wrapped.state_schema is _State + assert tuple(wrapped.transformers) == (_transformer,) + + +def test_lifecycle_hooks_delegate_to_the_inner(): + inner = _LifecycleObserver() + wrapped = IsolatedMiddleware(inner, "obs:install", lambda d: None) + assert wrapped.before_model("state", "runtime") == {"seen": "before_model"} + assert wrapped.after_agent("state", "runtime") == {"seen": "after_agent"} + assert asyncio.run(wrapped.abefore_agent("state", "runtime")) == {"seen": "abefore_agent"} + assert asyncio.run(wrapped.aafter_model("state", "runtime")) == {"seen": "aafter_model"} + assert inner.calls == ["before_model", "after_agent", "abefore_agent", "aafter_model"] + + +def test_failing_lifecycle_hook_degrades_to_none_with_a_diagnostic(): + """Lifecycle hooks have no handler to fall through to; the fail-open + degradation is returning no state update.""" + + class _FailingObserver(AgentMiddleware): + def before_model(self, state, runtime): + raise ValueError("observation exploded") + + async def aafter_model(self, state, runtime): + raise ValueError("observation exploded") + + errors = [] + wrapped = IsolatedMiddleware(_FailingObserver(), "bad:install", errors.append) + assert wrapped.before_model("state", "runtime") is None + assert asyncio.run(wrapped.aafter_model("state", "runtime")) is None + assert [d.level for d in errors] == ["error", "error"] + assert "before_model" in errors[0].message + assert "aafter_model" in errors[1].message + + +def test_graph_bubble_up_propagates_from_lifecycle_hooks(): + """Interrupts ride on lifecycle hooks too (human-in-the-loop pauses from + after_model); isolation must not swallow graph control flow.""" + + class _Interrupting(AgentMiddleware): + def after_model(self, state, runtime): + raise GraphBubbleUp() + + async def abefore_model(self, state, runtime): + raise GraphBubbleUp() + + wrapped = IsolatedMiddleware(_Interrupting(), "hitl:install", lambda d: None) + with pytest.raises(GraphBubbleUp): + wrapped.after_model("state", "runtime") + with pytest.raises(GraphBubbleUp): + asyncio.run(wrapped.abefore_model("state", "runtime")) + + +def test_middleware_implements_agrees_with_the_wrapper(): + """Placement-guarantee checks reason about hook participation through + middleware_implements(); the wrapper must not distort it.""" + from deerflow.extensions.stack import middleware_implements + + wrapped = IsolatedMiddleware(_LifecycleObserver(), "obs:install", lambda d: None) + for hook in _LIFECYCLE_HOOKS: + assert middleware_implements(wrapped, hook), hook + assert not middleware_implements(wrapped, "wrap_model_call") + + +def test_create_agent_runs_the_wrapped_hooks_and_registers_the_wrapped_tools(): + """End to end through a real langchain.agents.create_agent graph: the + wrapped middleware's before_model must actually execute and its tools must + actually be callable.""" + from _agent_e2e_helpers import build_single_tool_call_model + from langchain.agents import create_agent + from langchain_core.messages import HumanMessage + from langchain_core.tools import tool + + tool_calls: list[str] = [] + + @tool + def ext_echo(text: str) -> str: + """Echo the text back.""" + tool_calls.append(text) + return f"echo:{text}" + + class _Contributing(AgentMiddleware): + def __init__(self) -> None: + super().__init__() + self.tools = [ext_echo] + self.before_model_calls = 0 + + def before_model(self, state, runtime): + self.before_model_calls += 1 + return None + + inner = _Contributing() + wrapped = IsolatedMiddleware(inner, "contrib:install", lambda d: None) + + model = build_single_tool_call_model(tool_name="ext_echo", tool_args={"text": "hello"}) + agent = create_agent(model=model, tools=[], middleware=[wrapped]) + result = agent.invoke({"messages": [HumanMessage(content="say hello")]}) + + assert inner.before_model_calls > 0, "the wrapped before_model hook never entered the graph" + assert tool_calls == ["hello"], "the wrapped middleware's tool was never registered" + assert any(getattr(m, "content", "") == "echo:hello" for m in result["messages"]) diff --git a/backend/tests/test_extension_loader.py b/backend/tests/test_extension_loader.py new file mode 100644 index 000000000..b5a564ae2 --- /dev/null +++ b/backend/tests/test_extension_loader.py @@ -0,0 +1,381 @@ +"""Tests for config-driven extension loading.""" + +from __future__ import annotations + +import pytest + +from deerflow.extensions.loader import ( + Diagnostic, + ExtensionLoadError, + ExtensionSpec, + load_extensions, +) +from extension_test_fixtures import demo_extensions + +_FIXTURE = "extension_test_fixtures.demo_extensions" + + +@pytest.fixture(autouse=True) +def _reset_fixture_state(): + demo_extensions.INSTALLED.clear() + yield + demo_extensions.INSTALLED.clear() + + +def test_no_specs_yields_empty_result(): + loaded, diagnostics = load_extensions([]) + assert diagnostics == [] + assert loaded.has_middleware_contributors is False + + +def test_successful_install_registers_and_attributes(): + spec = ExtensionSpec(use=f"{_FIXTURE}:install_ok") + loaded, diagnostics = load_extensions([spec]) + assert diagnostics == [] + assert demo_extensions.INSTALLED == ["ok"] + assert loaded.middleware_contributors[0][0] == f"{_FIXTURE}:install_ok" + + +def test_config_block_is_passed_through_verbatim(): + spec = ExtensionSpec(use=f"{_FIXTURE}:install_reads_config", config={"mode": "fast"}) + load_extensions([spec]) + assert demo_extensions.INSTALLED == ["config:fast"] + + +def test_disabled_extension_registers_nothing(): + spec = ExtensionSpec(use=f"{_FIXTURE}:install_disabled", config={"enabled": False}) + loaded, diagnostics = load_extensions([spec]) + assert diagnostics == [] + assert loaded.has_middleware_contributors is False + + +def test_load_order_follows_config_order(): + specs = [ + ExtensionSpec(use=f"{_FIXTURE}:install_ok"), + ExtensionSpec(use=f"{_FIXTURE}:install_stamped"), + ] + load_extensions(specs) + assert demo_extensions.INSTALLED == ["ok", "stamped"] + + +def test_unresolvable_entry_point_is_skipped_with_an_error_diagnostic(): + specs = [ + ExtensionSpec(use="extension_test_fixtures.demo_extensions:does_not_exist"), + ExtensionSpec(use=f"{_FIXTURE}:install_ok"), + ] + loaded, diagnostics = load_extensions(specs) + assert [d.level for d in diagnostics] == ["error"] + assert "does_not_exist" in diagnostics[0].source + assert demo_extensions.INSTALLED == ["ok"], "a broken extension must not stop the rest" + + +def test_non_callable_entry_point_is_rejected(): + spec = ExtensionSpec(use=f"{_FIXTURE}:NOT_CALLABLE") + loaded, diagnostics = load_extensions([spec]) + assert diagnostics[0].level == "error" + assert "callable" in diagnostics[0].message + + +def test_install_failure_rolls_back_partial_registration(): + specs = [ + ExtensionSpec(use=f"{_FIXTURE}:install_partial_then_raise"), + ExtensionSpec(use=f"{_FIXTURE}:install_ok"), + ] + loaded, diagnostics = load_extensions(specs) + assert diagnostics[0].level == "error" + assert "boom" in diagnostics[0].message + sources = {source for source, _ in loaded.middleware_contributors} + assert sources == {f"{_FIXTURE}:install_ok"} + assert len(loaded.middleware_contributors) == 1, "rollback must clear every partial registration" + + +def test_rollback_does_not_remove_a_different_specs_registrations_sharing_the_same_use(): + """Two specs may legitimately share `use` with different config (e.g. the + same extension mounted twice with different settings). Rollback on the + second's install failure must be positional, not keyed by `use` — it must + not erase the first instance's already-successful registrations just + because they share a source string.""" + specs = [ + ExtensionSpec(use=f"{_FIXTURE}:install_shared_use", config={"label": "first"}), + ExtensionSpec(use=f"{_FIXTURE}:install_shared_use", config={"label": "second", "fail": True}), + ] + loaded, diagnostics = load_extensions(specs) + assert [d.level for d in diagnostics] == ["error"] + assert "boom-shared" in diagnostics[0].message + assert len(loaded.middleware_contributors) == 1 + source, contributor = loaded.middleware_contributors[0] + assert source == f"{_FIXTURE}:install_shared_use" + assert contributor.tag == "shared:first" + + +def test_required_extension_failure_aborts_startup(): + spec = ExtensionSpec(use=f"{_FIXTURE}:install_partial_then_raise", required=True) + with pytest.raises(ExtensionLoadError): + load_extensions([spec]) + + +def test_required_unresolvable_extension_aborts_startup(): + spec = ExtensionSpec(use="nope.nothing:here", required=True) + with pytest.raises(ExtensionLoadError): + load_extensions([spec]) + + +def test_incompatible_declared_api_is_refused_with_actionable_message(): + spec = ExtensionSpec(use=f"{_FIXTURE}:install_future_api") + loaded, diagnostics = load_extensions([spec]) + assert diagnostics[0].level == "error" + assert "99.0" in diagnostics[0].message + assert "pip install" in diagnostics[0].message + assert demo_extensions.INSTALLED == [], "an incompatible extension must not run" + + +def test_optional_extension_with_non_string_api_marker_is_skipped_with_a_diagnostic(monkeypatch): + monkeypatch.setattr(demo_extensions.install_ok, "__deerflow_api__", 101, raising=False) + spec = ExtensionSpec(use=f"{_FIXTURE}:install_ok") + + loaded, diagnostics = load_extensions([spec]) + + assert loaded.has_middleware_contributors is False + assert demo_extensions.INSTALLED == [], "an invalid API marker must be rejected before install()" + assert len(diagnostics) == 1 + assert diagnostics[0].level == "error" + assert diagnostics[0].source == spec.use + assert "invalid extension-api version marker" in diagnostics[0].message + assert "int" in diagnostics[0].message + + +def test_required_extension_with_non_string_iterable_api_marker_fails_closed(monkeypatch): + class _IterableAPIMarker: + def split(self, separator: str) -> list[object]: + return [object()] + + def __str__(self) -> str: + return "non-string iterable marker" + + monkeypatch.setattr( + demo_extensions.install_ok, + "__deerflow_api__", + _IterableAPIMarker(), + raising=False, + ) + spec = ExtensionSpec( + use=f"{_FIXTURE}:install_ok", + required=True, + ) + + with pytest.raises(ExtensionLoadError, match="declares invalid api marker"): + load_extensions([spec]) + + assert demo_extensions.INSTALLED == [], "an invalid API marker must be rejected before install()" + + +def test_optional_extension_with_unrenderable_api_marker_still_returns_a_diagnostic(monkeypatch): + class _UnrenderableAPIMarker: + def __str__(self) -> str: + raise RuntimeError("API marker string rendering exploded") + + monkeypatch.setattr( + demo_extensions.install_ok, + "__deerflow_api__", + _UnrenderableAPIMarker(), + raising=False, + ) + spec = ExtensionSpec(use=f"{_FIXTURE}:install_ok") + + loaded, diagnostics = load_extensions([spec]) + + assert loaded.has_middleware_contributors is False + assert demo_extensions.INSTALLED == [] + assert len(diagnostics) == 1 + assert diagnostics[0].level == "error" + assert "invalid extension-api version marker" in diagnostics[0].message + assert "_UnrenderableAPIMarker" in diagnostics[0].message + + +@pytest.mark.parametrize("required", [False, True]) +def test_extension_api_marker_getter_failure_obeys_required_policy(monkeypatch, required): + class _ExplodingMarkerInstall: + @property + def __deerflow_api__(self): + raise RuntimeError("API marker getter exploded") + + def __call__(self, registry, config): + raise AssertionError("install must not run after marker inspection fails") + + monkeypatch.setattr( + "deerflow.extensions.loader.resolve_variable", + lambda path: _ExplodingMarkerInstall(), + ) + spec = ExtensionSpec(use="hostile_extension:install", required=required) + + if required: + with pytest.raises(ExtensionLoadError, match="could not inspect api marker"): + load_extensions([spec]) + return + + loaded, diagnostics = load_extensions([spec]) + assert loaded.has_middleware_contributors is False + assert len(diagnostics) == 1 + assert "could not inspect extension-api version marker" in diagnostics[0].message + + +def test_string_subclass_api_marker_cannot_break_incompatibility_diagnostics(monkeypatch): + class _HostileString(str): + def split(self, separator: str): + raise RuntimeError("API marker split exploded") + + def __str__(self) -> str: + raise RuntimeError("API marker string rendering exploded") + + def __format__(self, format_spec: str) -> str: + raise RuntimeError("API marker formatting exploded") + + monkeypatch.setattr( + demo_extensions.install_ok, + "__deerflow_api__", + _HostileString("99.0"), + raising=False, + ) + spec = ExtensionSpec(use=f"{_FIXTURE}:install_ok") + + loaded, diagnostics = load_extensions([spec]) + + assert loaded.has_middleware_contributors is False + assert demo_extensions.INSTALLED == [] + assert len(diagnostics) == 1 + assert "99.0" in diagnostics[0].message + + +def test_compatible_string_subclass_api_marker_can_load(monkeypatch): + class _HostileString(str): + def split(self, separator: str): + raise RuntimeError("API marker split exploded") + + def __str__(self) -> str: + raise RuntimeError("API marker string rendering exploded") + + def __format__(self, format_spec: str) -> str: + raise RuntimeError("API marker formatting exploded") + + monkeypatch.setattr( + demo_extensions.install_ok, + "__deerflow_api__", + _HostileString("0.1.0"), + raising=False, + ) + + loaded, diagnostics = load_extensions([ExtensionSpec(use=f"{_FIXTURE}:install_ok")]) + + assert diagnostics == [] + assert loaded.has_middleware_contributors is True + assert demo_extensions.INSTALLED == ["ok"] + + +def test_newer_minor_declared_api_is_refused(): + """Before 1.0, minors carry no compatibility promise: an extension written + against 0.2 may use contracts a 0.1 host does not implement, and the host + must refuse it with an actionable message.""" + spec = ExtensionSpec(use=f"{_FIXTURE}:install_newer_minor_api") + loaded, diagnostics = load_extensions([spec]) + assert diagnostics[0].level == "error" + assert "0.2" in diagnostics[0].message + assert "pip install" in diagnostics[0].message + assert demo_extensions.INSTALLED == [], "a newer-minor extension must not run on an older host" + + +def test_newer_minor_required_extension_aborts_startup(): + spec = ExtensionSpec(use=f"{_FIXTURE}:install_newer_minor_api", required=True) + with pytest.raises(ExtensionLoadError): + load_extensions([spec]) + + +def test_compatible_declared_api_loads(): + spec = ExtensionSpec(use=f"{_FIXTURE}:install_stamped") + loaded, diagnostics = load_extensions([spec]) + assert diagnostics == [] + assert demo_extensions.INSTALLED == ["stamped"] + + +def test_compatible_follows_semver_windows(): + """0.x: minors may break — the window is same major.minor with patches + additive (host >= declared). From 1.0 on: contracts only grow within a + major. Comparisons are numeric (1.10 > 1.9), not lexicographic.""" + from deerflow.extensions.loader import _compatible + + # 0.x window: same major.minor, patch-level growth only. + assert _compatible("0.1", "0.1") + assert _compatible("0.1", "0.1.1"), "patch growth stays compatible" + assert not _compatible("0.1.1", "0.1"), "a newer patch declaration exceeds what the host provides" + assert not _compatible("0.2", "0.1"), "0.x minors may break: a 0.1 host must refuse 0.2 extensions" + assert not _compatible("0.1", "0.2"), "0.x minors promise nothing in the other direction either" + + # 1.x+ window: same major, contracts only grow. + assert _compatible("1.0", "1.0") + assert _compatible("1.0", "1.1"), "a newer host still provides everything a 1.0 extension declared" + assert _compatible("1.9", "1.10"), "minor comparison is numeric, not lexicographic" + assert not _compatible("1.1", "1.0"), "the 1.0 host lacks the 1.1 contract additions" + assert not _compatible("1.10", "1.9") + assert not _compatible("1.0.1", "1.0"), "even a newer patch declaration exceeds what the host provides" + assert not _compatible("2.0", "1.5"), "major mismatch" + assert not _compatible("1.0", "2.0"), "major mismatch" + assert not _compatible("not-a-version", "1.0"), "unparseable versions are refused, not waved through" + + +def test_undeclared_api_is_allowed(): + """The decorator is optional; pip constraints remain the primary gate.""" + spec = ExtensionSpec(use=f"{_FIXTURE}:install_ok") + _, diagnostics = load_extensions([spec]) + assert diagnostics == [] + + +def test_a_successful_load_is_reported(caplog): + """Every other branch is failure-only, so without this line an operator has + no way to tell a clean load from a `plugins:` block the host never read.""" + with caplog.at_level("INFO", logger="deerflow.extensions.loader"): + load_extensions([ExtensionSpec(use=f"{_FIXTURE}:install_ok")]) + + assert f"Extensions loaded: 1/1 ({_FIXTURE}:install_ok)" in caplog.text + + +def test_the_report_counts_skipped_extensions_apart_from_loaded_ones(caplog): + specs = [ + ExtensionSpec(use=f"{_FIXTURE}:install_ok"), + ExtensionSpec(use="does.not.exist:install"), + ] + with caplog.at_level("INFO", logger="deerflow.extensions.loader"): + load_extensions(specs) + + assert f"Extensions loaded: 1/2 ({_FIXTURE}:install_ok)" in caplog.text + + +def test_an_all_failed_load_reports_none_rather_than_an_empty_list(caplog): + with caplog.at_level("INFO", logger="deerflow.extensions.loader"): + load_extensions([ExtensionSpec(use="does.not.exist:install")]) + + assert "Extensions loaded: 0/1 (none)" in caplog.text + + +def test_no_configured_plugins_stays_off_the_info_log(caplog): + """The default state for nearly every deployment; a line here is boot noise.""" + with caplog.at_level("INFO", logger="deerflow.extensions.loader"): + load_extensions([]) + + assert "Extensions loaded" not in caplog.text + + +def test_diagnostic_helpers_set_level(): + assert Diagnostic.error("s", "m").level == "error" + assert Diagnostic.warning("s", "m").level == "warning" + assert Diagnostic.info("s", "m").level == "info" + assert Diagnostic.debug("s", "m").level == "debug" + + +def test_host_registry_satisfies_the_public_contract(): + """Extensions annotate install(registry: ExtensionRegistry, ...) against + the contract package alone; the host's concrete registry must satisfy that + Protocol, or every correctly-annotated extension is lying about its types.""" + from deerflow_extension_api import ExtensionRegistry as ContractRegistry + + from deerflow.extensions.registry import ExtensionRegistry as HostRegistry + + assert isinstance(HostRegistry(), ContractRegistry) diff --git a/backend/tests/test_extension_ordering.py b/backend/tests/test_extension_ordering.py new file mode 100644 index 000000000..16841fdc4 --- /dev/null +++ b/backend/tests/test_extension_ordering.py @@ -0,0 +1,155 @@ +"""Tests for declarative middleware ordering constraints.""" + +from __future__ import annotations + +import pytest + +from deerflow.extensions.isolation import IsolatedMiddleware +from deerflow.extensions.ordering import OrderingConstraint, assert_ordering + + +class _Outer: + pass + + +class _Inner: + pass + + +class _Unrelated: + pass + + +_CONSTRAINTS = (OrderingConstraint(outer=_Outer, inner=_Inner, reason="outer must wrap inner"),) + + +def test_correct_order_passes(): + assert_ordering([_Outer(), _Inner()], {}, _CONSTRAINTS) + + +def test_reversed_order_raises(): + with pytest.raises(RuntimeError, match="outer must wrap inner"): + assert_ordering([_Inner(), _Outer()], {}, _CONSTRAINTS) + + +def test_missing_participant_is_not_a_violation(): + """The stack is conditionally built; an absent middleware means the + constraint simply does not apply.""" + assert_ordering([_Outer(), _Unrelated()], {}, _CONSTRAINTS) + assert_ordering([_Unrelated()], {}, _CONSTRAINTS) + + +def test_violation_message_names_the_responsible_extension(): + """Without attribution, an operator cannot tell which extension to remove.""" + stack = [_Inner(), _Outer()] + provenance = {0: "bad_ext:install"} + with pytest.raises(RuntimeError) as excinfo: + assert_ordering(stack, provenance, _CONSTRAINTS) + assert "bad_ext:install" in str(excinfo.value) + + +def test_violation_without_extensions_says_core(): + stack = [_Inner(), _Outer()] + with pytest.raises(RuntimeError) as excinfo: + assert_ordering(stack, {}, _CONSTRAINTS) + assert "core" in str(excinfo.value).lower() + + +def test_violation_does_not_blame_an_uninvolved_extension(): + """Only the two positions in the violating pair can be responsible. A + provenance entry for some other index — an extension that contributed + elsewhere in the stack but is not part of this constraint — must not be + named, even though it is the only entry in `provenance`.""" + stack = [_Inner(), _Outer(), _Unrelated()] + provenance = {2: "innocent_ext:install"} + with pytest.raises(RuntimeError) as excinfo: + assert_ordering(stack, provenance, _CONSTRAINTS) + message = str(excinfo.value) + assert "innocent_ext:install" not in message + assert "core" in message.lower() + + +def test_reversed_order_raises_when_a_participant_is_wrapped(): + """Extension-contributed middlewares are wrapped in IsolatedMiddleware before + they reach the merged stack. If _index_of matched only the wrapper's own + type, a wrapped participant would read as absent and the constraint would + silently stop being enforced instead of raising — the worst possible failure + mode for a safety check.""" + stack = [IsolatedMiddleware(_Inner(), "bad_ext:install", lambda d: None), _Outer()] + with pytest.raises(RuntimeError, match="outer must wrap inner"): + assert_ordering(stack, {}, _CONSTRAINTS) + + +def test_every_duplicate_participant_must_satisfy_the_constraint(): + """A valid first pair must not hide a later contributed violation.""" + stack = [_Outer(), _Inner(), _Outer()] + + with pytest.raises(RuntimeError) as exc_info: + assert_ordering(stack, {2: "late:install"}, _CONSTRAINTS) + + assert "late:install" in str(exc_info.value) + + +def test_core_constraints_are_declared(): + from deerflow.agents.middlewares.tool_error_handling_middleware import ToolErrorHandlingMiddleware + from deerflow.agents.middlewares.tool_progress_middleware import ToolProgressMiddleware + from deerflow.extensions.ordering import core_ordering_constraints + + pairs = {(c.outer, c.inner) for c in core_ordering_constraints()} + assert (ToolProgressMiddleware, ToolErrorHandlingMiddleware) in pairs + + +def test_core_constraints_are_a_plain_tuple(): + """Deferred resolution must not be paid for with an object that reports one + thing when iterated and another when measured. + + The predecessor was a ``tuple`` subclass whose backing storage stayed empty + (a tuple cannot fill its own storage after construction), so ``len`` was 0, + ``bool`` was False, ``in`` was always False and it compared unequal to the + very tuples tests substitute for it — while iteration yielded the real + constraints. + """ + from deerflow.extensions.ordering import core_ordering_constraints + + constraints = core_ordering_constraints() + iterated = list(constraints) + + assert type(constraints) is tuple + assert len(constraints) == len(iterated) + assert bool(constraints) is bool(iterated) + assert constraints == tuple(iterated) + assert all(constraint in constraints for constraint in iterated) + assert constraints[0] is iterated[0] + assert list(reversed(constraints)) == list(reversed(iterated)) + + +def test_resolution_stays_deferred_until_first_use(): + """Importing this module must not drag in the middleware layer. + + ``extensions/`` is the layer the middleware layer calls into, so resolving + the table at import time would give it a backward dependency on the layer + it exists to serve — an import cycle waiting for the first middleware that + imports anything under ``extensions/`` at module level. Resolution belongs + at ``assert_ordering`` time, which already runs inside the middleware + builder. + """ + import os + import subprocess + import sys + from pathlib import Path + + backend_root = Path(__file__).resolve().parents[1] + env = {**os.environ, "PYTHONPATH": os.pathsep.join([str(backend_root), str(backend_root / "packages" / "harness"), os.environ.get("PYTHONPATH", "")])} + probe = ( + "import sys\n" + "from deerflow.extensions import ordering\n" + "targets = ('deerflow.agents.middlewares.tool_progress_middleware', 'deerflow.agents.middlewares.tool_error_handling_middleware')\n" + "print('after_import', [t for t in targets if t in sys.modules])\n" + "ordering.core_ordering_constraints()\n" + "print('after_call', sorted(t for t in targets if t in sys.modules))\n" + ) + result = subprocess.run([sys.executable, "-c", probe], capture_output=True, text=True, env=env) + + assert result.returncode == 0, result.stderr + assert "after_import []" in result.stdout, "importing extensions.ordering must not load the middleware layer" + assert "after_call ['deerflow.agents.middlewares.tool_error_handling_middleware', 'deerflow.agents.middlewares.tool_progress_middleware']" in result.stdout diff --git a/backend/tests/test_extension_placement_guarantees.py b/backend/tests/test_extension_placement_guarantees.py new file mode 100644 index 000000000..e4e9b8942 --- /dev/null +++ b/backend/tests/test_extension_placement_guarantees.py @@ -0,0 +1,262 @@ +"""The guarantee each Placement makes must be met by the real stack. + +Neither the type system nor the version number can catch a broken guarantee: +if a new request-transforming middleware is appended inner of the +MODEL_PHYSICAL anchor, the anchor table, the types and the pip constraints all +stay valid while the promise silently stops holding. These tests are the only +thing standing between that change and a released extension observing the wrong +data. +""" + +from __future__ import annotations + +from deerflow_extension_api import AgentScope, MiddlewarePlacement, Placement +from langchain.agents.middleware import AgentMiddleware + +from deerflow.agents.lead_agent.agent import build_middlewares +from deerflow.agents.middlewares.tool_error_handling_middleware import ( + build_subagent_runtime_middlewares, +) +from deerflow.config.app_config import AppConfig +from deerflow.config.extensions_config import ExtensionsConfig +from deerflow.config.sandbox_config import SandboxConfig +from deerflow.extensions.isolation import IsolatedMiddleware +from deerflow.extensions.registry import ExtensionRegistry +from deerflow.extensions.stack import middleware_implements + + +class _Probe(AgentMiddleware): + def __init__(self, tag: str) -> None: + super().__init__() + self.tag = tag + + +def _extensions(*placements: MiddlewarePlacement): + class _C: + def contribute_middlewares(self, app_store, ctx): + return placements + + registry = ExtensionRegistry() + with registry.attributed_to("probe:install"): + registry.middlewares(_C()) + return registry.build() + + +def _stack_with(*placements: MiddlewarePlacement): + return build_middlewares( + config={"configurable": {}}, + model_name="gpt-4o", + app_config=AppConfig(sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider")), + extensions=_extensions(*placements), + ) + + +def _subagent_stack_with(*placements: MiddlewarePlacement): + return build_subagent_runtime_middlewares( + app_config=AppConfig(sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider")), + model_name="gpt-4o", + extensions=_extensions(*placements), + ) + + +def _index_of_probe(stack, tag: str) -> int: + for index, middleware in enumerate(stack): + target = middleware.inner if isinstance(middleware, IsolatedMiddleware) else middleware + if isinstance(target, _Probe) and target.tag == tag: + return index + raise AssertionError(f"probe {tag!r} not found in stack") + + +def _unwrap(middleware): + return middleware.inner if isinstance(middleware, IsolatedMiddleware) else middleware + + +def test_model_physical_sees_the_final_request(): + """Nothing inner of MODEL_PHYSICAL may transform the model request.""" + stack = _stack_with(MiddlewarePlacement(_Probe("physical"), Placement.MODEL_PHYSICAL)) + index = _index_of_probe(stack, "physical") + offenders = [type(_unwrap(m)).__name__ for m in stack[index + 1 :] if middleware_implements(_unwrap(m), "wrap_model_call")] + assert offenders == [], ( + f"these middlewares sit inner of the MODEL_PHYSICAL anchor and wrap model calls, breaking its documented guarantee: {offenders}. Either move them outer of the anchor or update the anchor table in deerflow/extensions/stack.py." + ) + + +def test_lead_model_physical_uses_the_innermost_tail_anchor(): + """A configured duplicate must not capture the lead's type-based anchor.""" + app_config = AppConfig(sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider")) + app_config.extensions = ExtensionsConfig(middlewares=["deerflow.agents.middlewares.safety_finish_reason_middleware:SafetyFinishReasonMiddleware"]) + stack = build_middlewares( + config={"configurable": {}}, + model_name="gpt-4o", + app_config=app_config, + extensions=_extensions( + MiddlewarePlacement( + _Probe("physical"), + Placement.MODEL_PHYSICAL, + scope=AgentScope.LEAD, + ) + ), + ) + + index = _index_of_probe(stack, "physical") + offenders = [type(_unwrap(m)).__name__ for m in stack[index + 1 :] if middleware_implements(_unwrap(m), "wrap_model_call")] + assert offenders == [] + + +def test_lead_model_physical_reports_when_the_safety_anchor_is_disabled(): + from deerflow.extensions import get_runtime_diagnostics, reset_runtime_diagnostics + + app_config = AppConfig(sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider")) + app_config.safety_finish_reason.enabled = False + reset_runtime_diagnostics() + try: + build_middlewares( + config={"configurable": {}}, + model_name="gpt-4o", + app_config=app_config, + extensions=_extensions( + MiddlewarePlacement( + _Probe("physical"), + Placement.MODEL_PHYSICAL, + scope=AgentScope.LEAD, + ) + ), + ) + diagnostics = get_runtime_diagnostics() + finally: + reset_runtime_diagnostics() + + assert any("MODEL_PHYSICAL fell back to a secondary anchor" in diagnostic.message for diagnostic in diagnostics) + + +def test_subagent_model_physical_sees_the_final_request(): + """Subagents provide the same final-request guarantee as the lead agent.""" + stack = _subagent_stack_with( + MiddlewarePlacement( + _Probe("physical"), + Placement.MODEL_PHYSICAL, + scope=AgentScope.SUBAGENT, + ) + ) + index = _index_of_probe(stack, "physical") + offenders = [type(_unwrap(m)).__name__ for m in stack[index + 1 :] if middleware_implements(_unwrap(m), "wrap_model_call")] + assert offenders == [], f"these subagent middlewares sit inner of the MODEL_PHYSICAL anchor and wrap model calls, breaking its documented guarantee: {offenders}" + + +def test_subagent_model_physical_uses_the_innermost_core_coalescer(): + """A configured duplicate must not capture the type-based primary anchor.""" + app_config = AppConfig(sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider")) + app_config.extensions = ExtensionsConfig(middlewares=["deerflow.agents.middlewares.system_message_coalescing_middleware:SystemMessageCoalescingMiddleware"]) + stack = build_subagent_runtime_middlewares( + app_config=app_config, + model_name="gpt-4o", + extensions=_extensions( + MiddlewarePlacement( + _Probe("physical"), + Placement.MODEL_PHYSICAL, + scope=AgentScope.SUBAGENT, + ) + ), + ) + + index = _index_of_probe(stack, "physical") + offenders = [type(_unwrap(m)).__name__ for m in stack[index + 1 :] if middleware_implements(_unwrap(m), "wrap_model_call")] + assert offenders == [] + + +#: The single documented middleware allowed inner of TOOL_RAW. +#: +#: ClarificationMiddleware must stay last — it short-circuits the tool loop with +#: Command(goto=END) — and it only intercepts ask_clarification, never +#: transforming the result of a tool that actually executes. So TOOL_RAW still +#: sees raw results. This carve-out is named explicitly rather than the +#: assertion being loosened: any OTHER middleware appearing here is a real +#: regression, and adding to this set must be a deliberate, argued change. +_TOOL_RAW_CARVE_OUT = {"ClarificationMiddleware"} + + +def test_tool_raw_is_adjacent_to_the_callable_boundary(): + """Nothing inner of TOOL_RAW may wrap tool calls, bar one documented case.""" + stack = _stack_with(MiddlewarePlacement(_Probe("raw"), Placement.TOOL_RAW)) + index = _index_of_probe(stack, "raw") + offenders = [name for m in stack[index + 1 :] if middleware_implements(_unwrap(m), "wrap_tool_call") and (name := type(_unwrap(m)).__name__) not in _TOOL_RAW_CARVE_OUT] + assert offenders == [], ( + f"these middlewares sit inner of TOOL_RAW and wrap tool calls: {offenders}. " + "Either move them outer of the anchor or update the anchor table in " + "deerflow/extensions/stack.py — do not add them to the carve-out without " + "an argument for why TOOL_RAW still sees raw results." + ) + + +def test_the_tool_raw_carve_out_is_actually_needed(): + """Guards the carve-out itself: if ClarificationMiddleware ever stops + sitting inner of TOOL_RAW, the exemption is dead and must be deleted rather + than quietly hiding a future regression.""" + stack = _stack_with(MiddlewarePlacement(_Probe("raw"), Placement.TOOL_RAW)) + index = _index_of_probe(stack, "raw") + inner = {type(_unwrap(m)).__name__ for m in stack[index + 1 :]} + assert _TOOL_RAW_CARVE_OUT <= inner, f"the carve-out lists middlewares that are no longer inner of TOOL_RAW: {_TOOL_RAW_CARVE_OUT - inner}" + + +def test_tool_visible_sees_the_final_result(): + """Nothing outer of TOOL_VISIBLE may wrap tool calls.""" + stack = _stack_with(MiddlewarePlacement(_Probe("visible"), Placement.TOOL_VISIBLE)) + index = _index_of_probe(stack, "visible") + offenders = [type(_unwrap(m)).__name__ for m in stack[:index] if middleware_implements(_unwrap(m), "wrap_tool_call")] + assert offenders == [], f"these middlewares sit outer of TOOL_VISIBLE and wrap tool calls: {offenders}" + + +def test_model_logical_is_outer_of_the_retry_loop(): + """One logical decision must stay one event across provider retries.""" + from deerflow.agents.middlewares.llm_error_handling_middleware import LLMErrorHandlingMiddleware + + stack = _stack_with(MiddlewarePlacement(_Probe("logical"), Placement.MODEL_LOGICAL)) + logical = _index_of_probe(stack, "logical") + retry = next(index for index, m in enumerate(stack) if isinstance(_unwrap(m), LLMErrorHandlingMiddleware)) + assert logical < retry, "MODEL_LOGICAL must be outer of the retry middleware" + + +def test_logical_and_physical_nest_in_the_right_order(): + """Step:Attempt is 1:N — the logical probe must enclose the physical one.""" + stack = _stack_with( + MiddlewarePlacement(_Probe("logical"), Placement.MODEL_LOGICAL), + MiddlewarePlacement(_Probe("physical"), Placement.MODEL_PHYSICAL), + ) + assert _index_of_probe(stack, "logical") < _index_of_probe(stack, "physical") + + +def test_visible_and_raw_bracket_the_tool_chain(): + """Visible/raw form a pair around truncation and sanitization; that gap is + what lets an extension see how much a tool result was altered.""" + stack = _stack_with( + MiddlewarePlacement(_Probe("visible"), Placement.TOOL_VISIBLE), + MiddlewarePlacement(_Probe("raw"), Placement.TOOL_RAW), + ) + visible = _index_of_probe(stack, "visible") + raw = _index_of_probe(stack, "raw") + assert visible < raw + between = [type(_unwrap(m)).__name__ for m in stack[visible + 1 : raw] if middleware_implements(_unwrap(m), "wrap_tool_call")] + assert between, "the tool-processing chain must sit between the two probes" + + +def test_all_four_probes_coexist_without_violating_ordering(): + stack = _stack_with( + MiddlewarePlacement(_Probe("visible"), Placement.TOOL_VISIBLE), + MiddlewarePlacement(_Probe("logical"), Placement.MODEL_LOGICAL), + MiddlewarePlacement(_Probe("physical"), Placement.MODEL_PHYSICAL), + MiddlewarePlacement(_Probe("raw"), Placement.TOOL_RAW), + ) + indices = [_index_of_probe(stack, tag) for tag in ("visible", "logical", "physical", "raw")] + assert len(set(indices)) == 4 + + +def test_middleware_implements_detects_overrides(): + class _Wraps(AgentMiddleware): + def wrap_tool_call(self, request, handler): + return handler(request) + + class _Plain(AgentMiddleware): + pass + + assert middleware_implements(_Wraps(), "wrap_tool_call") is True + assert middleware_implements(_Plain(), "wrap_tool_call") is False diff --git a/backend/tests/test_extension_registry.py b/backend/tests/test_extension_registry.py new file mode 100644 index 000000000..088420e22 --- /dev/null +++ b/backend/tests/test_extension_registry.py @@ -0,0 +1,86 @@ +"""Tests for the extension registry and its immutable build product.""" + +from __future__ import annotations + +import pytest + +from deerflow.extensions.registry import EMPTY_EXTENSIONS, ExtensionRegistry + + +class _Contributor: + def __init__(self, tag: str = "") -> None: + self.tag = tag + + +def test_empty_registry_builds_with_all_flags_false(): + loaded = ExtensionRegistry().build() + assert loaded.has_middleware_contributors is False + assert loaded.needs_task_store is False + + +def test_empty_singleton_matches_an_empty_build(): + assert EMPTY_EXTENSIONS.has_middleware_contributors is False + assert EMPTY_EXTENSIONS.needs_task_store is False + + +def test_middleware_contributions_require_a_task_store(): + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_Contributor()) + + assert registry.build().needs_task_store is True + + +def test_entries_carry_their_source(): + registry = ExtensionRegistry() + contributor = _Contributor("mw") + with registry.attributed_to("demo_ext:install"): + registry.middlewares(contributor) + loaded = registry.build() + assert loaded.middleware_contributors == (("demo_ext:install", contributor),) + assert loaded.has_middleware_contributors is True + + +def test_registration_order_is_preserved(): + registry = ExtensionRegistry() + first, second = _Contributor("a"), _Contributor("b") + with registry.attributed_to("a_ext:install"): + registry.middlewares(first) + with registry.attributed_to("b_ext:install"): + registry.middlewares(second) + loaded = registry.build() + assert [source for source, _ in loaded.middleware_contributors] == ["a_ext:install", "b_ext:install"] + + +def test_discard_removes_every_entry_of_one_source(): + """A partially-registered extension is worse than an absent one: the data + it produces looks complete but is not.""" + registry = ExtensionRegistry() + keep, drop = _Contributor("keep"), _Contributor("drop") + with registry.attributed_to("good:install"): + registry.middlewares(keep) + with registry.attributed_to("bad:install"): + registry.middlewares(drop) + registry.discard("bad:install") + loaded = registry.build() + assert loaded.middleware_contributors == (("good:install", keep),) + + +def test_registering_outside_attributed_to_raises(): + registry = ExtensionRegistry() + with pytest.raises(RuntimeError, match="attributed_to"): + registry.middlewares(_Contributor()) + + +def test_build_result_is_frozen(): + loaded = ExtensionRegistry().build() + with pytest.raises(Exception): + loaded.middleware_contributors = () # type: ignore[misc] + + +def test_app_store_is_created_at_build_time(): + """The app store must exist before binding so the registration phase and + the binding phase see the same object.""" + loaded = ExtensionRegistry().build() + assert loaded.app_store is not None + assert loaded.app_store.scope_id == "app" diff --git a/backend/tests/test_extension_stack_wiring.py b/backend/tests/test_extension_stack_wiring.py new file mode 100644 index 000000000..4f7cdc772 --- /dev/null +++ b/backend/tests/test_extension_stack_wiring.py @@ -0,0 +1,410 @@ +"""Tests for wiring extension contributions into the real middleware builders.""" + +from __future__ import annotations + +import pytest +from deerflow_extension_api import AgentScope, MiddlewarePlacement, Placement +from langchain.agents.middleware import AgentMiddleware + +from deerflow.agents.lead_agent.agent import build_middlewares +from deerflow.config.app_config import AppConfig +from deerflow.config.sandbox_config import SandboxConfig +from deerflow.extensions.isolation import IsolatedMiddleware +from deerflow.extensions.registry import ExtensionRegistry +from deerflow.extensions.stack import PLACEMENT_ANCHORS + + +def _app_config() -> AppConfig: + # AppConfig.sandbox has no default (`use` is a required field), so a bare + # AppConfig() always fails pydantic validation in this repo. The brief's + # verbatim test code assumed a default-constructible AppConfig; every + # other builder test in this suite (e.g. test_lead_agent_model_resolution.py) + # supplies this same minimal sandbox stanza for the same reason. + return AppConfig(sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider")) + + +class _Probe(AgentMiddleware): + def __init__(self, tag: str) -> None: + super().__init__() + self.tag = tag + + +def _extensions(*placements: MiddlewarePlacement): + class _C: + def contribute_middlewares(self, app_store, ctx): + return placements + + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_C()) + return registry.build() + + +def _tags(stack): + out = [] + for m in stack: + target = m.inner if isinstance(m, IsolatedMiddleware) else m + out.append(target.tag if isinstance(target, _Probe) else type(target).__name__) + return out + + +def _lead_stack(extensions=None, app_config=None, configurable=None): + return build_middlewares( + config={"configurable": configurable or {}}, + model_name="gpt-4o", + app_config=app_config or _app_config(), + extensions=extensions, + ) + + +def test_anchor_table_covers_every_placement(): + assert set(PLACEMENT_ANCHORS) == set(Placement) + + +def test_zero_extensions_leaves_the_stack_unchanged(): + baseline = _tags(_lead_stack()) + with_empty = _tags(_lead_stack(ExtensionRegistry().build())) + assert baseline == with_empty + assert not any(isinstance(m, IsolatedMiddleware) for m in _lead_stack()) + + +def test_zero_extensions_skip_policy_projection(monkeypatch): + from deerflow.agents.middlewares.tool_error_handling_middleware import ( + build_subagent_runtime_middlewares, + ) + from deerflow.extensions import policy as policy_module + + def _unexpected_projection(app_config): + raise AssertionError("zero-extension path constructed an extension payload") + + monkeypatch.setattr(policy_module, "project_host_policy", _unexpected_projection) + empty = ExtensionRegistry().build() + + _lead_stack(empty) + build_subagent_runtime_middlewares(app_config=_app_config(), extensions=empty) + + +def test_zero_extension_composition_reuses_the_built_stack(): + from deerflow_extension_api import AgentBuildContext + + from deerflow.extensions.stack import compose_with_extensions + + middlewares = [] + result = compose_with_extensions( + middlewares, + AgentScope.LEAD, + AgentBuildContext(scope=AgentScope.LEAD), + ExtensionRegistry().build(), + ) + + assert result is middlewares + + +def test_bound_build_snapshot_is_used_by_lead_and_subagent_fallbacks(): + from deerflow.agents.middlewares.tool_error_handling_middleware import ( + build_subagent_runtime_middlewares, + ) + from deerflow.extensions import bind_agent_build_extensions + + lead_capture = _CtxCapture() + subagent_capture = _CtxCapture() + lead_extensions = _extensions_with_contributor(lead_capture) + subagent_extensions = _extensions_with_contributor(subagent_capture) + + with bind_agent_build_extensions(lead_extensions): + _lead_stack() + with bind_agent_build_extensions(subagent_extensions): + build_subagent_runtime_middlewares(app_config=_app_config(), model_name="gpt-4o") + + assert lead_capture.ctx is not None + assert subagent_capture.ctx is not None + + +def test_tool_visible_lands_at_the_outermost_position(): + stack = _lead_stack(_extensions(MiddlewarePlacement(_Probe("visible"), Placement.TOOL_VISIBLE))) + assert _tags(stack)[0] == "visible" + + +def test_model_logical_lands_outside_the_retry_middleware(): + stack = _lead_stack(_extensions(MiddlewarePlacement(_Probe("decision"), Placement.MODEL_LOGICAL))) + tags = _tags(stack) + assert tags.index("decision") < tags.index("LLMErrorHandlingMiddleware") + + +def test_tool_raw_lands_inside_tool_error_handling(): + stack = _lead_stack(_extensions(MiddlewarePlacement(_Probe("raw"), Placement.TOOL_RAW))) + tags = _tags(stack) + assert tags.index("raw") > tags.index("ToolErrorHandlingMiddleware") + + +def test_runtime_isolation_failure_is_recorded_after_stack_composition(): + from deerflow.extensions import get_runtime_diagnostics, reset_runtime_diagnostics + + class _FailingObserver(AgentMiddleware): + def wrap_tool_call(self, request, handler): + raise ValueError("observation exploded") + + reset_runtime_diagnostics() + try: + stack = _lead_stack( + _extensions( + MiddlewarePlacement( + _FailingObserver(), + Placement.TOOL_VISIBLE, + ) + ) + ) + isolated = next(middleware for middleware in stack if isinstance(middleware, IsolatedMiddleware)) + handler_calls = 0 + + def handler(request): + nonlocal handler_calls + handler_calls += 1 + return "core-result" + + assert isolated.wrap_tool_call("request", handler) == "core-result" + diagnostics = get_runtime_diagnostics() + finally: + reset_runtime_diagnostics() + + assert handler_calls == 1 + assert len(diagnostics) == 1 + assert diagnostics[0].source == "demo:install" + assert diagnostics[0].level == "error" + assert "wrap_tool_call" in diagnostics[0].message + + +def test_build_and_runtime_diagnostics_are_each_recorded_once(): + from deerflow.extensions import get_runtime_diagnostics, reset_runtime_diagnostics + + class _FailingObserver(AgentMiddleware): + def wrap_model_call(self, request, handler): + raise ValueError("observation exploded") + + app_config = _app_config() + app_config.safety_finish_reason.enabled = False + reset_runtime_diagnostics() + try: + stack = _lead_stack( + _extensions( + MiddlewarePlacement( + _FailingObserver(), + Placement.MODEL_PHYSICAL, + ) + ), + app_config=app_config, + ) + isolated = next(middleware for middleware in stack if isinstance(middleware, IsolatedMiddleware)) + + assert isolated.wrap_model_call("request", lambda request: "core-result") == "core-result" + diagnostics = get_runtime_diagnostics() + finally: + reset_runtime_diagnostics() + + assert [diagnostic.level for diagnostic in diagnostics] == ["warning", "error"] + assert sum("fell back to a secondary anchor" in diagnostic.message for diagnostic in diagnostics) == 1 + assert sum("wrap_model_call" in diagnostic.message for diagnostic in diagnostics) == 1 + + +def test_lead_only_contribution_is_absent_from_subagent_stack(): + from deerflow.agents.middlewares.tool_error_handling_middleware import ( + build_subagent_runtime_middlewares, + ) + + extensions = _extensions(MiddlewarePlacement(_Probe("lead-only"), Placement.STANDARD, scope=AgentScope.LEAD)) + stack = build_subagent_runtime_middlewares(app_config=_app_config(), extensions=extensions) + assert "lead-only" not in _tags(stack) + + +def test_first_subagent_build_resolves_every_lazy_anchor(monkeypatch): + """A lazy anchor table must be populated before its subagent copy is made. + + ``dict(dict_subclass)`` bypasses the subclass's ``__iter__``/``__len__`` + hooks in CPython. Building a subagent first therefore used to copy an + empty table, then populate only MODEL_PHYSICAL as it installed the + subagent-specific override. + """ + from deerflow.agents.middlewares.tool_error_handling_middleware import ( + build_subagent_runtime_middlewares, + ) + from deerflow.extensions import stack as stack_module + + fresh_table = stack_module._AnchorTable() + monkeypatch.setattr(stack_module._AnchorTable, "_loaded", False) + monkeypatch.setattr(stack_module, "PLACEMENT_ANCHORS", fresh_table) + + extensions = _extensions( + MiddlewarePlacement( + _Probe("subagent-standard"), + Placement.STANDARD, + scope=AgentScope.SUBAGENT, + ) + ) + stack = build_subagent_runtime_middlewares( + app_config=_app_config(), + extensions=extensions, + ) + + assert "subagent-standard" in _tags(stack) + + +def test_core_ordering_table_is_enforced_against_a_real_stack(monkeypatch): + """The default constraint table must be consulted on the REAL built stack. + + Task 7 deleted the in-builder guard and the test that forced a real + misordering. Exercising assert_ordering with synthetic classes proves the + function works; it does not prove the composing builder calls it with + core_ordering_constraints() against the stack it actually produces. This + test is the only thing that does. + + It patches the default table rather than reordering real middlewares: the + builder imports its middlewares inside the function body, so reordering + them would require stubbing sys.modules entries, which tests the stubbing + more than the wiring. Patching the table asserts exactly the claim at issue. + """ + from deerflow.agents.middlewares.input_sanitization_middleware import InputSanitizationMiddleware + from deerflow.agents.middlewares.tool_error_handling_middleware import ToolErrorHandlingMiddleware + from deerflow.extensions import ordering as ordering_mod + + # InputSanitization is the outermost wrapper and ToolErrorHandling sits deep + # in the tail, so demanding the reverse is a constraint the real stack breaks. + impossible = ( + ordering_mod.OrderingConstraint( + outer=ToolErrorHandlingMiddleware, + inner=InputSanitizationMiddleware, + reason="deliberately inverted for this test", + ), + ) + monkeypatch.setattr(ordering_mod, "core_ordering_constraints", lambda: impossible) + + with pytest.raises(RuntimeError) as excinfo: + _lead_stack() + message = str(excinfo.value) + assert "ToolErrorHandlingMiddleware" in message + assert "InputSanitizationMiddleware" in message + assert "core middleware order" in message, "with no extension at either participating index the blame must fall on core order" + + +def test_core_ordering_table_passes_on_the_unmodified_stack(): + """The real stack must satisfy the real constraints — otherwise the test + above would pass for the wrong reason.""" + _lead_stack() # must not raise + + +def test_ordering_violation_raises_and_names_the_extension(): + """A contribution that inverts a core invariant must fail loudly at build + time — the resulting behaviour would otherwise be wrong without an error.""" + from deerflow.agents.middlewares.tool_error_handling_middleware import ToolErrorHandlingMiddleware + from deerflow.extensions.ordering import OrderingConstraint, assert_ordering + + stack = [ToolErrorHandlingMiddleware(app_config=_app_config()), _Probe("x")] + constraints = (OrderingConstraint(outer=_Probe, inner=ToolErrorHandlingMiddleware, reason="test"),) + with pytest.raises(RuntimeError, match="demo:install"): + assert_ordering(stack, {1: "demo:install"}, constraints) + + +# --- AgentBuildContext.policy ----------------------------------------------- +# +# Extensions read ctx.policy to adapt metering/behaviour to the limits the +# host actually enforces. Both builders must fill it from the resolved +# AppConfig — the field defaults to an all-disabled snapshot, so an omission +# is silent and hands extensions wrong values. + + +class _CtxCapture: + def __init__(self) -> None: + self.ctx = None + + def contribute_middlewares(self, app_store, ctx): + self.ctx = ctx + return () + + +def _extensions_with_contributor(contributor): + registry = ExtensionRegistry() + with registry.attributed_to("capture:install"): + registry.middlewares(contributor) + return registry.build() + + +def _policy_config() -> AppConfig: + from deerflow.config.subagents_config import SubagentsAppConfig + from deerflow.config.token_budget_config import TokenBudgetConfig + + return AppConfig( + sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider"), + token_budget=TokenBudgetConfig( + enabled=True, + max_tokens=12345, + max_input_tokens=1000, + max_output_tokens=2000, + warn_threshold=0.7, + hard_stop_threshold=0.95, + ), + subagents=SubagentsAppConfig( + max_total_per_run=7, + token_budget=TokenBudgetConfig( + enabled=True, + max_tokens=54321, + max_input_tokens=3000, + max_output_tokens=4000, + warn_threshold=0.6, + hard_stop_threshold=0.9, + ), + ), + ) + + +def _assert_projected_policy(policy) -> None: + assert policy.token_budget_enabled is True + assert policy.max_input_tokens == 1000 + assert policy.max_output_tokens == 2000 + assert policy.max_total_tokens == 12345 + assert policy.budget_warn_fraction == 0.7 + assert policy.budget_hard_fraction == 0.95 + assert policy.max_subagents_per_run is None + + +def _assert_subagent_policy(policy) -> None: + assert policy.token_budget_enabled is True + assert policy.max_input_tokens == 3000 + assert policy.max_output_tokens == 4000 + assert policy.max_total_tokens == 54321 + assert policy.budget_warn_fraction == 0.6 + assert policy.budget_hard_fraction == 0.9 + assert policy.max_subagents_per_run is None + + +def test_lead_build_context_carries_the_projected_host_policy(): + capture = _CtxCapture() + _lead_stack(extensions=_extensions_with_contributor(capture), app_config=_policy_config()) + assert capture.ctx is not None, "the contributor was never consulted" + _assert_projected_policy(capture.ctx.policy) + + +def test_subagent_build_context_carries_the_projected_host_policy(): + from deerflow.agents.middlewares.tool_error_handling_middleware import build_subagent_runtime_middlewares + + capture = _CtxCapture() + build_subagent_runtime_middlewares( + app_config=_policy_config(), + model_name="gpt-4o", + extensions=_extensions_with_contributor(capture), + ) + assert capture.ctx is not None, "the contributor was never consulted" + _assert_subagent_policy(capture.ctx.policy) + + +def test_lead_build_context_projects_the_effective_delegation_override(): + capture = _CtxCapture() + _lead_stack( + extensions=_extensions_with_contributor(capture), + app_config=_policy_config(), + configurable={ + "subagent_enabled": True, + "max_total_subagents": 3, + }, + ) + + assert capture.ctx is not None + assert capture.ctx.policy.max_subagents_per_run == 3 diff --git a/backend/tests/test_extension_task_store_runtime.py b/backend/tests/test_extension_task_store_runtime.py new file mode 100644 index 000000000..568aadd1f --- /dev/null +++ b/backend/tests/test_extension_task_store_runtime.py @@ -0,0 +1,384 @@ +"""Runtime task-store wiring required by contributed middleware.""" + +from __future__ import annotations + +import sys +from types import ModuleType, SimpleNamespace +from unittest.mock import AsyncMock, MagicMock + +import pytest +from deerflow_extension_api import EXTENSION_TASK_STORE_KEY, ExtensionData +from langchain_core.messages import AIMessage +from langgraph.checkpoint.memory import InMemorySaver + +from deerflow.extensions import ( + EXTENSION_SNAPSHOT_CONTEXT_KEY, + get_agent_build_extensions, + reset_loaded_extensions, + resolve_run_extensions, + set_loaded_extensions, +) +from deerflow.extensions.registry import ExtensionRegistry +from deerflow.runtime.runs.manager import RunManager +from deerflow.runtime.runs.worker import RunContext, _build_runtime_context, run_agent + + +def test_build_runtime_context_installs_the_extension_store(): + store = ExtensionData("task-1") + + context = _build_runtime_context("thread-1", "run-1", None, None, store) + + assert context[EXTENSION_TASK_STORE_KEY] is store + + +def test_build_runtime_context_omits_the_key_without_a_store(): + context = _build_runtime_context("thread-1", "run-1", None, None, None) + + assert EXTENSION_TASK_STORE_KEY not in context + + +def test_build_runtime_context_installs_the_run_extension_snapshot(): + loaded = ExtensionRegistry().build() + + context = _build_runtime_context("thread-1", "run-1", None, None, None, loaded) + + assert context[EXTENSION_SNAPSHOT_CONTEXT_KEY] is loaded + + +def test_build_runtime_context_omits_the_snapshot_key_without_extensions(): + context = _build_runtime_context("thread-1", "run-1", None, None, None, None) + + assert EXTENSION_SNAPSHOT_CONTEXT_KEY not in context + + +def test_build_runtime_context_never_keeps_a_caller_supplied_snapshot(): + """``runtime.context`` is caller-mergeable, so the run's own snapshot must + win and a caller value must never survive when the run has none.""" + loaded = ExtensionRegistry().build() + forged = ExtensionRegistry().build() + + overridden = _build_runtime_context("thread-1", "run-1", {EXTENSION_SNAPSHOT_CONTEXT_KEY: forged}, None, None, loaded) + dropped = _build_runtime_context("thread-1", "run-1", {EXTENSION_SNAPSHOT_CONTEXT_KEY: forged}, None, None, None) + + assert overridden[EXTENSION_SNAPSHOT_CONTEXT_KEY] is loaded + assert EXTENSION_SNAPSHOT_CONTEXT_KEY not in dropped + + +def test_resolve_run_extensions_rejects_a_foreign_context_value(): + loaded = ExtensionRegistry().build() + + assert resolve_run_extensions({EXTENSION_SNAPSHOT_CONTEXT_KEY: loaded}) is loaded + assert resolve_run_extensions({EXTENSION_SNAPSHOT_CONTEXT_KEY: "not-a-snapshot"}) is None + assert resolve_run_extensions({}) is None + assert resolve_run_extensions(None) is None + + +def test_gateway_run_context_captures_the_app_extension_snapshot(monkeypatch): + from app.gateway import deps + + loaded = ExtensionRegistry().build() + request = SimpleNamespace( + app=SimpleNamespace( + state=SimpleNamespace( + extensions=loaded, + run_events_config=None, + checkpoint_channel_mode="full", + checkpoint_snapshot_frequency=None, + ) + ) + ) + monkeypatch.setattr(deps, "get_checkpointer", lambda request: None) + monkeypatch.setattr(deps, "get_store", lambda request: None) + monkeypatch.setattr(deps, "get_run_event_store", lambda request: None) + monkeypatch.setattr(deps, "get_thread_store", lambda request: None) + monkeypatch.setattr(deps, "get_config", lambda: SimpleNamespace()) + + context = deps.get_run_context(request) + + assert context.extensions is loaded + + +@pytest.fixture +def _isolated_extensions(): + reset_loaded_extensions() + yield + reset_loaded_extensions() + + +class _MiddlewareContributor: + def contribute_middlewares(self, app_store, ctx): + return () + + +class _TaskStoreReadingAgent: + def __init__(self) -> None: + self.task_store = None + self.extensions = None + + async def astream(self, graph_input, config=None, stream_mode=None, subgraphs=False): + runtime = (config or {})["configurable"]["__pregel_runtime"] + self.task_store = runtime.context.get(EXTENSION_TASK_STORE_KEY) + self.extensions = resolve_run_extensions(runtime.context) + yield {"messages": []} + + +def _bridge(): + return SimpleNamespace(publish=AsyncMock(), publish_end=AsyncMock(), cleanup=AsyncMock()) + + +_MOCKED_SUBAGENT_MODULES = ( + "deerflow.agents", + "deerflow.agents.thread_state", + "deerflow.agents.middlewares", + "deerflow.agents.middlewares.thread_data_middleware", + "deerflow.sandbox", + "deerflow.sandbox.middleware", + "deerflow.sandbox.security", + "deerflow.models", + "deerflow.skills.storage", +) + + +@pytest.fixture +def _subagent_env(): + """Import the real executor behind tests/conftest.py's cycle-breaking mock.""" + original_modules = {name: sys.modules.get(name) for name in _MOCKED_SUBAGENT_MODULES} + original_executor = sys.modules.get("deerflow.subagents.executor") + missing = object() + subagents_pkg = sys.modules.get("deerflow.subagents") + original_executor_attr = getattr(subagents_pkg, "executor", missing) if subagents_pkg is not None else missing + + sys.modules.pop("deerflow.subagents.executor", None) + if subagents_pkg is not None and hasattr(subagents_pkg, "executor"): + delattr(subagents_pkg, "executor") + + try: + for name in _MOCKED_SUBAGENT_MODULES: + sys.modules[name] = MagicMock() + storage_module = ModuleType("deerflow.skills.storage") + storage_module.get_or_new_skill_storage = lambda **kwargs: SimpleNamespace(load_skills=lambda *, enabled_only: []) + storage_module.get_or_new_user_skill_storage = lambda user_id, **kwargs: SimpleNamespace(load_skills=lambda *, enabled_only: []) + sys.modules["deerflow.skills.storage"] = storage_module + + from deerflow.subagents.config import SubagentConfig + from deerflow.subagents.executor import SubagentExecutor + + executor_module = sys.modules["deerflow.subagents.executor"] + executor_module.get_app_config = lambda: SimpleNamespace( + tool_search=SimpleNamespace(enabled=False), + authorization=SimpleNamespace(enabled=False), + ) + yield SimpleNamespace(SubagentConfig=SubagentConfig, SubagentExecutor=SubagentExecutor) + finally: + for name, original in original_modules.items(): + if original is None: + sys.modules.pop(name, None) + else: + sys.modules[name] = original + if original_executor is None: + sys.modules.pop("deerflow.subagents.executor", None) + else: + sys.modules["deerflow.subagents.executor"] = original_executor + subagents_pkg = sys.modules.get("deerflow.subagents") + if subagents_pkg is not None: + if original_executor_attr is missing: + if hasattr(subagents_pkg, "executor"): + delattr(subagents_pkg, "executor") + else: + setattr(subagents_pkg, "executor", original_executor_attr) + + +class _CapturingSubagent: + def __init__(self, seen: dict) -> None: + self._seen = seen + + async def astream(self, *args, **kwargs): + self._seen["context"] = kwargs.get("context") + yield {"messages": [AIMessage(content="done")]} + + +async def _run_subagent(monkeypatch, env, *, seen: dict): + async def _initial_state(self, task): + return ({}, [], None) + + monkeypatch.setattr(env.SubagentExecutor, "_build_initial_state", _initial_state) + monkeypatch.setattr(env.SubagentExecutor, "_create_agent", lambda self, tools, **kwargs: _CapturingSubagent(seen)) + config = env.SubagentConfig(name="researcher", description="d", system_prompt="p", tools=[]) + executor = env.SubagentExecutor(config=config, tools=[], thread_id="thread-1", run_id=None) + return await executor._aexecute("do the thing") + + +@pytest.mark.anyio +async def test_lead_middleware_receives_a_run_scoped_store(_isolated_extensions): + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_MiddlewareContributor()) + set_loaded_extensions(registry.build()) + + run_manager = RunManager() + record = await run_manager.create("thread-ext-middleware") + agent = _TaskStoreReadingAgent() + + await run_agent( + _bridge(), + run_manager, + record, + ctx=RunContext(checkpointer=InMemorySaver()), + agent_factory=lambda *, config: agent, + graph_input={}, + config={}, + ) + + assert agent.task_store is not None + assert agent.task_store.scope_id == record.run_id + + +@pytest.mark.anyio +async def test_lead_agent_factory_receives_the_same_extension_snapshot_as_the_store(_isolated_extensions): + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_MiddlewareContributor()) + loaded = registry.build() + set_loaded_extensions(ExtensionRegistry().build()) + run_manager = RunManager() + record = await run_manager.create("thread-ext-snapshot") + agent = _TaskStoreReadingAgent() + seen = {} + + def agent_factory(*, config): + seen["extensions"] = get_agent_build_extensions() + return agent + + await run_agent( + _bridge(), + run_manager, + record, + ctx=RunContext(checkpointer=None, extensions=loaded), + agent_factory=agent_factory, + graph_input={}, + config={}, + ) + + assert seen["extensions"] is loaded + assert agent.task_store is not None + + +@pytest.mark.anyio +async def test_lead_run_publishes_its_snapshot_for_delegated_work(_isolated_extensions): + """Delegation happens during graph execution, long after the graph-build + contextvar binding has exited, so the run's snapshot has to travel through + runtime context to stay reachable from ``task_tool``.""" + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_MiddlewareContributor()) + loaded = registry.build() + set_loaded_extensions(ExtensionRegistry().build()) + run_manager = RunManager() + record = await run_manager.create("thread-ext-delegation") + agent = _TaskStoreReadingAgent() + + await run_agent( + _bridge(), + run_manager, + record, + ctx=RunContext(checkpointer=None, extensions=loaded), + agent_factory=lambda *, config: agent, + graph_input={}, + config={}, + ) + + assert agent.extensions is loaded + + +@pytest.mark.anyio +async def test_subagent_middleware_without_parent_run_receives_its_task_store(monkeypatch, _isolated_extensions, _subagent_env): + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_MiddlewareContributor()) + set_loaded_extensions(registry.build()) + seen: dict = {} + + result = await _run_subagent(monkeypatch, _subagent_env, seen=seen) + + store = (seen.get("context") or {}).get(EXTENSION_TASK_STORE_KEY) + assert isinstance(store, ExtensionData) + assert store.scope_id == result.task_id + + +@pytest.mark.anyio +async def test_subagent_builder_receives_the_same_extension_snapshot_as_the_store(monkeypatch, _isolated_extensions, _subagent_env): + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_MiddlewareContributor()) + loaded = registry.build() + set_loaded_extensions(loaded) + seen: dict = {} + + async def _initial_state(self, task): + set_loaded_extensions(ExtensionRegistry().build()) + return ({}, [], None) + + def _create_agent(self, tools, *, deferred_setup=None, extensions=None): + seen["extensions"] = extensions + return _CapturingSubagent(seen) + + monkeypatch.setattr(_subagent_env.SubagentExecutor, "_build_initial_state", _initial_state) + monkeypatch.setattr(_subagent_env.SubagentExecutor, "_create_agent", _create_agent) + config = _subagent_env.SubagentConfig(name="researcher", description="d", system_prompt="p", tools=[]) + executor = _subagent_env.SubagentExecutor(config=config, tools=[], thread_id="thread-1", run_id=None) + + result = await executor._aexecute("do the thing") + + assert seen["extensions"] is loaded + store = (seen.get("context") or {}).get(EXTENSION_TASK_STORE_KEY) + assert isinstance(store, ExtensionData) + assert store.scope_id == result.task_id + + +@pytest.mark.anyio +async def test_subagent_prefers_the_parent_run_snapshot_over_the_singleton(monkeypatch, _isolated_extensions, _subagent_env): + """A singleton replacement between the lead run's start and the subagent's + execution must not mix two extension generations within one run.""" + registry = ExtensionRegistry() + with registry.attributed_to("demo:install"): + registry.middlewares(_MiddlewareContributor()) + run_snapshot = registry.build() + replaced_singleton = ExtensionRegistry().build() + set_loaded_extensions(replaced_singleton) + seen: dict = {} + + async def _initial_state(self, task): + return ({}, [], None) + + def _create_agent(self, tools, *, deferred_setup=None, extensions=None): + seen["extensions"] = extensions + return _CapturingSubagent(seen) + + monkeypatch.setattr(_subagent_env.SubagentExecutor, "_build_initial_state", _initial_state) + monkeypatch.setattr(_subagent_env.SubagentExecutor, "_create_agent", _create_agent) + config = _subagent_env.SubagentConfig(name="researcher", description="d", system_prompt="p", tools=[]) + executor = _subagent_env.SubagentExecutor( + config=config, + tools=[], + thread_id="thread-1", + run_id=None, + extensions=run_snapshot, + ) + + result = await executor._aexecute("do the thing") + + assert seen["extensions"] is run_snapshot + # The store follows the same snapshot: the replaced singleton contributes + # no middleware and would have allocated nothing. + store = (seen.get("context") or {}).get(EXTENSION_TASK_STORE_KEY) + assert isinstance(store, ExtensionData) + assert store.scope_id == result.task_id + + +@pytest.mark.anyio +async def test_subagent_without_task_store_contributors_does_not_inject_one(monkeypatch, _isolated_extensions, _subagent_env): + seen: dict = {} + + await _run_subagent(monkeypatch, _subagent_env, seen=seen) + + assert EXTENSION_TASK_STORE_KEY not in (seen.get("context") or {}) diff --git a/backend/tests/test_task_tool_core_logic.py b/backend/tests/test_task_tool_core_logic.py index 6de04fed2..a6535c6ae 100644 --- a/backend/tests/test_task_tool_core_logic.py +++ b/backend/tests/test_task_tool_core_logic.py @@ -259,6 +259,73 @@ def test_task_tool_returns_error_for_unknown_subagent(monkeypatch): assert message.additional_kwargs[SUBAGENT_ERROR_KEY] == "Unknown subagent type 'general-purpose'. Available: general-purpose" +def test_task_tool_forwards_the_run_extension_snapshot_to_executor(monkeypatch): + """The lead run binds one immutable extension snapshot; delegation must + carry that same object rather than re-reading the process singleton, which + a concurrent replacement could have swapped underneath the run.""" + from deerflow.extensions import EXTENSION_SNAPSHOT_CONTEXT_KEY + from deerflow.extensions.registry import ExtensionRegistry + + loaded = ExtensionRegistry().build() + runtime = _make_runtime() + runtime.context[EXTENSION_SNAPSHOT_CONTEXT_KEY] = loaded + captured = {} + + class DummyExecutor: + def __init__(self, **kwargs): + captured["executor_kwargs"] = kwargs + + def execute_async(self, prompt, task_id=None): + return task_id or "generated-task-id" + + monkeypatch.setattr(task_tool_module, "SubagentStatus", FakeSubagentStatus) + monkeypatch.setattr(task_tool_module, "SubagentExecutor", DummyExecutor) + monkeypatch.setattr(task_tool_module, "get_subagent_config", lambda _: _make_subagent_config()) + monkeypatch.setattr( + task_tool_module, + "get_background_task_result", + lambda _: _make_result(FakeSubagentStatus.COMPLETED, result="done"), + ) + monkeypatch.setattr(task_tool_module, "get_stream_writer", lambda: lambda _event: None) + monkeypatch.setattr(task_tool_module.asyncio, "sleep", _no_sleep) + monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: []) + + _run_task_tool(runtime=runtime, description="test", prompt="p", subagent_type="general-purpose", tool_call_id="tc-ext") + + assert captured["executor_kwargs"]["extensions"] is loaded + + +def test_task_tool_omits_extensions_without_a_run_snapshot(monkeypatch): + """Callers outside the Gateway run path (embedded client, standalone + LangGraph Server) install no snapshot; the executor must keep its existing + singleton fallback instead of receiving a forged or missing value.""" + runtime = _make_runtime() + captured = {} + + class DummyExecutor: + def __init__(self, **kwargs): + captured["executor_kwargs"] = kwargs + + def execute_async(self, prompt, task_id=None): + return task_id or "generated-task-id" + + monkeypatch.setattr(task_tool_module, "SubagentStatus", FakeSubagentStatus) + monkeypatch.setattr(task_tool_module, "SubagentExecutor", DummyExecutor) + monkeypatch.setattr(task_tool_module, "get_subagent_config", lambda _: _make_subagent_config()) + monkeypatch.setattr( + task_tool_module, + "get_background_task_result", + lambda _: _make_result(FakeSubagentStatus.COMPLETED, result="done"), + ) + monkeypatch.setattr(task_tool_module, "get_stream_writer", lambda: lambda _event: None) + monkeypatch.setattr(task_tool_module.asyncio, "sleep", _no_sleep) + monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: []) + + _run_task_tool(runtime=runtime, description="test", prompt="p", subagent_type="general-purpose", tool_call_id="tc-no-ext") + + assert "extensions" not in captured["executor_kwargs"] + + def test_task_tool_forwards_channel_user_id_to_executor(monkeypatch): """The IM-channel sender identity must survive delegation: in group chats one thread serves many senders, so a subagent's bash commands need the diff --git a/backend/tests/test_tool_error_handling_middleware.py b/backend/tests/test_tool_error_handling_middleware.py index 4dadaf8d1..03d493b6e 100644 --- a/backend/tests/test_tool_error_handling_middleware.py +++ b/backend/tests/test_tool_error_handling_middleware.py @@ -223,13 +223,17 @@ def test_tool_progress_middleware_is_outer_relative_to_error_handling(monkeypatc assert progress_idx < error_idx, f"ToolProgressMiddleware (index {progress_idx}) must be outer (lower index) than ToolErrorHandlingMiddleware (index {error_idx}); order: {[type(m).__name__ for m in middlewares]}" -def test_middleware_ordering_guard_raises_when_progress_is_inner(monkeypatch: pytest.MonkeyPatch): - """_build_runtime_middlewares must raise RuntimeError when ToolProgressMiddleware ends up - at a higher index than ToolErrorHandlingMiddleware. +def test_middleware_ordering_guard_moved_to_declarative_constraints(monkeypatch: pytest.MonkeyPatch): + """_build_runtime_middlewares no longer hand-validates ordering; the invariant is now + declared in deerflow.extensions.ordering (core_ordering_constraints / assert_ordering) and + is checked once the composing builder merges extension contributions in (Task 9). - We trigger the wrong-order condition by patching SandboxAuditMiddleware to be an actual - ToolErrorHandlingMiddleware instance, which appears BEFORE ToolProgressMiddleware in the - list. The guard's isinstance() check finds it first, making error_idx < progress_idx. + This test previously monkeypatched SandboxAuditMiddleware to a ToolErrorHandlingMiddleware + instance to force the wrong-order condition and asserted that the builder itself raised. + That in-builder guard was deleted on purpose: validating here would check a stack that + hasn't received extension contributions yet. Building under the same wrong-order condition + must no longer raise inside this builder; deerflow.extensions.ordering has the equivalent + coverage (see test_extension_ordering.py and test_core_constraints_are_declared). """ from deerflow.agents.middlewares.tool_error_handling_middleware import ( ToolErrorHandlingMiddleware, @@ -240,7 +244,7 @@ def test_middleware_ordering_guard_raises_when_progress_is_inner(monkeypatch: py _stub_runtime_middleware_imports(monkeypatch) # Override the SandboxAuditMiddleware stub with a real ToolErrorHandlingMiddleware so it # becomes the FIRST ToolErrorHandlingMiddleware in the list, appearing before - # ToolProgressMiddleware and triggering the ordering guard. + # ToolProgressMiddleware — the same wrong-order condition the deleted guard used to catch. monkeypatch.setitem( sys.modules, "deerflow.agents.middlewares.sandbox_audit_middleware", @@ -253,8 +257,9 @@ def test_middleware_ordering_guard_raises_when_progress_is_inner(monkeypatch: py app_config = _make_app_config() app_config = app_config.model_copy(update={"tool_progress": ToolProgressConfig(enabled=True)}) - with pytest.raises(RuntimeError, match="ToolProgressMiddleware must be outer"): - build_lead_runtime_middlewares(app_config=app_config, lazy_init=False) + # No raise here: the invariant is enforced by assert_ordering at the composing builder, + # not inside _build_runtime_middlewares. + build_lead_runtime_middlewares(app_config=app_config, lazy_init=False) def test_lead_runtime_middlewares_thread_app_config_to_tool_error_handling(monkeypatch: pytest.MonkeyPatch): diff --git a/backend/uv.lock b/backend/uv.lock index f3b4db618..474b00dfa 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -13,6 +13,7 @@ resolution-markers = [ [manifest] members = [ "deer-flow", + "deerflow-extension-api", "deerflow-harness", ] overrides = [{ name = "websockets", specifier = "==16.0" }] @@ -872,6 +873,11 @@ dev = [ { name = "textual", specifier = ">=0.80" }, ] +[[package]] +name = "deerflow-extension-api" +version = "0.1.0" +source = { editable = "packages/extension-api" } + [[package]] name = "deerflow-harness" version = "2.1.0" @@ -884,6 +890,7 @@ dependencies = [ { name = "croniter" }, { name = "cryptography" }, { name = "ddgs" }, + { name = "deerflow-extension-api" }, { name = "dotenv" }, { name = "duckdb" }, { name = "e2b-code-interpreter" }, @@ -960,6 +967,7 @@ requires-dist = [ { name = "croniter", specifier = ">=6.0.0" }, { name = "cryptography", specifier = ">=48.0.1" }, { name = "ddgs", specifier = ">=9.10.0" }, + { name = "deerflow-extension-api", editable = "packages/extension-api" }, { name = "dotenv", specifier = ">=0.9.9" }, { name = "duckdb", specifier = ">=1.4.4" }, { name = "e2b-code-interpreter", specifier = ">=2.8.0" }, diff --git a/config.example.yaml b/config.example.yaml index 3da2f42c6..8d7a54ed9 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -2473,3 +2473,35 @@ authorization: # # Security features (enabled by default): # pkce_enabled: true # PKCE (S256) for authorization code flow # nonce_enabled: true # Nonce validation in ID tokens + +# --- Plugins ---------------------------------------------------------------- +# Extension packages, loaded once at startup in the order listed here. Each +# entry names an install entry point as "module.path:install"; the `config` +# block is handed to that package verbatim and validated by the package itself. +# +# Loading is explicit on purpose: an installed package does nothing until it is +# listed here, and the list order is what fixes middleware ordering. +# +# This is deliberately separate from the `extensions:` block above (MCP servers, +# skills, config-declared middlewares). That one is backed by +# extensions_config.json, which the Gateway can rewrite through an HTTP +# endpoint; a list that causes code to be imported must stay in this +# operator-controlled file only. +# +# This initial extension-system slice accepts middleware contributors. Their +# install functions register semantically placed middlewares for the lead and/or +# subagent stack; the examples below intentionally cover only that surface. +# +# plugins: +# - use: acme_request_observer:install +# config: +# enabled: true +# placement: model_logical +# - use: acme_tool_observer:install +# config: +# enabled: true +# placement: tool_raw +# # `required: true` turns a load failure into a startup failure. Use it for +# # packages whose absence changes behaviour rather than just observability. +# - use: acme_required_middleware:install +# required: true