Nan Gao 13f0a7f263
feat(extensions): let an out-of-tree extension observe what the agent did (#4863)
* feat(extensions): let an out-of-tree extension observe what the agent did

DeerFlow's extension system can contribute middleware, services and routes,
but an extension cannot answer basic questions about a run without reaching
into host internals. Several of the facts it would need are destroyed by the
operations that produce them:

  * The middleware chain injects and rewrites a lot of context — date
    reminders, recalled memory, compaction summaries, durable-context data,
    image payloads, activated skill bodies. Downstream, none of it is
    attributable: at the model-call boundary an injected HumanMessage is
    indistinguishable from the user's own, and anything wanting to tell them
    apart has to pattern-match prompt wording, which breaks on the next copy
    edit.

  * Two runs of "the same agent" are only comparable if the chain enforced the
    same limits, prompts and thresholds. Recovering that from outside means
    reading private attributes and guessing which of them change behaviour — a
    guess that rots silently as middlewares gain fields.

  * The lead-agent factory resolves a model after runtime overrides, renders a
    prompt, filters tools through authorization and composes a stack, all
    inside one synchronous call, and none of it survives: a middleware sees its
    neighbours but not the prompt, the run worker sees a graph but not what
    went into it.

  * Summarization is destructive by design. N messages leave the context and
    one summary enters it; afterwards only the summary exists, so "which
    messages became this?" is not reconstructible.

This adds seven neutral facilities so those facts are recorded where they are
still true, and releases the contract package as 0.2.0.

Message provenance
  Producers stamp `deerflow_content_kind` / `deerflow_producer_kind` onto the
  messages they inject or rewrite. Stamping is unconditional — a fact whose
  presence depends on whether an observer is installed is not a fact — and the
  keys are server-owned, so provenance cannot be forged from a request.

Middleware self-description
  Twelve middlewares declare their own behaviour-affecting parameters through
  a duck-typed `release_policy_parameters()`. Long text is hashed rather than
  embedded: a declaration is an identity, not a copy of the prompt.

Agent assembly descriptor
  `assemble_lead_agent()` returns the graph plus a descriptor whose fingerprint
  answers "did anything about this agent change between these two runs?".
  `make_lead_agent()` keeps its graph-only signature — it is the LangGraph
  Server ABI declared in langgraph.json. Tools and skills are sorted before
  hashing because their assembly order is incidental; middlewares are not,
  because stack order decides what wraps what. Host build identity is reported
  but excluded from the fingerprint, so a redeploy does not invalidate every
  agent's identity.

Context compaction observation
  Summarization emits the content hashes of the messages it is about to remove
  joined to the summary that replaced them. Content is the only identity
  available at that seam: the summary does not become a message, and what later
  projects it into a request renders it bounded and escaped rather than
  verbatim.

Neutral policy, transform and MCP-source facts
  Guardrail decisions are published to runtime context under a `__`-prefixed
  key; result-rewriting middlewares append a declared, ordered transform trail;
  MCP tools carry their credential-free logical origin.

Extension route identity
  Contributed routes are session-authenticated and cannot opt out, but
  "logged in" and "administrator" are different questions. Extensions get a
  neutral projection of the caller rather than the host's auth context, and
  `require_admin` fails closed when identity cannot be determined.

Extension-owned tables
  An extension that persists data owns its own MetaData and migration chain, so
  its tables are absent from Base.metadata and `alembic revision --autogenerate`
  proposes dropping them. Extensions declare a table prefix, which is rejected
  at registration if it would shadow a host table.

The contract package stays dependency-free and imports no host code; every new
Protocol method has a default so later additions remain additive. The loader's
pre-1.0 rule requires an exact major.minor match, so extensions written against
0.1 are now refused at startup with an actionable install hint rather than
loading into a host that implements a different surface.

uv.lock records the contract package's new version, so `uv sync --locked` still
resolves on a fresh checkout.

* fix(backend): sort gateway service imports
2026-08-23 09:57:12 +08:00

125 lines
5.0 KiB
Python

"""What an agent was assembled from, captured where it is knowable.
The lead-agent factory resolves a model (after runtime overrides), renders a
system prompt, filters a tool list through authorization, and composes a
middleware stack. All four are decided inside one synchronous call and none of
them survives to any later observation point: a middleware sees its neighbours
but not the prompt, the run worker sees the graph but not what went into it.
The factory therefore emits a descriptor alongside the graph. Its fingerprint
is what makes "did anything about this agent change between these two runs?"
answerable.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from functools import cached_property
from typing import Any, Protocol
from deerflow_extension_api.release import canonical_hash
from deerflow_extension_api.state import ExtensionData
@dataclass(frozen=True)
class ToolDescriptor:
name: str
description_hash: str
schema_hash: str
source: str
mcp_server: str | None = None
mcp_transport: str | None = None
@dataclass(frozen=True)
class MiddlewareDescriptor:
name: str
module: str
policy_parameters: dict[str, Any] = field(default_factory=dict)
#: Extension this middleware was contributed by, or ``None`` for a host
#: middleware. Contributed middlewares reach the stack inside a wrapper
#: whose class name is shared by all of them, so without this two
#: extensions' middlewares would be indistinguishable here.
extension: str | None = None
@dataclass(frozen=True)
class AgentAssemblyDescriptor:
namespace: str
agent_name: str
requested_model: str | None
effective_model: str
model_parameters: dict[str, Any]
thinking_enabled: bool
reasoning_effort: Any
base_prompt_hash: str
tools: tuple[ToolDescriptor, ...]
middlewares: tuple[MiddlewareDescriptor, ...]
deferred_tool_names: tuple[str, ...]
enabled_skills: tuple[str, ...]
effective_policies: dict[str, Any]
#: Which host build produced this assembly (package version, image digest,
#: git commit). Reported, but deliberately outside ``fingerprint`` — see
#: the note there.
build: dict[str, Any] = field(default_factory=dict)
@cached_property
def fingerprint(self) -> str:
"""Identity of everything that changes how this agent behaves.
Tools and skills are sorted: their assembly order is incidental.
Middlewares are not: stack order decides what wraps what.
Two fields are reported but deliberately excluded:
* ``build`` — the fingerprint answers "did this agent's assembly
change", which is a finer question than "did the host binary
change". Folding the build in would change every agent's fingerprint
on every redeploy, making the fine question unanswerable; leaving it
out keeps both answerable, because a consumer can still compare
``build`` directly.
* ``requested_model`` — only ``effective_model`` reaches the provider,
so a request that resolves to the same effective model is not a
behavioural difference.
"""
return canonical_hash(
{
"namespace": self.namespace,
"agent_name": self.agent_name,
"effective_model": self.effective_model,
"model_parameters": self.model_parameters,
"thinking_enabled": self.thinking_enabled,
"reasoning_effort": self.reasoning_effort,
"base_prompt_hash": self.base_prompt_hash,
"tools": sorted(
[
{
"name": tool.name,
"description_hash": tool.description_hash,
"schema_hash": tool.schema_hash,
"source": tool.source,
"mcp_server": tool.mcp_server,
"mcp_transport": tool.mcp_transport,
}
for tool in self.tools
],
key=lambda entry: entry["name"],
),
"middlewares": [{"name": m.name, "module": m.module, "extension": m.extension, "policy_parameters": m.policy_parameters} for m in self.middlewares],
"deferred_tool_names": sorted(self.deferred_tool_names),
"enabled_skills": sorted(self.enabled_skills),
"effective_policies": self.effective_policies,
}
)
class AgentAssemblyObserver(Protocol):
def on_agent_assembled(self, app_store: ExtensionData, descriptor: AgentAssemblyDescriptor) -> None:
"""Called synchronously at the end of agent construction.
Synchronous because construction is: there is no loop to await on, and
the descriptor must be captured before the graph is handed out.
Implementations must be cheap and must not raise.
"""
return None