mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 21:26:39 +00:00
* feat(authz): add plugin resource targets, provenance, action and management enforcement
PR1 of the extension-tool-authorization series
(docs/plans/2026-09-24-extension-tool-authorization-spec.md): plugin
authorization plumbing, no new author-facing declaration.
- `authz/plugin_targets.py` is the single encoder/validator for composite
targets (`namespace/part`), with one validating composer behind three
kind-checked constructors; invalid input raises instead of degrading.
- `authz/plugin_authz.py` is the harness decision layer (`plugin_action`,
`plugin_management`, plus the batch filters the page slice will consume):
disabled is a no-op, an explicit deny raises, and provider errors follow
`fail_closed` like the sandbox gate.
- `POST /api/plugins/{ns}/actions/{name}` decides `plugin_action:invoke`
after the action resolves and before the body is streamed.
- `deerflow_extension_api.auth` publishes `require_plugin_management` /
`arequire_plugin_management` with two `app.state` resolver keys; the
Gateway installs both. Contract version 0.2.3 -> 0.2.4.
- `tools/tool_provenance.py` records host-written plugin provenance for
display (assembly descriptor, Layer-2 context later); no decision reads a
label.
- Plugin providers use a `(config signature, loop key)` cache so a
loop-affine provider is never handed to another loop or to a sync caller.
- RBAC gains `plugin_actions` / `plugin_management` aliases;
`config_version` 47 -> 48 across the three mirrors.
Tests: new plugin target/provenance/action/management suites plus a
blocking-IO anchor whose offloads were reverted to prove red->green.
* fix(authz): decide plugin resources from the request's config snapshot
Review round 1 on PR #5842 (willem-bd).
P1 — the decision layer no longer reloads the configuration. The six
`authz/plugin_authz.py` functions take a required `app_config` snapshot (the
same request-scoped config the Gateway used to resolve the provider), and
`resolve_/aresolve_plugin_authorization` return it as a third element, so
`enabled`/`fail_closed` can never come from a second read that drifted from
the one that produced the provider. A snapshot the caller could not read at
all is *unavailable*, not disabled: it fails closed as
`authz.config_unavailable`. The Gateway's `_plugin_app_config` now separates
an absent `config.yaml` (no gate, today's behavior) from a config that exists
but cannot be read or validated (deny) instead of swallowing both through
`safe_app_config`.
P2 — `_validated_decision` requires `type(decision.allow) is bool` on both
the sync `authorize` and async `aauthorize` paths, so a provider returning
`AuthzDecision(allow="false")` is a malformed verdict that follows
`fail_closed` instead of being read as an allow.
Both fixes are covered by regression tests that were verified red against the
pre-fix behavior (P1: 200 vs 403 and resolver True vs False; P2: 12 cases
across fail_closed/fail_open and sync/async), plus tests pinning that an
absent config and `authorization.enabled: false` keep today's behavior.
* fix(authz): treat a request-time config absence as unavailable
Review round 2 on PR #5842 (willem-bd).
P1 — `_plugin_app_config[_async]` no longer maps a missing `config.yaml` to
`None` ("no gate"). A Gateway cannot have started without a readable config
(`app/gateway/app.py` reads it through this same accessor in `lifespan`), so
absence *during a request* means the configuration this process was running on
became unavailable — a hot reload, an atomic replace, a ConfigMap remount. It is
unavailable, not disabled: the error propagates,
`resolve_/aresolve_plugin_authorization` raise
`_PluginAuthorizationUnavailable(fail_closed=True)`, the action dispatcher
returns 403 and the management resolvers answer `False`. The dead
`(None, None, None)` allow path is gone, the resolvers return a non-optional
`AppConfig` as their third element, and callers key "authorization disabled" on
`provider is None` alone. This matches the rule the decision layer already
documented (`authz.config_unavailable`) and the public docs.
P2 — `_deny_reason_code` is total: a `reasons` that is not iterable degrades the
code to `authz.denied` instead of raising `TypeError` outside the guarded
provider path, which turned a denial into an unhandled error. It is deliberately
not folded into `_validated_decision`: `allow` was a real `bool`, so the verdict
is a valid denial, and treating it as malformed would flip it to an allow under
`fail_closed: false`.
Both fixes are covered by regression tests verified red against the pre-fix
behavior (P1: resolver True vs False, async `PermissionError` not raised, route
200 vs 403; P2: `TypeError: 'NoneType' object is not iterable` at
`plugin_authz.py:113`). The action-route tests in
`tests/test_plugin_contributions.py` now install a readable
`authorization.enabled: false` config instead of relying on a config-less
process, and `docs/full-stack-plugins.md` plus the PR1 decision log record the
new rule.
* fix(authz): apply the running policy when the plugin config is unreadable
Follow-up to f03d2ea9 (review round 2 on PR #5842, willem-bd).
The first attempt read the P1 finding as "absence is never disabled" and failed
every config read closed unconditionally. That is too broad for this repo: a
config-less host is deliberately treated as having no policy to apply
(`_get_route_authorization_config` falls back to a disabled config,
`sandbox_authz.safe_app_config` documents the same no-op for "CI runners and
direct-call tests"), and the rule turned the config-less e2e plugin host
(`extension_test_fixtures.bookmark_plugin_gateway`) into a 403 — the three
`bookmark-plugin.spec.ts` cases lost the save action. Reproduced locally:
`POST /api/plugins/community.bookmarks/actions/save` against that fixture host
returned 403 before, 200 after this change.
The rule is now "has this process loaded a configuration?", not "was the last
read successful?":
- No loaded config (`peek_loaded_app_config()`, new read-only accessor on
`deerflow.config.app_config`) and no `config.yaml`: no policy to apply, so the
gate stays a no-op — unchanged for CI runners, direct-call tests and hosts
that mount only the plugins router.
- A loaded config whose file disappears or cannot be read: *unavailable*, never
*disabled*. The resolvers raise `_PluginAuthorizationUnavailable` carrying the
failure flag of the configuration this process runs on (authorization
disabled there -> no gate; enabled -> that policy's `fail_closed`, `True` by
default), so an enabled Gateway answers 403/False in the window instead of
letting protected plugin actions and management routes through.
- A config file that exists but cannot be read or validated right now: fail
closed when nothing is loaded (round 1's rule), otherwise the loaded policy
decides.
`peek_loaded_app_config()` is preferred over a Gateway-side startup snapshot: it
reflects the last successfully loaded config, so a hot-reloaded policy is what
applies during the window, and there is no lifecycle hook to keep in sync.
Evidence: the three no-config/lost-config branches in
`tests/test_plugin_management_guard.py` and
`tests/test_plugin_action_authorization.py`, round 1's unreadable-config tests
unchanged, `test_peek_loaded_app_config_survives_a_missing_file` in
`tests/test_app_config_reload.py`, and the fixture-host probe above. Docs
(`docs/full-stack-plugins.md`, `reference.mdx` en/zh) and the PR1 decision log
state the final rule; the log records the superseded variant.
* fix(authz): keep a denial whose reasons raise while being read
* fix(authz): intersect plugin batch filters with the candidates
The plugin_page/plugin_management batch helpers returned the provider's
`filter_resources` answer verbatim, so a custom provider could hand the
projections a target that was never asked about ("*", another namespace,
a phantom page). Every other filter call site re-projects from its own
candidate list; do the same here, state the intersection on the provider
contract, and pin it with a regression test that fails on the old
pass-through.
* docs(plugins): map a management denial to 403 in the route example
The contributed-route example called `arequire_plugin_management` and let
the documented `PermissionError` escape. The Gateway installs no
`PermissionError` handler, so a route copied from the example answered 500
on a normal policy denial instead of 403. Both example routes now translate
the denial, and the surrounding bullets say why the translation is the
route's job.
* docs(extensions): point the 0.2.4 contract row at the PR that ships it
Every other row in the version-history table links the pull request that
shipped the version; 0.2.4 linked the authorization tracking issue instead.
Both the English and Chinese tables now link this PR.
187 lines
7.7 KiB
Python
187 lines
7.7 KiB
Python
"""Plugin authorization must keep its blocking work off the async event loop.
|
|
|
|
Anchors the placement contract of ``authz/plugin_authz.py`` and
|
|
``app/gateway/authz.py``: config loading, provider *discovery* and the
|
|
synchronous ``filter_resources`` call all run in a worker, while provider
|
|
*construction* and ``aauthorize`` stay on the calling loop. Each test drives a
|
|
real production entry point with a probe that performs genuine blocking file IO,
|
|
so the gate fires if the offload is reverted.
|
|
|
|
Each test records that its probe really ran, so an anchor cannot pass because
|
|
the production path stopped calling it. `test_gate_smoke.py` protects the gate
|
|
itself from silently losing teeth.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import sys
|
|
from pathlib import Path
|
|
from types import ModuleType, SimpleNamespace
|
|
from uuid import uuid4
|
|
|
|
import pytest
|
|
from deerflow_extension_api.auth import (
|
|
EXTENSION_PRINCIPAL_RESOLVER_KEY,
|
|
ExtensionPrincipal,
|
|
)
|
|
from deerflow_extension_api.plugins import BackendAction, PluginContribution
|
|
from fastapi import HTTPException
|
|
from starlette.requests import Request
|
|
|
|
from app.gateway import authz as gateway_authz
|
|
from app.gateway.app import _resolve_extension_plugin_management_async
|
|
from app.gateway.auth_disabled import AUTH_SOURCE_SESSION
|
|
from app.gateway.routers.plugins import invoke_plugin_action
|
|
from deerflow.authz.plugin_authz import afilter_plugin_management
|
|
from deerflow.authz.provider import Principal
|
|
from deerflow.authz.rbac import RbacAuthorizationProvider
|
|
from deerflow.config.app_config import AppConfig, reset_app_config, set_app_config
|
|
from deerflow.config.authorization_config import AuthorizationConfig, AuthorizationProviderConfig
|
|
from deerflow.config.sandbox_config import SandboxConfig
|
|
from deerflow.extensions.registry import ExtensionRegistry
|
|
|
|
pytestmark = pytest.mark.asyncio
|
|
|
|
NAMESPACE = "community.check"
|
|
RBAC = "deerflow.authz.rbac:RbacAuthorizationProvider"
|
|
|
|
|
|
def _blocking_probe(tmp_path: Path) -> Path:
|
|
path = tmp_path / "plugin-authz-blocking-probe"
|
|
path.write_text("probe", encoding="utf-8")
|
|
return path
|
|
|
|
|
|
def _app_config(*, roles: dict, provider_use: str = RBAC, provider_config: dict | None = None) -> AppConfig:
|
|
return AppConfig(
|
|
sandbox=SandboxConfig(use="deerflow.sandbox.local:LocalSandboxProvider"),
|
|
authorization=AuthorizationConfig(
|
|
enabled=True,
|
|
fail_closed=True,
|
|
default_role="user",
|
|
provider=AuthorizationProviderConfig(use=provider_use, config={"roles": roles} if provider_config is None else provider_config),
|
|
),
|
|
)
|
|
|
|
|
|
def _plugin_extensions() -> object:
|
|
async def check(payload, context): # pragma: no cover - the decisions under test deny first
|
|
return {}
|
|
|
|
plugin = PluginContribution(namespace=NAMESPACE, title="Check", enabled=True, backend=(BackendAction("check", check),))
|
|
registry = ExtensionRegistry()
|
|
with registry.attributed_to("test:install"):
|
|
registry.plugin(plugin)
|
|
return registry.build()
|
|
|
|
|
|
def _unknown_request(*, extensions: object, user_id: str = "user-1") -> SimpleNamespace:
|
|
return SimpleNamespace(
|
|
app=SimpleNamespace(state=SimpleNamespace(extensions=extensions)),
|
|
state=SimpleNamespace(user=SimpleNamespace(id=user_id, system_role="user", oauth_provider=None, oauth_id=None), auth_source=AUTH_SOURCE_SESSION),
|
|
headers={},
|
|
)
|
|
|
|
|
|
def _action_request(extensions: object) -> Request:
|
|
"""A real Starlette Request for the action route, without an ASGI round trip."""
|
|
app = SimpleNamespace(state=SimpleNamespace(extensions=extensions))
|
|
user = SimpleNamespace(id=uuid4(), system_role="user", oauth_provider=None, oauth_id=None)
|
|
request = Request({"type": "http", "app": app, "headers": [], "method": "POST"})
|
|
request.state.user = user
|
|
request.state.auth_source = AUTH_SOURCE_SESSION
|
|
setattr(app.state, EXTENSION_PRINCIPAL_RESOLVER_KEY, lambda incoming: ExtensionPrincipal(str(incoming.state.user.id)))
|
|
return request
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _isolated_authorization():
|
|
reset_app_config()
|
|
gateway_authz._plugin_provider_cache.clear()
|
|
yield
|
|
gateway_authz._plugin_provider_cache.clear()
|
|
reset_app_config()
|
|
|
|
|
|
async def test_action_route_offloads_config_load_and_provider_discovery(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""Deny path of POST /api/plugins/{ns}/actions/{name} (the real route function)."""
|
|
probe = _blocking_probe(tmp_path)
|
|
app_config = _app_config(roles={"user": {"plugin_actions": {"allow": []}}})
|
|
set_app_config(app_config)
|
|
|
|
config_loads: list[str] = []
|
|
discoveries: list[str] = []
|
|
|
|
def blocking_config_load() -> AppConfig:
|
|
probe.read_text(encoding="utf-8")
|
|
config_loads.append("loaded")
|
|
return app_config
|
|
|
|
discover = gateway_authz.resolve_authorization_provider_spec
|
|
|
|
def blocking_discovery(config: AuthorizationConfig):
|
|
probe.read_text(encoding="utf-8")
|
|
discoveries.append("discovered")
|
|
return discover(config)
|
|
|
|
monkeypatch.setattr(gateway_authz, "_plugin_app_config", blocking_config_load)
|
|
monkeypatch.setattr(gateway_authz, "resolve_authorization_provider_spec", blocking_discovery)
|
|
|
|
request = _action_request(_plugin_extensions())
|
|
with pytest.raises(HTTPException) as error:
|
|
await invoke_plugin_action(request, namespace=NAMESPACE, action_name="check")
|
|
|
|
assert error.value.status_code == 403
|
|
# Both blocking helpers really ran — inside workers, or the gate would have raised.
|
|
assert config_loads
|
|
assert discoveries == ["discovered"]
|
|
|
|
|
|
async def test_async_management_resolver_offloads_the_same_resolution(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""The resolver installed at EXTENSION_PLUGIN_AUTHZ_RESOLVER_ASYNC_KEY."""
|
|
probe = _blocking_probe(tmp_path)
|
|
app_config = _app_config(roles={"user": {"plugin_management": {"allow": []}}})
|
|
set_app_config(app_config)
|
|
|
|
config_loads: list[str] = []
|
|
|
|
def blocking_config_load() -> AppConfig:
|
|
probe.read_text(encoding="utf-8")
|
|
config_loads.append("loaded")
|
|
return app_config
|
|
|
|
discover = gateway_authz.resolve_authorization_provider_spec
|
|
|
|
def blocking_discovery(config: AuthorizationConfig):
|
|
probe.read_text(encoding="utf-8")
|
|
return discover(config)
|
|
|
|
monkeypatch.setattr(gateway_authz, "_plugin_app_config", blocking_config_load)
|
|
monkeypatch.setattr(gateway_authz, "resolve_authorization_provider_spec", blocking_discovery)
|
|
|
|
answer = await _resolve_extension_plugin_management_async(_unknown_request(extensions=_plugin_extensions()), NAMESPACE, "read")
|
|
|
|
assert answer is False # denied by policy, decided without blocking the loop
|
|
assert config_loads # reached inside a worker, or the gate would have raised
|
|
|
|
|
|
async def test_batch_decision_offloads_the_synchronous_provider_call(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""``filter_resources`` is synchronous, so the batch helpers hop to a worker."""
|
|
probe = _blocking_probe(tmp_path)
|
|
module = ModuleType("plugin_blocking_io_test_provider")
|
|
|
|
class BlockingFilterProvider(RbacAuthorizationProvider):
|
|
def filter_resources(self, principal, resource_type, candidates):
|
|
probe.read_text(encoding="utf-8")
|
|
return list(candidates)
|
|
|
|
module.BlockingFilterProvider = BlockingFilterProvider
|
|
monkeypatch.setitem(sys.modules, module.__name__, module)
|
|
|
|
app_config = _app_config(roles={"user": {}}, provider_use=f"{module.__name__}:BlockingFilterProvider", provider_config={"roles": {"user": {}}})
|
|
candidates = [f"{NAMESPACE}/permissions.read"]
|
|
|
|
allowed = await afilter_plugin_management(principal=Principal(user_id="user-1", role="user"), app_config=app_config, candidates=candidates)
|
|
|
|
assert allowed == frozenset(candidates)
|