mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
82671596f3
* feat(authz): add plugin resource targets, provenance, action and management enforcement
PR1 of the extension-tool-authorization series
(docs/plans/2026-09-24-extension-tool-authorization-spec.md): plugin
authorization plumbing, no new author-facing declaration.
- `authz/plugin_targets.py` is the single encoder/validator for composite
targets (`namespace/part`), with one validating composer behind three
kind-checked constructors; invalid input raises instead of degrading.
- `authz/plugin_authz.py` is the harness decision layer (`plugin_action`,
`plugin_management`, plus the batch filters the page slice will consume):
disabled is a no-op, an explicit deny raises, and provider errors follow
`fail_closed` like the sandbox gate.
- `POST /api/plugins/{ns}/actions/{name}` decides `plugin_action:invoke`
after the action resolves and before the body is streamed.
- `deerflow_extension_api.auth` publishes `require_plugin_management` /
`arequire_plugin_management` with two `app.state` resolver keys; the
Gateway installs both. Contract version 0.2.3 -> 0.2.4.
- `tools/tool_provenance.py` records host-written plugin provenance for
display (assembly descriptor, Layer-2 context later); no decision reads a
label.
- Plugin providers use a `(config signature, loop key)` cache so a
loop-affine provider is never handed to another loop or to a sync caller.
- RBAC gains `plugin_actions` / `plugin_management` aliases;
`config_version` 47 -> 48 across the three mirrors.
Tests: new plugin target/provenance/action/management suites plus a
blocking-IO anchor whose offloads were reverted to prove red->green.
* fix(authz): decide plugin resources from the request's config snapshot
Review round 1 on PR #5842 (willem-bd).
P1 — the decision layer no longer reloads the configuration. The six
`authz/plugin_authz.py` functions take a required `app_config` snapshot (the
same request-scoped config the Gateway used to resolve the provider), and
`resolve_/aresolve_plugin_authorization` return it as a third element, so
`enabled`/`fail_closed` can never come from a second read that drifted from
the one that produced the provider. A snapshot the caller could not read at
all is *unavailable*, not disabled: it fails closed as
`authz.config_unavailable`. The Gateway's `_plugin_app_config` now separates
an absent `config.yaml` (no gate, today's behavior) from a config that exists
but cannot be read or validated (deny) instead of swallowing both through
`safe_app_config`.
P2 — `_validated_decision` requires `type(decision.allow) is bool` on both
the sync `authorize` and async `aauthorize` paths, so a provider returning
`AuthzDecision(allow="false")` is a malformed verdict that follows
`fail_closed` instead of being read as an allow.
Both fixes are covered by regression tests that were verified red against the
pre-fix behavior (P1: 200 vs 403 and resolver True vs False; P2: 12 cases
across fail_closed/fail_open and sync/async), plus tests pinning that an
absent config and `authorization.enabled: false` keep today's behavior.
* fix(authz): treat a request-time config absence as unavailable
Review round 2 on PR #5842 (willem-bd).
P1 — `_plugin_app_config[_async]` no longer maps a missing `config.yaml` to
`None` ("no gate"). A Gateway cannot have started without a readable config
(`app/gateway/app.py` reads it through this same accessor in `lifespan`), so
absence *during a request* means the configuration this process was running on
became unavailable — a hot reload, an atomic replace, a ConfigMap remount. It is
unavailable, not disabled: the error propagates,
`resolve_/aresolve_plugin_authorization` raise
`_PluginAuthorizationUnavailable(fail_closed=True)`, the action dispatcher
returns 403 and the management resolvers answer `False`. The dead
`(None, None, None)` allow path is gone, the resolvers return a non-optional
`AppConfig` as their third element, and callers key "authorization disabled" on
`provider is None` alone. This matches the rule the decision layer already
documented (`authz.config_unavailable`) and the public docs.
P2 — `_deny_reason_code` is total: a `reasons` that is not iterable degrades the
code to `authz.denied` instead of raising `TypeError` outside the guarded
provider path, which turned a denial into an unhandled error. It is deliberately
not folded into `_validated_decision`: `allow` was a real `bool`, so the verdict
is a valid denial, and treating it as malformed would flip it to an allow under
`fail_closed: false`.
Both fixes are covered by regression tests verified red against the pre-fix
behavior (P1: resolver True vs False, async `PermissionError` not raised, route
200 vs 403; P2: `TypeError: 'NoneType' object is not iterable` at
`plugin_authz.py:113`). The action-route tests in
`tests/test_plugin_contributions.py` now install a readable
`authorization.enabled: false` config instead of relying on a config-less
process, and `docs/full-stack-plugins.md` plus the PR1 decision log record the
new rule.
* fix(authz): apply the running policy when the plugin config is unreadable
Follow-up to f03d2ea9 (review round 2 on PR #5842, willem-bd).
The first attempt read the P1 finding as "absence is never disabled" and failed
every config read closed unconditionally. That is too broad for this repo: a
config-less host is deliberately treated as having no policy to apply
(`_get_route_authorization_config` falls back to a disabled config,
`sandbox_authz.safe_app_config` documents the same no-op for "CI runners and
direct-call tests"), and the rule turned the config-less e2e plugin host
(`extension_test_fixtures.bookmark_plugin_gateway`) into a 403 — the three
`bookmark-plugin.spec.ts` cases lost the save action. Reproduced locally:
`POST /api/plugins/community.bookmarks/actions/save` against that fixture host
returned 403 before, 200 after this change.
The rule is now "has this process loaded a configuration?", not "was the last
read successful?":
- No loaded config (`peek_loaded_app_config()`, new read-only accessor on
`deerflow.config.app_config`) and no `config.yaml`: no policy to apply, so the
gate stays a no-op — unchanged for CI runners, direct-call tests and hosts
that mount only the plugins router.
- A loaded config whose file disappears or cannot be read: *unavailable*, never
*disabled*. The resolvers raise `_PluginAuthorizationUnavailable` carrying the
failure flag of the configuration this process runs on (authorization
disabled there -> no gate; enabled -> that policy's `fail_closed`, `True` by
default), so an enabled Gateway answers 403/False in the window instead of
letting protected plugin actions and management routes through.
- A config file that exists but cannot be read or validated right now: fail
closed when nothing is loaded (round 1's rule), otherwise the loaded policy
decides.
`peek_loaded_app_config()` is preferred over a Gateway-side startup snapshot: it
reflects the last successfully loaded config, so a hot-reloaded policy is what
applies during the window, and there is no lifecycle hook to keep in sync.
Evidence: the three no-config/lost-config branches in
`tests/test_plugin_management_guard.py` and
`tests/test_plugin_action_authorization.py`, round 1's unreadable-config tests
unchanged, `test_peek_loaded_app_config_survives_a_missing_file` in
`tests/test_app_config_reload.py`, and the fixture-host probe above. Docs
(`docs/full-stack-plugins.md`, `reference.mdx` en/zh) and the PR1 decision log
state the final rule; the log records the superseded variant.
* fix(authz): keep a denial whose reasons raise while being read
* fix(authz): intersect plugin batch filters with the candidates
The plugin_page/plugin_management batch helpers returned the provider's
`filter_resources` answer verbatim, so a custom provider could hand the
projections a target that was never asked about ("*", another namespace,
a phantom page). Every other filter call site re-projects from its own
candidate list; do the same here, state the intersection on the provider
contract, and pin it with a regression test that fails on the old
pass-through.
* docs(plugins): map a management denial to 403 in the route example
The contributed-route example called `arequire_plugin_management` and let
the documented `PermissionError` escape. The Gateway installs no
`PermissionError` handler, so a route copied from the example answered 500
on a normal policy denial instead of 403. Both example routes now translate
the denial, and the surrounding bullets say why the translation is the
route's job.
* docs(extensions): point the 0.2.4 contract row at the PR that ships it
Every other row in the version-history table links the pull request that
shipped the version; 0.2.4 linked the authorization tracking issue instead.
Both the English and Chinese tables now link this PR.
118 lines
4.7 KiB
TOML
118 lines
4.7 KiB
TOML
[project]
|
|
name = "deerflow-harness"
|
|
version = "2.1.0"
|
|
description = "DeerFlow agent harness framework"
|
|
requires-python = ">=3.12"
|
|
dependencies = [
|
|
"agent-client-protocol>=0.4.0",
|
|
"agent-sandbox>=0.0.30",
|
|
"beautifulsoup4>=4.12",
|
|
"html5lib>=1.1",
|
|
"croniter>=6.0.0",
|
|
# Exact pin by design (extension-system version contract): the host pins
|
|
# the contract version it implements, extensions declare ranges. A range
|
|
# here would let pip resolve a newer contract package than this harness
|
|
# implements, making newer extensions look supported at runtime.
|
|
"deerflow-extension-api==0.2.4",
|
|
"dotenv>=0.9.9",
|
|
"exa-py>=1.0.0",
|
|
"httpx>=0.28.0",
|
|
"jsonschema>=4.26.0",
|
|
"kubernetes>=30.0.0",
|
|
# Lower bound reflects what the lockfile resolves and tests run against
|
|
# (langgraph 1.2.9 pulls langchain >=1.3 transitively).
|
|
"langchain>=1.3",
|
|
"langchain-anthropic>=1.4.1",
|
|
"langchain-deepseek>=1.0.1",
|
|
"langchain-mcp-adapters>=0.2.2",
|
|
"langchain-openviking==0.1.0",
|
|
"langchain-openai>=1.2.1",
|
|
"langfuse>=3.4.1",
|
|
"langgraph>=1.2.9,<1.3",
|
|
"langgraph-api>=0.8.1",
|
|
"langgraph-cli>=0.4.24",
|
|
# Standalone Studio's pre-runtime persistence repair is validated against
|
|
# the 0.30.0 store lifecycle and must not resolve an older implementation.
|
|
"langgraph-runtime-inmem>=0.30.0",
|
|
"markdownify>=1.2.2",
|
|
"markitdown[all,xlsx]>=0.0.1a2",
|
|
"packaging>=24.2",
|
|
"pydantic>=2.12.5",
|
|
"pyyaml>=6.0.3",
|
|
"readabilipy>=0.3.0",
|
|
"tavily-python>=0.7.17",
|
|
"firecrawl-py>=1.15.0",
|
|
"tiktoken>=0.8.0",
|
|
"ddgs>=9.10.0",
|
|
"duckdb>=1.4.4",
|
|
"langchain-google-genai>=4.2.1",
|
|
# Floor matters: the InMemorySaver delta-history patch was removed once
|
|
# 4.2.0 fixed the dropped first post-migration write upstream (#8526) and
|
|
# kept its own override, so a 4.1.x resolution would silently reintroduce
|
|
# the bug. langgraph's own constraint (langgraph-checkpoint>=4.1.0) does
|
|
# not exclude that version.
|
|
"langgraph-checkpoint>=4.2.0,<5.0",
|
|
"langgraph-checkpoint-sqlite>=3.1.1,<3.2",
|
|
"langgraph-sdk>=0.1.51",
|
|
"sqlalchemy[asyncio]>=2.0,<3.0",
|
|
"aiosqlite>=0.19",
|
|
"alembic>=1.13",
|
|
"cryptography>=50.0.0",
|
|
"e2b-code-interpreter>=2.8.0",
|
|
# zoneinfo fallback: ZoneInfo reads the OS zone database first and this
|
|
# wheel when the image has none (stripped containers, Windows without
|
|
# tzdata), keeping the DEER_FLOW_DATE_TIMEZONE knob portable.
|
|
"tzdata>=2025.1",
|
|
]
|
|
|
|
[project.scripts]
|
|
deerflow = "deerflow.tui.cli:main"
|
|
|
|
[project.optional-dependencies]
|
|
# Terminal workbench (TUI). Kept optional so the core harness install stays lean;
|
|
# the `deerflow` console script degrades to headless help when textual is absent.
|
|
tui = ["textual>=0.80"]
|
|
# GroundRoute needs no extra packages (httpx is already a core dependency). This
|
|
# empty extra exists so the documented `uv add 'deerflow-harness[groundroute]'`
|
|
# install command resolves cleanly without an "unknown extra" warning.
|
|
groundroute = []
|
|
ollama = ["langchain-ollama>=0.3.0"]
|
|
postgres = [
|
|
"asyncpg>=0.29",
|
|
# 3.1.2 finds plain-value delta seeds (upstream #8535); 3.1.1 walks past
|
|
# them to the thread root instead.
|
|
"langgraph-checkpoint-postgres>=3.1.2,<3.2",
|
|
"psycopg[binary]>=3.3.3",
|
|
"psycopg-pool>=3.3.0",
|
|
]
|
|
# Cross-process SSE stream bridge (stream_bridge.type: redis). Optional so
|
|
# single-process / memory-bridge installs do not pull redis. The Docker image
|
|
# always installs this extra because Docker defaults to the redis bridge.
|
|
redis = ["redis>=5.0.0"]
|
|
pymupdf = ["pymupdf4llm>=0.0.17"]
|
|
boxlite = ["boxlite>=0.9.7"]
|
|
# Tenki cloud sandbox provider (deerflow.community.tenki). Optional so a default
|
|
# install stays free of the Tenki SDK; only pulled in when the provider is used.
|
|
# The distribution is ``tenki``; it still ships the ``tenki_sandbox`` module that
|
|
# provider.py and sandbox.py import.
|
|
tenki = ["tenki>=1.0.0"]
|
|
# OpenSandbox remote sandbox provider (deerflow.community.opensandbox). The
|
|
# sync SDK is loaded only when this provider is selected.
|
|
opensandbox = ["opensandbox>=0.1.15,<0.2.0"]
|
|
# Agent observability (Monocle). Optional so a default install stays free of the
|
|
# OpenTelemetry stack; only pulled in when MONOCLE_TRACING is used.
|
|
monocle = ["monocle_apptrace>=0.8.8"]
|
|
# Agentic browser control (browser_navigate/click/type/... tool group). Optional
|
|
# so the core harness install stays lean; import is lazy inside the private
|
|
# Playwright loop. After install, run `playwright install chromium` once.
|
|
browser = ["playwright>=1.40"]
|
|
# Optional Chinese tokenization for the FTS5 memory retrieval adapter.
|
|
memory-zh = ["jieba>=0.42.1"]
|
|
|
|
[build-system]
|
|
requires = ["hatchling"]
|
|
build-backend = "hatchling.build"
|
|
|
|
[tool.hatch.build.targets.wheel]
|
|
packages = ["deerflow"]
|