11 Commits

Author SHA1 Message Date
hataa
540940bac1
feat(authz): enforce model authorization at Gateway routes and runtime (#4063 Phase 3) (#4540)
* feat(authz): enforce model authorization at Gateway routes and runtime (#4063 Phase 3)

Phase 3 / Models — the first of three resource-type PRs (Models, Skills,
Sandbox). The RBAC provider already maps "model" → config key "models"
(rbac.py _RESOURCE_POLICY_KEYS), so no schema change is needed.

Gateway route layer (mirrors Phase 2A):
- resolve_model_authorization() in authz.py returns (provider, principal),
  reusing _get_cached_route_provider and build_principal_from_context,
  including the INTERNAL_SYSTEM_ROLE → None pop for internal callers.
- list_models filters via provider.filter_resources(principal, "model", names).
- get_model checks provider.authorize("model", "use"). Deny → 403 (not 404,
  since the model exists but the role lacks permission).

Runtime resolution layer (mirrors Phase 1B):
- _authorize_model_name() in agent.py runs after _resolve_model_name. On deny,
  falls back to the first allowed model (RFC §9: graceful, not crash). All
  models denied + fail_closed → ValueError (matches existing contract).

authorization.enabled: false is a complete no-op on both layers. Anonymous
requests (user=None) bypass filtering. 18 new tests + 314 existing tests pass.

* fix(authz): enforce model:use on the embedded DeerFlowClient path (Phase 3 follow-up)

Round 4 review (willem-bd): _authorize_model_name only covered the Gateway
runtime path (_make_lead_agent). The parallel lead-agent construction path
DeerFlowClient._ensure_agent (client.py) filtered tools but not the model,
so a library/embedded consumer with role-scoped model policies could run a
model the role is denied model:use for.

- Insert _authorize_model_name in _ensure_agent, mirroring _make_lead_agent.
- Resolve None default to the first configured model before the gate so the
  implicit default (create_chat_model(name=None)) is also authorized.
- Update test_authorization_filters_framework_tools_and_reuses_provider: the
  stub provider now returns an allow decision for model:use (checked during
  assembly) and patches resolve_authorization_provider in the agent namespace.
- Add 3 DeerFlowClient._ensure_agent path tests (real-path fallback,
  None-default resolution, disabled no-op); 24 tests total.

* docs(authz): document get_model provider-unavailable fail-open path + test

zhfeng review (round 5): get_model's docstring only mentioned the deny→403
path, not the provider-resolution-error + fail-open path (which allows the
request, mirroring list_models's documented fail-open semantics). The
behavior itself is correct and symmetric with list_models, but it was
undocumented and the _AuthorizationUnavailable path had no test coverage.

- Extend get_model docstring to state the provider-error fail-closed/fail-open
  outcome, matching list_models's wording.
- Add test_get_model_provider_unavailable_fail_closed_vs_open exercising the
  _AuthorizationUnavailable path (provider cannot be resolved at all), pinning
  fail-closed→403 / fail-open→200.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-08-01 22:34:11 +08:00
RongfuShuiping
2aaf74b0f8
feat(memory): add OpenViking HTTP backend (#4509)
* feat(memory): add OpenViking HTTP backend

* fix(memory): harden OpenViking lifecycle

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-28 23:36:25 +08:00
hataa
6091ce7576
feat(authz): derive Gateway route permissions from AuthorizationProvider (#4439)
* feat(authz): derive Gateway route permissions from AuthorizationProvider (Phase 2A, #4063)

Phase 2A: replace legacy _ALL_PERMISSIONS with provider-derived route
permissions. When authorization.enabled, each threads:*/runs:* permission
is evaluated independently via provider.aauthorize(resource='route').
Disabled mode preserves legacy behavior. owner_check and require_admin_user
remain unchanged.

10 new tests: disabled/enabled/RBAC policy/fail-closed/fail-open/
authenticate integration/middleware integration.

* fix(authz): move AuthorizationConfig import to runtime for fallback in _get_route_authorization_config
2026-07-27 14:19:04 +08:00
hataa
7857fa0cce
feat(authz): enforce tool authorization at assembly and runtime (#4370)
* feat(authz): enforce tool authorization at assembly and runtime

* fix(middleware): guard deferred tool setup lookup (#4370)

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-23 22:51:35 +08:00
RongfuShuiping
4bf028d048
feat(memory): add incremental agent-scoped Markdown fact storage (#4279)
* feat(memory): add Markdown fact storage repository

* docs(memory): explain storage rewrite for beginners

* docs(memory): fix plan markdown formatting

* refactor(memory): separate global summaries from agent facts

* fix(memory): make Markdown fact updates incremental and safe

* Update STORAGE_REWRITE_CHANGES.md

* Delete docs/plans/STORAGE_REWRITE_PLAN.md

* Delete docs/plans/STORAGE_REWRITE_CHANGES.md

* fix(memory): address Markdown storage review feedback

* fix(memory): complete review follow-ups

* fix(memory): resolve storage review findings

* feat(memory): add proactive Markdown migration CLI

* fix(memory): harden Markdown storage concurrency

* fix(memory): harden markdown storage migration

* fix(memory): close migration review gaps

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: qin-chenghan <qinchenghan@Huawei.com>
2026-07-22 07:46:03 +08:00
Ryker_Feng
fa496c0c8d
feat(browser): add agentic browser control (#4187)
* feat(browser): add agentic browser control

* fix(frontend): format browser view changes

* fix(browser): keep browser optional and isolate sidecar layout

* fix(browser): address PR review security and IME findings

- Nginx: add a browser-stream WebSocket location before the generic
  /api/threads regex so Live upgrades instead of downgrading to HTTP
  (both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
  navigate, and tear down the browser session on thread deletion so a
  later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
  context-level route guard covering redirects, popups, iframes, and
  subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
  CJK candidate with Enter no longer submits the remote page form.

Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.

* fix(frontend): smooth streaming in long tool threads

* Revert "fix(frontend): smooth streaming in long tool threads"

This reverts commit f0462516eabe77f138d4027ea1c714fb226683cf.

* fix(browser): address review security and lifecycle findings

- Reject cross-origin WebSocket upgrades on the live browser stream
  (Origin allow-list reuse of CORS/same-origin helpers) to close a
  WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
  off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
  (browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
  and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
  can't permanently stall after the cumulative attempt cap.

* fix(frontend): reduce long tool thread render stalls

Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.

* fix(browser): keep live control responsive during continuous input

Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").

What:
- Frontend forwards one `click` per physical click instead of also emitting
  `down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
  start a rate-limited background refresh loop (leading frame + bounded cadence)
  that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
  inputs coalesce, and continuous input keeps refreshing before it stops.

Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.

* fix(browser): harden worker and session lifecycle

* fix(browser): address latest review feedback

* fix(frontend): preserve optimistic new-chat message

* test(e2e): preserve mocked message run ids

* fix(browser): address capability review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-21 11:46:33 +08:00
hataa
92c8f2f03b
feat(authz): add built-in RBAC provider and provider factory (#4260)
* feat(authz): add built-in RBAC provider and provider factory (Phase 1A-2, #4063)

Phase 1A-2: RBAC provider + provider factory. No runtime behavior change.

New authz/rbac.py — RbacAuthorizationProvider:
- allow: '*' / True / list / [] / missing → deny-wins semantics
- deny always overrides all allow forms
- resource name explicit mapping (tool→tools, model→models, etc.)
- unknown/missing role raises ValueError (never silent allow)
- config compiled to immutable frozensets at construction
- filter_resources preserves order, no mutation, consistent with authorize
- sync == async decisions

New authz/runtime.py — resolve_authorization_provider:
- disabled → None (no import attempted)
- enabled + no provider → ValueError
- invalid class path / construction failure → ValueError with path
- isinstance Protocol check post-construction
- no caching, no fail_closed/default_role injection

48 tests (37 RBAC + 11 factory). No config schema change, no config_version bump.
Per RFC #4063 Phase 1A-2. Layer 1/Layer 2 wiring deferred to Phase 1B.

* fix(authz): reject unknown RBAC provider config

Fail fast on misspelled top-level RBAC settings, cover factory error propagation, and record Phase 1B policy and audit caveats.

* fix(authz): reject unreachable resource aliases

Fail fast when RBAC config uses reserved request-side aliases, preserve same-name and custom resources, and add regression coverage for every mapped alias.

* fix(authz): validate RBAC request identifiers
2026-07-21 09:23:14 +08:00
Ryker_Feng
a028dfd5fb
feat(auth): add keep me signed in login option (#4255)
* feat(auth): add keep me signed in login option

* fix(auth): address remember-login review feedback
2026-07-18 22:17:15 +08:00
hataa
10890e10a8
feat(authz): propagate trusted authorization principal context (#4203) 2026-07-17 14:49:51 +08:00
hataa
1300c6d36b
feat(authz): add pluggable AuthorizationProvider protocol and config scaffolding (Phase 0, #4063) (#4127)
Phase 0 of the RFC in #4063: scaffolding only, zero behavior change.

New deerflow/authz/ package (sibling to deerflow/guardrails/):
- AuthorizationProvider Protocol: authorize() + aauthorize() + filter_resources()
- Principal/AuthzRequest/AuthzDecision/AuthzReason dataclasses
- GuardrailAuthorizationAdapter: bridges AuthorizationProvider → GuardrailProvider
  so existing GuardrailMiddleware can enforce authz decisions without a new middleware

New authorization config section (default enabled: false):
- AuthorizationConfig wired into AppConfig alongside guardrails
- Singleton load/reset mirrors GuardrailsConfig pattern
- config.example.yaml documents the RBAC provider schema

29 tests covering protocol conformance, dataclass construction, adapter
request/decision mapping, and config singleton behavior.

Per RFC #4063 Phase 0 (foundations). Layer 1/2 wiring and Principal builder
in services.py deferred to Phase 1.
2026-07-15 10:03:33 +08:00
totoyang
2d1f90d5dc
feat(tracing): add optional Langfuse support (#1717)
* feat(tracing): add optional Langfuse support

* Fix tracing fail-fast behavior for explicitly enabled providers

* fix(lint)
2026-04-02 13:06:10 +08:00