3 Commits

Author	SHA1	Message	Date
Zeren Wang	5951c89b5b	feat(projects): project workspaces with scoped chats and thread membership (#5265) ... * feat(projects): project workspaces with scoped chats and thread membership Backend: - projects table model and migration; fail-closed ProjectRepository with ownership checks, CRUD/archive/restore/delete router, and atomic thread move between projects - threads_meta.project_id column exposed as reserved deerflow_project_id metadata; project-aware thread create/search with pagination bounds and membership echoed in create responses - first-run admission assigns the project only at genuine first run, seeded at write time and dropped when invalid; serialized against project deletion and thread assignment - branch creation inherits the source thread's project membership (an archived/deleted project degrades the branch to unassigned instead of failing the request) Frontend: - projects data layer, thread move API, and sidebar projects section with flat/grouped modes, archived-project threads, and stable virtual-list offsets - project detail page with project-scoped new chat (/workspace/chats/new?project=) and paginated thread list - move-to-project thread menu, new-project dialog, archived-project gates - project-scoped new chats pre-create the thread with membership before the first submit or /goal set, so runs never proceed outside the project - goal-set preparation is fenced against conversation switches: a stale continuation is dropped instead of saving the goal or launching the abandoned submission on the newly opened conversation - project thread lists join thread lifecycle invalidations (stop, pin) so an open project page never keeps stale titles, recency, or pagination * fix(chats): keep archive undo toast when the sidebar row unmounts The archive success toast was fired from per-mutate callbacks passed to mutation.mutate. React Query drops those handlers when the observer component unmounts before the mutation settles; archiving the open chat removes its sidebar row mid-flight, so the undo toast never appeared and the e2e archive-undo test timed out waiting for it. Move the success/error handlers to the mutation level (useArchiveThread options, same pattern as useMoveThreadToProject) where callbacks are delivered even after the originating row unmounts. * fix(projects): pin project thread listing contract and exclude archived chats GET /api/projects/{id}/threads returned the thread store row verbatim (list[dict], no response_model): user_id/assistant_id leaked, any future ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty. Return a narrow ProjectThreadResponse (the exact fields ProjectThread declares) with the same metadata secret redaction the surrounding thread endpoints get from _MetadataRedactingResponse. The listing also ran search() without the archived filter, so a retired chat rendered as a normal row on the project page while the sidebar hid it. Search archived=False to mirror the sidebar's archived:false lists; restore stays on the global Archived tab. Both regressions pinned by new router tests: wire-shape allowlist and archived-member exclusion. * docs(migrations): record the 0019/0020 chain against the bootstrap reservation The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id, so migrations/AGENTS.md was stale twice over: the revision index stopped at 0018 and the rolling-forward section still claimed the tree 'deliberately remains at 0018'. Document the new head and record the intentional numeric-prefix reuse of 0019: 0019_projects is in-chain while 0019_thread_incarnations stays the reserved, allowlisted out-of-tree rollout id. The owning rollout revision must re-parent onto this tree's head when it merges so alembic never sees two heads off 0018; bootstrap.py now cross-references that note next to _FORWARD_COMPATIBLE_REVISION. * fix(chats): invalidate project thread lists on archive/restore useArchiveThread refreshed the infinite sidebar cache, threads/search and the per-thread metadata cache but not the project-scoped list ([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread mutation not wired to that key, after usePinThread, useRenameThread, useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches. An archive from a sidebar row while a project page is open therefore left the archived chat rendered as a normal row until remount (and undo left it missing). Invalidate the prefix in the mutation-level success handler. Regression test asserts the project-list prefix is invalidated on success. * fix(projects): fetch project discovery only in grouped sidebar mode RecentChatList mounted two useProjects queries per sidebar render, but knownProjectIds is consumed only by the grouped-mode exclusion filter; in the default flat mode every page load paid two GET /api/projects?status= round trips for data nothing read. Gate both queries on grouped mode — GroupedProjectList fetches the same keys when the toggle is on and TanStack dedupes the observers. Also set retry: false on useProject: a deleted or foreign project 404s deterministically, and the page renders a dedicated not-found state for it, so the default 1s/2s/4s retry backoff kept deep links in 'loading' for ~7s before that state appeared. Matches useThreadMetadata / useThreadTokenUsage. * fix(threads): fail closed on project-scoped create in memory mode MemoryThreadMetaStore.create accepted project_id and silently ignored it, making memory mode the one membership path that fails open: POST /api/threads with a project id returned 200 and the run started unassigned, violating the invariant that a run never proceeds outside the selected project (the SQL store raises ProjectNotAssignableError inside the insert transaction for the same request). Raise ProjectNotAssignableError whenever project_id is present so the router's existing 404 mapping applies, the frontend keeps the composer text for a retry, and memory mode behaves exactly like SQL mode. set_project already reports rejection; create now matches it. Store-level test (raises, nothing persisted, project filter stays empty, unscoped creates still work) plus a router-level test asserting the 404 and that no row is left behind. * fix(projects): window the project page thread list ProjectThreadsSection rendered every loaded page as a plain Link row, so a long-lived project accumulated unbounded DOM on the page's scroll surface: each load-more appended another 100 rows and every formatTimeAgo tick re-rendered the whole list. Reuse VirtualThreadList (now generic over any row shape with a thread_id), pointing its scroll parent at this page's ScrollArea viewport via the shared [data-slot="scroll-area-viewport"] selector used by /workspace/chats; under the 60-row threshold it falls back to the plain render, so small projects are unchanged. * fix(projects): restore row dividers and pin them with a render test The row class template literal concatenated transition-colors directly with the conditional border-b token, so non-final rows rendered the invalid class 'transition-colorsborder-b' and lost both the divider and the transition. Compose the row classes with cn() and a boolean guard instead. The section moved out of page.tsx into a testable component so the row markup finally has coverage: a DOM test asserts every row except the final data row carries border-b (index-based, not last: — correct under virtualization where the last mounted row is not the last data row), and the untitled fallback plus load-more button render for a partial page. * fix(projects): validate forward schemas and fence membership reads	2026-09-08 17:00:26 +08:00
Sunshine	a956bbc030	fix(runs): reject cancel actions on GET stream joins (#5092) ... * fix(runs): reject cancel actions on GET stream joins stream_existing_run is registered for both GET and POST, and its ?action=interrupt|rollback branch cancels the run. The CSRF middleware exempts GET, so a session-authenticated browser could be forced cross-site (img/script/top-level navigation) into GET /api/threads/{id}/runs/{run_id}/stream?action=interrupt|rollback — a state-changing GET that bypasses the CSRF protection guarding the POST variant. Introduced with the dual registration in #1403. The handler's docstring already documents cancel-then-stream as POST-only (the LangGraph SDK's joinStream/useStream stop button uses POST); enforce it: GET with an action answers 405, action-less GET joins and POST cancel-then-stream are unchanged. Regression drives the real router: GET+action is 405 with the run left running, plain GET join still streams, POST+action still cancels. * fix(runs): scope the 405 detail to the action requirement "GET is a read-only stream join" overstates the current main: on a locally-owned run with the default on_disconnect=cancel, a GET join's disconnect can still trigger cancellation. That observer-disconnect vector is closed by #5041; the detail here should only claim what this guard enforces. * fix(runs): harden GET stream action rejection * fix(runs): align stream schema with method contract * test(runs): pin GET stream action 405 through the production stack Review follow-up (defence-in-depth): the GET-action suite drove bare FastAPI() apps, so nothing pinned that a session-authenticated cross-site GET reaches the route gate at all once CSRF exempts the safe method. test_pat_auth.py already assembles the production middleware order (AuthMiddleware inner, CSRFMiddleware outer), so its mirror app now registers the real _reject_get_stream_action dependency on a GET join route. The new case pins the end-to-end premise: an authenticated GET ?action=interrupt is answered 405 + Allow: POST by the production route dependency, while the same unauthenticated GET dies at AuthMiddleware's 401 before any route logic runs. Validation: focused suites (test_pat_auth, test_stream_get_action, test_csrf_middleware) — 62 passed; ruff check + format clean; the new case errors on the pre-fix baseline (guard absent), confirming the pin.	2026-09-01 15:51:37 +08:00
Sunshine	bf740ffa90	feat(auth): add personal access tokens for programmatic API access (#5041) ... * feat(auth): add personal access tokens for programmatic API access (#4849) Backend-first implementation of the PAT contract from #4849: show-once dfp_ tokens bound to their owning user (AUTH_SOURCE_PAT, is_internal=false), digest-only storage (migration 0017), strict credential precedence (invalid Bearer is a 401, never cookie fallback), CSRF double-submit skipped only for Bearer requests while auth-endpoint origin checks still run, scopes intersecting the authz route permissions, session-auth-only PAT management and password changes, and throttled best-effort last_used_at stamps. * fix(auth): harden PAT scope boundary and schema parity from adversarial review Independent review of the initial draft found: (1) scopes only constrained the threads/runs permission axis while admin routes treated a PAT as its (possibly admin) owner — is_admin_user now rejects PAT callers outright since no scope grants admin capability; (2) the model declared a column UNIQUE constraint while migration 0017 created a named unique index, so downgrade failed on create_all-bootstrapped DBs — both now use the named unique index; (3) auth-disabled mode is an operator override and now stays ahead of the Bearer check so a stray Authorization header cannot 401 an E2E sandbox; plus wiring the previously-unused constants, bounding the last_used_at stamp cache, and four new tests (middleware-level expiry, expires_in_days, admin-capability rejection with session control, and the auth-disabled precedence). * docs(api): document personal access tokens for programmatic API access * fix(auth): close PAT security boundaries from review (default-deny routes, extension admin suppression) P1-1: scope intersection only constrains @require_permission routes, so undecorated mutation routes (DELETE /api/memory, POST /api/agents, Lark credential switching, channel config) accepted a PAT holding a single read scope. AuthMiddleware now enforces a default-deny route policy in auth/pat.py: PAT requests are admitted only to the thread/run lifecycle routes the v1 scopes govern; everything else answers 403 regardless of scopes. Session-cookie callers are unaffected. P1-2: the extension principal resolver projected is_admin/roles from the raw system_role, so an admin-owned PAT passed deerflow_extension_api.require_admin on contributed routes despite the documented no-admin guarantee. The projection is now PAT-aware and suppresses every admin signal for PAT callers, mirroring deps.is_admin_user. Both fixes carry regression tests (route outside policy 403 + session control; production resolver admin suppression), and API.md documents the default-deny boundary. * fix(auth): enforce PAT scopes on stateless run entry and harden decorator Follow-up hardening from an independent audit of the P1 fixes: - POST /api/runs/stream and /api/runs/wait were the only allowlisted run entrypoints without @require_permission, so a threads:read-only PAT could still start runs (same bug class as P1-1, now closed): both now carry @require_permission("runs", "create"). POST /api/threads and POST /api/threads/search gain threads:write / threads:read for the same reason. Authorization-disabled deployments see no change (the permission set resolves to all permissions). - require_permission now binds the wrapped signature to locate a positionally-passed request before injecting the test stub, fixing 'got multiple values for argument' on direct positional unit-test calls. - API.md: the intro PAT example used GET /api/models, which the new default-deny policy 403s — replaced with GET /api/threads; the default-deny route list now spells out method sets. Regression test: threads:read-only PAT is 403 on the decorated stateless entry while a runs:create PAT passes. * fix(auth): address review P2s (empty Authorization header, PAT name trimming, API example) - CSRFMiddleware treats an explicitly empty Authorization header as present (is None), so an invalid credential always reaches AuthMiddleware's uniform 401 instead of a CSRF 403 that varies by method/CSRF state. Regression: empty-header request dies at auth. - PATCreateRequest strips the name and rejects whitespace-only values before token generation; created names are stored trimmed. - API.md intro PAT example now uses the implemented POST /api/threads/search endpoint (GET /api/threads does not exist). - AGENTS.md trimmed back under the guidance soft budget after the upstream merge. * fix(auth): tighten PAT route policy to implemented methods only The allowlist admitted GET /api/threads, a method no router implements. Pre-authorizing a dead method weakens the default-deny boundary: a future GET collection route added without a permission decorator would become PAT-reachable without an explicit policy change. Restrict the rule to POST, fix the stale GET description in API.md's PAT constraints, and document the default-deny boundary accurately in the gateway AGENTS.md guidance (only the threads/runs allowlist is PAT-reachable; every other authenticated route 403s PAT callers). Audited every remaining rule against the mounted routers: all other method+path entries map to real routes. Regression: test_pat_policy_does_not_pre_authorize_unimplemented_methods. * test(auth): guarantee the negative digest test mutates the token token[:-1] + "X" is identical to the original whenever the generated token already ends in X (1/62), making the negative digest assertion fail intermittently. Choose the replacement character based on the existing tail so the mutated token always differs. * fix(auth): require runs:cancel for cancel-then-stream requests stream_existing_run is gated at runs:read so action-less stream joins work with read-only credentials, but its ?action=interrupt|rollback branch cancels the run — a separate permission. A runs:read-only PAT passed both the PAT route policy and the route decorator and could interrupt or roll back an active run, bypassing the runs:cancel scope. Decorators cannot express query-parameter-conditional permissions, so the check lives in require_cancel_permission_when_action(), applied at the top of the handler. Regression drives the real helper through the production middleware: runs:read-only PAT + action is 403, the same token joins action-less, runs:read+cancel passes, session control unaffected. * docs(changelog): add the PAT feature entry * docs(readme): add personal access tokens section Repo documentation-update policy requires user-facing features to update README.md in the same changeset; the PAT feature previously touched only backend/docs/API.md and the gateway AGENTS.md. * fix(auth): require runs:cancel for mutating multitask strategies All five run-creation entrypoints were gated only by runs:create, but RunCreateRequest.multitask_strategy accepts interrupt/rollback and start_run forwards it to create_or_reject, which terminates an already-active run. A runs:create-only PAT could therefore kill an existing run through a create request, bypassing runs:cancel. Decorators cannot express body-parameter-conditional permissions, and per-route checks leave the same hole for the next entrypoint, so the gate lives in start_run itself — the single choke point every run-creation path (HTTP routes and internal launchers) flows through. Regenerate launches pass multitask_strategy="reject" and are unaffected; requests without a stamped auth context (internal/test compositions) skip the gate. The check is the shared authz.require_cancel_permission_if primitive; require_cancel_permission_when_action now delegates to it, so every request dimension that carries cancel capability (query action, body strategy) flows through one gate. Regression drives the real middleware stack: runs:create-only PAT + interrupt/rollback is 403 with the exact detail, reject (explicit and default) stays available, runs:create+cancel passes, session control unaffected; a source anchor pins the gate inside start_run. * fix(runs): keep observer joins from applying creator cancel-on-disconnect sse_consumer's finally block applied the record's on_disconnect=cancel policy on ANY consumer's disconnect. The join surfaces (GET /join and the action-less GET/POST stream join) feed it the existing RunRecord, so anyone with thread read access — including a runs:read-only PAT — could cancel a locally-owned running run simply by closing the SSE connection, without runs:cancel. The policy expresses the creator's intent for their own connection; an observer's disconnect must never be read as that intent. sse_consumer gains apply_on_disconnect (default True). The two join surfaces pass False; the creating endpoints (thread-scoped and stateless create-and-stream) keep the creator semantics unchanged. wait_for_run_completion needs no change: its callers are creator-side or post-explicit-cancel paths only. Regression exercises a real generator close — the same machinery Starlette drives on client disconnect — against the production sse_consumer: creator stream disconnect cancels, observer join disconnect does not; a wiring anchor pins both join call sites and the creator defaults. API.md documents the cancel-capability constraint (this fix plus the action/strategy gates) in PAT Constraints. * test(auth): pin the multitask gate behaviorally; state wait invariant Independent adversarial review of the round-5 fixes found the P1-a regression only mirror-pinned: the source anchor could be satisfied by a comment, and deleting the gate from start_run would not fail the suite. This drives the production start_run directly — a create-only auth context gets 403 with the exact detail for interrupt, and a reject request with no cancel permission at all proceeds past the gate (never a permission 403). Also documents wait_for_run_completion's creator-side invariant (every caller is the creating endpoint or post-explicit-cancel) so a future observer wiring thinks twice before reusing it — the one-caller- away variant of the observer-disconnect P1. * docs(changelog): correct the PAT entry's digest and route-policy description The entry said HMAC digests (the implementation stores SHA-256 digests, as documented in API.md and pinned by the repository tests) and claimed the route policy admits 'implemented stateless endpoints' (it admits the thread/run lifecycle routes, narrowing further by scopes). Also notes the cancel-capability gate now covering action and multitask strategies. * fix(auth): enumerate the PAT runs route policy per implemented subroute The runs subtree rule was a GET|POST /runs(/.*)? wildcard — it pre-authorized every current and future subroute under /runs, including methods the router never implemented (e.g. GET /runs/stream), which is the same latent default-deny weakening the threads collection rule was tightened for: a future route added under /runs would become PAT-reachable without an explicit policy change. The wildcard is replaced with six segment-precise rules covering exactly the 14 implemented method+path combinations; the {run_id} slot necessarily matches any single segment, so the POST-only collection names (stream, wait, regenerate, edit-regenerate) are excluded from the GET run-id rule via negative lookahead — no dead method stays pre-authorized. Behavior for implemented routes is unchanged. test_pat_runs_policy_admits_exactly_the_mounted_routes derives the expected set from the mounted thread_runs router instead of a hand-maintained list: every implemented GET/POST route under /runs must be admitted, routes in this router outside the subtree stay denied, and representative unimplemented neighbors are denied — so adding a route under /runs now fails CI until it is explicitly allowlisted, and a removed route leaves a dead rule visible. API.md's PAT constraints list the enumerated routes and drops a feedback mention that belonged to the stateless /api/runs axis. * docs(migration): add the 0017 renumbering coordination note to 0017 The PR's migration-coordination comment states each migration file carries the note; the file did not. Adds it: numbering was generated against main head 0016 alongside #5078 and #4843; whoever merges first keeps the slot, the others renumber on rebase (revision/down_revision plus the bootstrap head assertions). * fix(auth): pad base62 tokens to a fixed 43-char width int.from_bytes discards leading zero bytes, so the unpadded encoder returned a variable-length body — empty for all-zero input, and shorter than 40 characters for any draw below 62**39 (~1 in 14.5M), leaving test_generate_pat_token_format probabilistically flaky and the token body without stable width (review round 6, P3). _base62 now left-pads with "0" to _base62_width(len(data)) — the exact integer digit count (62^43 > 2^256 > 62^42, so 43 for 32 bytes). The format test asserts the exact fixed width instead of a probabilistic floor, and a new unit test pins the all-zero, leading-zero-byte, and max-value edges deterministically.	2026-08-29 23:50:45 +08:00