Zeren Wang 5951c89b5b
feat(projects): project workspaces with scoped chats and thread membership (#5265)
* feat(projects): project workspaces with scoped chats and thread membership

Backend:
- projects table model and migration; fail-closed ProjectRepository with
  ownership checks, CRUD/archive/restore/delete router, and atomic thread
  move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
  metadata; project-aware thread create/search with pagination bounds and
  membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
  at write time and dropped when invalid; serialized against project
  deletion and thread assignment
- branch creation inherits the source thread's project membership (an
  archived/deleted project degrades the branch to unassigned instead of
  failing the request)

Frontend:
- projects data layer, thread move API, and sidebar projects section with
  flat/grouped modes, archived-project threads, and stable virtual-list
  offsets
- project detail page with project-scoped new chat
  (/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
  first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
  continuation is dropped instead of saving the goal or launching the
  abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
  an open project page never keeps stale titles, recency, or pagination

* fix(chats): keep archive undo toast when the sidebar row unmounts

The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.

Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.

* fix(projects): pin project thread listing contract and exclude archived chats

GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.

The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.

Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.

* docs(migrations): record the 0019/0020 chain against the bootstrap reservation

The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.

Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.

* fix(chats): invalidate project thread lists on archive/restore

useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.

An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.

Regression test asserts the project-list prefix is invalidated on success.

* fix(projects): fetch project discovery only in grouped sidebar mode

RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.

Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.

* fix(threads): fail closed on project-scoped create in memory mode

MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).

Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.

Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.

* fix(projects): window the project page thread list

ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.

Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.

* fix(projects): restore row dividers and pin them with a render test

The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.

The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.

* fix(projects): validate forward schemas and fence membership reads
2026-09-08 17:00:26 +08:00

178 lines
7.6 KiB
Python

"""SQLAlchemy-backed project repository.
Ownership discipline mirrors ``ThreadMetaRepository``: every method resolves
the caller via ``resolve_user_id(..., AUTO)`` and filters by ``user_id``;
a foreign project is indistinguishable from a missing one (callers map
``None``/``False`` to 404). ``delete`` runs the Phase-1 membership-clearing
transaction from RFC v2 §5.1 (the ``project_documents`` statement joins this
transaction in Phase 2).
"""
from __future__ import annotations
import logging
import uuid
from datetime import UTC, datetime
from typing import Any, Literal
from sqlalchemy import delete as sa_delete
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.projects.model import ProjectRow
from deerflow.persistence.thread_meta.model import ThreadMetaRow
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
from deerflow.utils.time import coerce_iso
logger = logging.getLogger(__name__)
ProjectStatus = Literal["active", "archived"]
class ProjectNotAssignableError(ValueError):
"""Raised when a thread cannot be assigned to a project.
The project is missing, foreign to the caller, or archived — the
atomicity rules (RFC v2 §5.2) make these indistinguishable inside the
mutating statement, so callers get one signal and map it to 404 (explicit
API) or a dropped key (run admission).
"""
class ProjectRepository:
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
self._sf = session_factory
@staticmethod
def _row_to_dict(row: ProjectRow) -> dict[str, Any]:
d = row.to_dict()
for key in ("created_at", "updated_at"):
val = d.get(key)
if isinstance(val, datetime):
d[key] = coerce_iso(val)
return d
async def create(
self,
*,
name: str,
instructions: str = "",
presentation: dict | None = None,
user_id: str | None | _AutoSentinel = AUTO,
) -> dict:
resolved_user_id = resolve_user_id(user_id, method_name="ProjectRepository.create")
now = datetime.now(UTC)
row = ProjectRow(
id=uuid.uuid4().hex,
user_id=resolved_user_id,
name=name,
instructions=instructions,
presentation=presentation or {},
status="active",
created_at=now,
updated_at=now,
)
async with self._sf() as session:
session.add(row)
await session.commit()
await session.refresh(row)
return self._row_to_dict(row)
async def get(self, project_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> dict | None:
resolved_user_id = resolve_user_id(user_id, method_name="ProjectRepository.get")
async with self._sf() as session:
row = await session.get(ProjectRow, project_id)
if row is None:
return None
if resolved_user_id is not None and row.user_id != resolved_user_id:
return None
return self._row_to_dict(row)
async def list(
self,
*,
status: ProjectStatus | None = None,
user_id: str | None | _AutoSentinel = AUTO,
) -> list[dict]:
resolved_user_id = resolve_user_id(user_id, method_name="ProjectRepository.list")
stmt = select(ProjectRow).order_by(ProjectRow.created_at.asc(), ProjectRow.id.asc())
if resolved_user_id is not None:
stmt = stmt.where(ProjectRow.user_id == resolved_user_id)
if status is not None:
stmt = stmt.where(ProjectRow.status == status)
async with self._sf() as session:
result = await session.execute(stmt)
return [self._row_to_dict(r) for r in result.scalars()]
async def patch(
self,
project_id: str,
*,
name: str | None = None,
instructions: str | None = None,
presentation: dict | None = None,
user_id: str | None | _AutoSentinel = AUTO,
) -> dict | None:
resolved_user_id = resolve_user_id(user_id, method_name="ProjectRepository.patch")
async with self._sf() as session:
row = await session.get(ProjectRow, project_id)
if row is None or (resolved_user_id is not None and row.user_id != resolved_user_id):
return None
if name is not None:
row.name = name
if instructions is not None:
row.instructions = instructions
if presentation is not None:
row.presentation = presentation
await session.commit()
await session.refresh(row)
return self._row_to_dict(row)
async def set_status(
self,
project_id: str,
status: ProjectStatus,
*,
user_id: str | None | _AutoSentinel = AUTO,
) -> dict | None:
"""Idempotent status flip; returns the current row or None (missing/foreign)."""
resolved_user_id = resolve_user_id(user_id, method_name="ProjectRepository.set_status")
async with self._sf() as session:
row = await session.get(ProjectRow, project_id)
if row is None or (resolved_user_id is not None and row.user_id != resolved_user_id):
return None
row.status = status
await session.commit()
await session.refresh(row)
return self._row_to_dict(row)
async def delete(self, project_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> bool:
"""Delete a project and clear membership in one transaction (no filesystem work).
Returns True when the project row was deleted; False when missing/foreign.
"""
resolved_user_id = resolve_user_id(user_id, method_name="ProjectRepository.delete")
async with self._sf() as session:
async with session.begin():
# Lock the project row before touching it (FOR UPDATE on
# Postgres; the clause renders nothing on SQLite). Membership
# assignment (ThreadMetaRepository.set_project/create) takes
# the same row lock before writing ``project_id``, so an
# assigner either commits first and has its membership cleared
# below, or blocks until this transaction commits and then
# re-reads the row as gone — no dangling ``threads_meta.project_id``
# (RFC v2 §14.14). Missing/foreign rows keep the False result.
lock_stmt = select(ProjectRow).where(ProjectRow.id == project_id)
if resolved_user_id is not None:
lock_stmt = lock_stmt.where(ProjectRow.user_id == resolved_user_id)
locked = (await session.execute(lock_stmt.with_for_update())).scalar_one_or_none()
if locked is None:
return False
if resolved_user_id is not None:
await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.project_id == project_id, ThreadMetaRow.user_id == resolved_user_id).values(project_id=None, updated_at=ThreadMetaRow.__table__.c.updated_at))
result = await session.execute(sa_delete(ProjectRow).where(ProjectRow.id == project_id, ProjectRow.user_id == resolved_user_id))
else:
await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.project_id == project_id).values(project_id=None, updated_at=ThreadMetaRow.__table__.c.updated_at))
result = await session.execute(sa_delete(ProjectRow).where(ProjectRow.id == project_id))
return result.rowcount > 0