mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-28 14:10:37 +00:00
553f85e788
Opt-in per-document summary column rendered into the injected <documents> index, list_project_documents JSON, and the documents API. Pipeline (projects/summaries.py, behind the narrow enqueue_summary interface): - The two shelf write routes enqueue a job on created=True; dedup hits never regenerate. Off by default (projects.summaries_enabled). - Bounded worker pool (summary_concurrency + bounded summary_queue_size; a full queue drops the job) fed by FIFO; shutdown drains queued jobs, delivers one sentinel per worker via blocking put (queue smaller than the worker pool cannot strand idle workers), and bounds only genuinely in-flight jobs with the grace period. - Jobs carry (document_id, project_id, user_id) explicitly; the job's owner is bound into the user ContextVar around the model call so tracing/usage attribution never falls back to 'default'. - Text source reuses read_text_serving_path (convertible-first rule, content validation); conversion has no deadline (await_drained), the model call has a caller-side wait limit — the worker slot is retained until the provider call actually finishes and late results are discarded. - Output contract: strip leading <think> reasoning -> unwrap fence/ quotes -> collapse to one line -> empty is NULL -> UTF-8 byte cap on a character boundary; length-capped responses (provider-normalized detectors) are failures. - set_summary pins updated_at (literal self-assignment) so a landing summary never reorders the shelf; owner-predicated UPDATE is a no-op on rows purged mid-generation. - Metrics: eligible/success/failed/skipped/queue_dropped/ uncapped_accepted. Schema/config: - project_documents.summary (migration 0036, nullable, no backfill; forward-compat floor unchanged). - Six projects.summary_* keys, all registered in STARTUP_ONLY_FIELDS (captured once at lifespan startup); config_version 57 -> 58. - run_oneshot_llm gains an optional model_overrides passthrough (passed only when set, existing callers unchanged) and a detailed variant exposing the raw response message for termination-signal detection. Docs: API.md response field, ARCHITECTURE.md pipeline paragraph, migrations AGENTS.md chain head, root AGENTS.md records the scoped all-core exception with the lift-out path.
72 lines
3.4 KiB
Python
72 lines
3.4 KiB
Python
"""ORM models for projects.
|
|
|
|
``ProjectRow``: one row per user-owned project. ``id`` (uuid4 hex) is the only
|
|
external identity; ``name``/``presentation`` are display attributes — renaming
|
|
touches exactly this row and never affects membership (threads reference
|
|
``threads_meta.project_id``). ``instructions`` is user-authored project
|
|
context; Phase 1 stores and PATCHes it, Phase 2 injects it. There is
|
|
deliberately no ``memory_mode``/sharing/agent-config column: no consumer
|
|
exists in Phase 1/2 (RFC v2 §3, §4.1).
|
|
|
|
``ProjectDocumentRow``: one row per project shelf document. ``stored_relpath``
|
|
is a server-generated content address relative to ``users/{user_id}/projects/``
|
|
that embeds both ``sha256`` and the row's own ``id``, so rows never share
|
|
files and a re-upload after trash always lands in a fresh namespace. Text
|
|
detection is a sampled read at serve time, and the shelf has no history by
|
|
design — hence no ``mime``/``is_text``/``version``/``deleted_by`` columns
|
|
(Phase-2 spec §6.1). ``source_*`` records promotion provenance (Slice C);
|
|
``trashed_at``/``trash_origin`` implement the recoverable trash tier: every
|
|
read query filters ``trashed_at IS NULL`` so trashed rows are invisible to the
|
|
index, tools, and listing APIs.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from datetime import UTC, datetime
|
|
|
|
from sqlalchemy import JSON, DateTime, Integer, String, Text
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
|
|
from deerflow.persistence.base import Base
|
|
|
|
|
|
class ProjectRow(Base):
|
|
__tablename__ = "projects"
|
|
|
|
id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
|
user_id: Mapped[str] = mapped_column(String(64), index=True)
|
|
name: Mapped[str] = mapped_column(String(128))
|
|
instructions: Mapped[str] = mapped_column(Text, default="")
|
|
presentation: Mapped[dict] = mapped_column(JSON, default=dict)
|
|
status: Mapped[str] = mapped_column(String(16), default="active", index=True)
|
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC))
|
|
updated_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True),
|
|
default=lambda: datetime.now(UTC),
|
|
onupdate=lambda: datetime.now(UTC),
|
|
)
|
|
|
|
|
|
class ProjectDocumentRow(Base):
|
|
__tablename__ = "project_documents"
|
|
|
|
id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
|
project_id: Mapped[str] = mapped_column(String(64), index=True)
|
|
user_id: Mapped[str] = mapped_column(String(64), index=True)
|
|
name: Mapped[str] = mapped_column(String(255))
|
|
stored_relpath: Mapped[str] = mapped_column(String)
|
|
sha256: Mapped[str] = mapped_column(String(64), index=True)
|
|
size_bytes: Mapped[int] = mapped_column(Integer)
|
|
summary: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
source_thread_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
source_kind: Mapped[str | None] = mapped_column(String(16), nullable=True)
|
|
source_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
trashed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
|
|
trash_origin: Mapped[dict | None] = mapped_column(JSON, nullable=True)
|
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC))
|
|
updated_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True),
|
|
default=lambda: datetime.now(UTC),
|
|
onupdate=lambda: datetime.now(UTC),
|
|
)
|