mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
Opt-in per-document summary column rendered into the injected <documents> index, list_project_documents JSON, and the documents API. Pipeline (projects/summaries.py, behind the narrow enqueue_summary interface): - The two shelf write routes enqueue a job on created=True; dedup hits never regenerate. Off by default (projects.summaries_enabled). - Bounded worker pool (summary_concurrency + bounded summary_queue_size; a full queue drops the job) fed by FIFO; shutdown drains queued jobs, delivers one sentinel per worker via blocking put (queue smaller than the worker pool cannot strand idle workers), and bounds only genuinely in-flight jobs with the grace period. - Jobs carry (document_id, project_id, user_id) explicitly; the job's owner is bound into the user ContextVar around the model call so tracing/usage attribution never falls back to 'default'. - Text source reuses read_text_serving_path (convertible-first rule, content validation); conversion has no deadline (await_drained), the model call has a caller-side wait limit — the worker slot is retained until the provider call actually finishes and late results are discarded. - Output contract: strip leading <think> reasoning -> unwrap fence/ quotes -> collapse to one line -> empty is NULL -> UTF-8 byte cap on a character boundary; length-capped responses (provider-normalized detectors) are failures. - set_summary pins updated_at (literal self-assignment) so a landing summary never reorders the shelf; owner-predicated UPDATE is a no-op on rows purged mid-generation. - Metrics: eligible/success/failed/skipped/queue_dropped/ uncapped_accepted. Schema/config: - project_documents.summary (migration 0036, nullable, no backfill; forward-compat floor unchanged). - Six projects.summary_* keys, all registered in STARTUP_ONLY_FIELDS (captured once at lifespan startup); config_version 57 -> 58. - run_oneshot_llm gains an optional model_overrides passthrough (passed only when set, existing callers unchanged) and a detailed variant exposing the raw response message for termination-signal detection. Docs: API.md response field, ARCHITECTURE.md pipeline paragraph, migrations AGENTS.md chain head, root AGENTS.md records the scoped all-core exception with the lift-out path.
DeerFlow Frontend
Like the original DeerFlow 1.0, we would love to give the community a minimalistic and easy-to-use web interface with a more modern and flexible architecture.
Tech Stack
- Framework: Next.js 16 with App Router
- UI: React 19, Tailwind CSS 4, Shadcn UI, MagicUI and React Bits
- AI Integration: LangGraph SDK and Vercel AI Elements
Quick Start
Prerequisites
- Node.js 22+
- pnpm 10.26.2+
Installation
# Install dependencies
pnpm install
# Copy environment variables
cp .env.example .env
# Edit .env with your configuration
Development
# Start development server
pnpm dev
# The app will be available at http://localhost:3000
Build & Test
# Type check
pnpm typecheck
# Check formatting
pnpm format
# Apply formatting
pnpm format:write
# Lint
pnpm lint
# Run unit tests
pnpm test
# One-time setup: install Playwright Chromium browser
pnpm exec playwright install chromium
# Run E2E tests (builds and starts production server automatically)
pnpm test:e2e
# Build for production
pnpm build
# Start production server
pnpm start
Site Map
├── / # Landing page
├── /chats # Chat list
├── /chats/new # New chat page
└── /chats/[thread_id] # A specific chat page
Configuration
Environment Variables
Key environment variables (see .env.example for full list):
# Backend API URL (optional, uses local Next.js/nginx proxy by default)
NEXT_PUBLIC_BACKEND_BASE_URL="http://localhost:8001"
# LangGraph-compatible API URL (optional, uses local Next.js/nginx proxy by default)
NEXT_PUBLIC_LANGGRAPH_BASE_URL="http://localhost:8001/api"
Project Structure
tests/
├── e2e/ # E2E tests (Playwright, Chromium, mocked backend)
└── unit/ # Unit tests (mirrors src/ layout)
src/
├── app/ # Next.js App Router pages
│ ├── api/ # API routes
│ ├── showcase/ # Allowlisted public read-only demos
│ ├── workspace/ # Main workspace pages
│ └── mock/ # Mock/demo pages
├── components/ # React components
│ ├── ui/ # Reusable UI components
│ ├── workspace/ # Workspace-specific components
│ ├── landing/ # Landing page components
│ └── ai-elements/ # AI-related UI elements
├── core/ # Core business logic
│ ├── api/ # API client & data fetching
│ ├── artifacts/ # Artifact management
│ ├── config/ # App configuration
│ ├── i18n/ # Internationalization
│ ├── mcp/ # MCP integration
│ ├── messages/ # Message handling
│ ├── models/ # Data models & types
│ ├── settings/ # User settings
│ ├── skills/ # Skills system
│ ├── threads/ # Thread management
│ ├── todos/ # Todo system
│ └── utils/ # Utility functions
├── hooks/ # Custom React hooks
├── lib/ # Shared libraries & utilities
├── server/ # Server-side code
│ └── better-auth/ # Authentication setup and session helpers
└── styles/ # Global styles
Scripts
| Command | Description |
|---|---|
pnpm dev |
Start development server with Webpack |
pnpm build |
Build for production |
pnpm start |
Start production server |
pnpm test |
Run unit tests with Rstest |
pnpm test:e2e |
Run E2E tests with Playwright |
pnpm format |
Check formatting with Prettier |
pnpm format:write |
Apply formatting with Prettier |
pnpm lint |
Run ESLint |
pnpm lint:fix |
Fix ESLint issues |
pnpm typecheck |
Run TypeScript type checking |
pnpm check |
Run both lint and typecheck |
Development Notes
- Uses pnpm workspaces (see
packageManagerin package.json) - Webpack is the default development bundler until the upstream Turbopack PostCSS worker leak is fixed in a stable Next.js release (#5132). Set
DEER_FLOW_DEV_BUNDLER=turboto opt in to Turbopack for local diagnosis, orDEER_FLOW_DEV_BUNDLER=webpackto select Webpack explicitly. Reconsider the default after the stable fix is verified on macOS arm64 and Linux. - Environment validation can be skipped with
SKIP_ENV_VALIDATION=1(useful for Docker) - Backend API URLs are optional; nginx proxy is used by default in development
License
MIT License. See LICENSE for details.