deer-flow/CONTRIBUTING.md

Contributing to DeerFlow

Thank you for your interest in contributing to DeerFlow! This guide will help you set up your development environment and understand our development workflow.

Development Environment Setup

We offer two development environments. Docker is recommended for the most consistent and hassle-free experience.

Option 1: Docker Development (Recommended)

Docker provides a consistent, isolated environment with all dependencies pre-configured. No need to install Node.js, Python, or nginx on your local machine.

Prerequisites

• Docker Desktop or Docker Engine
• Docker Compose v2.24 or newer (check with docker compose version). Both Compose files mark their env_file entries optional using the long-form path/required syntax; older clients reject it with services.gateway.env_file.0 must be a string. make docker-start verifies the version and tells you to upgrade — direct docker compose callers get that raw message instead.
• pnpm (for caching optimization)

Setup Steps

1. Configure the application:

   # Copy example configuration
   cp config.example.yaml config.yaml
   
   # Set your API keys
   export OPENAI_API_KEY="your-key-here"
   # or edit config.yaml directly
   

2. Initialize Docker environment (first time only):

   make docker-init
   

   This pulls the sandbox image that the container sandbox modes run, so the first sandbox container does not wait on the download. It is a no-op in local sandbox mode (the default), which needs no image.

3. Start development services:

   make docker-start
   

   make docker-start reads config.yaml and starts provisioner only for provisioner/Kubernetes sandbox mode.

   Prefer this wrapper over invoking Compose yourself: it checks your Compose version, creates the missing .env files, and exports DEER_FLOW_ROOT.

   If you do run Compose directly, run it from the repository root and set DEER_FLOW_ROOT to the absolute path of your checkout. The default command starts the local/AIO development stack without Kubernetes; add --profile provisioner only when you need provisioner/Kubernetes sandbox mode. Compose interpolates DEER_FLOW_ROOT into host-side paths (DEER_FLOW_HOST_BASE_DIR, THREADS_HOST_PATH) that the AIO and provisioner sandbox modes bind-mount; leaving it unset renders them as /backend/.deer-flow, so those mounts silently miss your checkout instead of failing:

   # macOS / Linux
   DEER_FLOW_ROOT="$PWD" docker compose -f docker/docker-compose-dev.yaml up --build
   

   # Windows PowerShell
   $env:DEER_FLOW_ROOT = (Get-Location).Path
   docker compose -f docker/docker-compose-dev.yaml up --build
   

   Do not reuse that -f path from inside docker/ — it resolves to docker/docker/docker-compose-dev.yaml and fails with a file-not-found error.

   All services will start with hot-reload enabled:

   • Frontend changes are automatically reloaded
   • Backend changes trigger automatic restart
   • Gateway-hosted LangGraph-compatible runtime supports hot-reload

4. Access the application:

   • Web Interface: http://localhost:2026
   • API Gateway: http://localhost:2026/api/*
   • LangGraph-compatible API: http://localhost:2026/api/langgraph/*

Docker Commands

# Pull the sandbox image used by the container sandbox modes
make docker-init
# Start Docker services (mode-aware, localhost:2026)
make docker-start
# Stop Docker development services
make docker-stop
# View Docker development logs
make docker-logs
# View Docker frontend logs
make docker-logs-frontend
# View Docker gateway logs
make docker-logs-gateway

If Docker builds are slow in your network, you can override the default package registries before running make docker-start:

export UV_INDEX_URL=https://pypi.org/simple
export NPM_REGISTRY=https://registry.npmjs.org

Keep UV_INDEX_URL at https://pypi.org/simple: the gateway image installs with uv sync --locked, and uv.lock is locked to that index, so a mirror makes the build fail with The lockfile at `uv.lock` needs to be updated, but `--locked` was provided.

Recommended host resources

Use these as practical starting points for development and review environments:

Scenario	Starting point	Recommended	Notes
make dev on one machine	4 vCPU, 8 GB RAM	8 vCPU, 16 GB RAM	Best when DeerFlow uses hosted model APIs.
make docker-start review environment	4 vCPU, 8 GB RAM	8 vCPU, 16 GB RAM	Docker image builds and sandbox containers need extra headroom.
Shared Linux test server	8 vCPU, 16 GB RAM	16 vCPU, 32 GB RAM	Prefer this for heavier multi-agent runs or multiple reviewers.

2 vCPU / 4 GB environments often fail to start reliably or become unresponsive under normal DeerFlow workloads.

Linux: Docker daemon permission denied

If make docker-init, make docker-start, or make docker-stop fails on Linux with an error like below, your current user likely does not have permission to access the Docker daemon socket:

unable to get image 'deer-flow-gateway': permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock

Recommended fix: add your current user to the docker group so Docker commands work without sudo.

1. Confirm the docker group exists:

   getent group docker
   

2. Add your current user to the docker group:

   sudo usermod -aG docker $USER
   

3. Apply the new group membership. The most reliable option is to log out completely and then log back in. If you want to refresh the current shell session instead, run:

   newgrp docker
   

4. Verify Docker access:

   docker ps
   

5. Retry the DeerFlow command:

   make docker-stop
   make docker-start
   

If docker ps still reports a permission error after usermod, fully log out and log back in before retrying.

Docker Architecture

Host Machine
  ↓
Docker Compose (deer-flow-dev)
  ├→ nginx (port 2026) ← Reverse proxy
  ├→ web (port 3000) ← Frontend with hot-reload
  ├→ gateway (port 8001) ← Gateway API + LangGraph-compatible runtime with hot-reload
  └→ provisioner (optional, port 8002) ← Started only in provisioner/K8s sandbox mode

Benefits of Docker Development:

• ✅ Consistent environment across different machines
• ✅ No need to install Node.js, Python, or nginx locally
• ✅ Isolated dependencies and services
• ✅ Easy cleanup and reset
• ✅ Hot-reload for all services
• ✅ Production-like environment

Option 2: Local Development

If you prefer to run services directly on your machine:

Prerequisites

Check that you have all required tools installed:

make check

Required tools:

• Node.js 22+
• pnpm
• uv (Python package manager)
• nginx

Setup Steps

1. Configure the application (same as Docker setup above)

2. Install dependencies (this also sets up pre-commit hooks):

   make install
   

3. Run development server (starts all services with nginx):

   make dev
   

4. Access the application:

   • Web Interface: http://localhost:2026
   • All API requests are automatically proxied through nginx

Manual Service Control

If you need to start services individually:

1. Start backend service:

   # Terminal 1: Start Gateway API + embedded agent runtime (port 8001)
   cd backend
   make dev
   
   # Terminal 2: Start Frontend (port 3000)
   cd frontend
   pnpm dev
   

2. Start nginx (run from the repo root):

   make nginx
   

   This runs scripts/nginx.sh, which launches nginx in the foreground the same way scripts/serve.sh (used by make dev / make start) does: it pre-creates the logs/ and temp/ directories and uses the local dev config at docker/nginx/nginx.local.conf.

3. Access the application:

   • Web Interface: http://localhost:2026

Nginx Configuration

The nginx configuration provides:

• Unified entry point on port 2026
• Rewrites /api/langgraph/* to Gateway's LangGraph-compatible API (8001)
• Routes other /api/* endpoints to Gateway API (8001)
• Routes non-API requests to Frontend (3000)
• Same-origin API routing; split-origin or port-forwarded browser clients should use the Gateway GATEWAY_CORS_ORIGINS allowlist
• SSE/streaming support for real-time agent responses
• Optimized timeouts for long-running operations

Project Structure

deer-flow/
├── config.example.yaml      # Configuration template
├── extensions_config.example.json  # MCP and Skills configuration template
├── Makefile                 # Build and development commands
├── scripts/
│   └── docker.sh           # Docker management script
├── docker/
│   ├── docker-compose-dev.yaml  # Docker Compose configuration
│   └── nginx/
│       ├── nginx.conf      # Nginx config for Docker
│       └── nginx.local.conf # Nginx config for local dev
├── backend/                 # Backend application
│   ├── packages/harness/   # deerflow-harness package (import: deerflow.*)
│   │   └── deerflow/       # Agents, tools, sandbox, MCP, skills, config
│   ├── app/                # FastAPI Gateway + IM channels (import: app.*)
│   │   ├── gateway/        # Gateway API and LangGraph-compatible runtime (port 8001)
│   │   └── channels/       # IM channel integrations
│   ├── docs/               # Backend documentation
│   └── Makefile            # Backend commands
├── frontend/               # Frontend application
│   └── Makefile            # Frontend commands
└── skills/                 # Agent skills
    ├── public/             # Public skills
    └── custom/             # Custom skills

Architecture

Browser
  ↓
Nginx (port 2026) ← Unified entry point
  ├→ Frontend (port 3000) ← / (non-API requests)
  └→ Gateway API (port 8001) ← /api/* and /api/langgraph/* (LangGraph-compatible agent interactions)

Development Workflow

1. Create a feature branch:

   git checkout -b feature/your-feature-name
   

2. Make your changes with hot-reload enabled

3. Format and lint your code (CI will reject unformatted code):

   # Backend
   cd backend
   make format   # ruff check --fix + ruff format
   
   # Frontend
   cd frontend
   pnpm format:write   # Prettier
   

4. Test your changes thoroughly

5. Commit your changes:

   git add .
   git commit -m "feat: description of your changes"
   

6. Push and create a Pull Request:

   git push origin feature/your-feature-name
   

AI assistance disclosure

DeerFlow is an AI project and we welcome AI-assisted contributions. To help reviewers calibrate how closely to read a change, every pull request must complete the "AI assistance" section of the PR template:

• which tool(s) you used (or none),
• how you used them, and
• a confirmation that a human has read, understands, and takes responsibility for the change.

Please don't delete the section. PRs that ignore it may be asked to fill it in before review.

Testing

# Public skill scripts (from the repository root)
python -m pip install -r tests/skills/requirements.txt
python -m pytest tests/skills -q

# Default backend tests (four parallel shards; excludes live and blocking-I/O tests)
cd backend
make test

# Run the same shards sequentially
make test TEST_JOBS=1

# Strict blocking-I/O tests
make test-blocking-io

# Live DeerFlowClient integration tests (explicit opt-in)
# Requires a valid root config.yaml and API credentials.
make test-live

# Frontend unit tests
cd frontend
make test

# Frontend E2E tests (requires Chromium; builds and auto-starts the Next.js production server)
cd frontend
make test-e2e

If a host-configured uv package mirror blocks dependencies while running the config upgrade tests, run make test-config-version from the repository root. That command uses public PyPI for this suite and its subprocesses without changing the host's uv settings or the normal config upgrade command.

make test-live calls real external APIs and may incur API costs or create local sandboxes, artifacts, and files. It is never run by the default backend test command or CI. Direct pytest invocations of tests/test_client_live.py must also set DEER_FLOW_RUN_LIVE_TESTS=1.

PR Regression Checks

Every pull request triggers the following CI workflows:

• Backend unit tests — .github/workflows/backend-unit-tests.yml
• Frontend unit tests — .github/workflows/frontend-unit-tests.yml
• Frontend E2E tests — .github/workflows/e2e-tests.yml (triggered only when frontend/ files change)

Test and diagnostics notes

Backend make test runs four duration-balanced shards concurrently and reports failure if any shard fails. Use make test TEST_JOBS=1 to run the shards sequentially. It excludes live external-API and blocking-I/O coverage. Run cd backend && make test-blocking-io for strict blocking-I/O checks. Maintainers can run the real DeerFlowClient suite with cd backend && make test-live. This command requires a valid root config.yaml and API credentials. It can incur API costs and create local sandboxes, artifacts, or files. Direct pytest runs additionally require DEER_FLOW_RUN_LIVE_TESTS=1.

Regression coverage includes Docker sandbox mode detection and provisioner kubeconfig-path handling tests in backend/tests/. Backend blocking-IO diagnostics are available from the repository root with make detect-blocking-io: it statically scans backend business code for blocking IO that may run on the backend event loop, prints a concise summary, and writes complete JSON findings to .deer-flow/blocking-io-findings.json. The JSON includes compact review records with priority, location, blocking_call, event_loop_exposure, reason, and code. Gateway artifact serving now forces active web content types (text/html and XML documents such as .xml, .xhtml, and .svg) to download as attachments instead of inline rendering, reducing XSS risk for generated artifacts.

Frontend route asset budgets can be checked with cd frontend && pnpm perf:check. The command measures /login from a normal production build, then performs a production static-demo build for the fixture-backed workspace routes. It measures the unique JavaScript and CSS referenced by representative routes and writes the detailed result to .next/performance-results.json.

Code Style

• Backend (Python): We use ruff for linting and formatting. Run make format before committing.
• Frontend (TypeScript): We use ESLint and Prettier. Run pnpm format:write before committing.
• CI enforces formatting — PRs with unformatted code will fail the lint check.

Documentation

• Configuration Guide - Setup and configuration
• Architecture Overview - Technical architecture
• MCP Setup Guide - Model Context Protocol configuration

Troubleshooting Bundle

For setup, configuration, sandbox, or runtime issues, generate a redacted support summary before filing:

make support-bundle

The command prints reporter next steps, writes a *-issue-summary.md file that you can paste into the issue, writes a *-issue-draft.md file for AI-assisted issue filing, and writes an optional evidence zip under .deer-flow/support-bundles/. The zip includes toolchain versions, sanitized config.yaml and extensions_config.json summaries, enabled tool/skill/MCP structure, git metadata, and redacted make doctor output.

When filing the issue, paste the generated *-issue-summary.md into the issue body. If an AI assistant files the issue, start from *-issue-draft.md and replace every REQUIRED placeholder before filing; the draft intentionally does not invent reproduction steps, expected behavior, or a problem summary. Attach the zip only if a maintainer asks for the evidence bundle, or if the summary alone is not enough to diagnose the issue. Maintainers and AI-assisted triage should start with triage.json, which contains stable signals such as config_missing, node_version_too_old, doctor_failed, and suggested next steps. The other JSON files are evidence for follow-up inspection.

It intentionally does not include .env, raw conversation messages, or the contents of files in thread workspaces/uploads/outputs. If you need to include a thread, run cd backend && uv run python ../scripts/support_bundle.py --thread-id <thread-id> --include-doctor; this adds file manifests only. Please still review the generated zip before attaching it to a public issue.

Need Help?

• Check existing Issues
• Read the Documentation
• Ask questions in Discussions

License

By contributing to DeerFlow, you agree that your contributions will be licensed under the MIT License.