deer-flow/docker/nginx/nginx.conf
Ryker_Feng fa496c0c8d
feat(browser): add agentic browser control (#4187)
* feat(browser): add agentic browser control

* fix(frontend): format browser view changes

* fix(browser): keep browser optional and isolate sidecar layout

* fix(browser): address PR review security and IME findings

- Nginx: add a browser-stream WebSocket location before the generic
  /api/threads regex so Live upgrades instead of downgrading to HTTP
  (both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
  navigate, and tear down the browser session on thread deletion so a
  later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
  context-level route guard covering redirects, popups, iframes, and
  subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
  CJK candidate with Enter no longer submits the remote page form.

Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.

* fix(frontend): smooth streaming in long tool threads

* Revert "fix(frontend): smooth streaming in long tool threads"

This reverts commit f0462516eabe77f138d4027ea1c714fb226683cf.

* fix(browser): address review security and lifecycle findings

- Reject cross-origin WebSocket upgrades on the live browser stream
  (Origin allow-list reuse of CORS/same-origin helpers) to close a
  WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
  off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
  (browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
  and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
  can't permanently stall after the cumulative attempt cap.

* fix(frontend): reduce long tool thread render stalls

Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.

* fix(browser): keep live control responsive during continuous input

Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").

What:
- Frontend forwards one `click` per physical click instead of also emitting
  `down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
  start a rate-limited background refresh loop (leading frame + bounded cadence)
  that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
  inputs coalesce, and continuous input keeps refreshing before it stops.

Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.

* fix(browser): harden worker and session lifecycle

* fix(browser): address latest review feedback

* fix(frontend): preserve optimistic new-chat message

* test(e2e): preserve mocked message run ids

* fix(browser): address capability review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-21 11:46:33 +08:00

294 lines
12 KiB
Nginx Configuration File

events {
worker_connections 1024;
}
pid /tmp/nginx.pid;
http {
# Basic settings
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# Logging
access_log /dev/stdout;
error_log /dev/stderr;
# Docker internal DNS (for resolving k3s hostname)
resolver 127.0.0.11 valid=10s ipv6=off;
# Preserve an upstream proxy's X-Forwarded-Proto so the Gateway sees the real
# client scheme when DeerFlow's nginx runs BEHIND another TLS-terminating
# reverse proxy (e.g. Pangolin/Traefik, Cloudflare, Caddy). Without this the
# Gateway treats HTTPS browser traffic as HTTP and rejects the login POST with
# 403 "Cross-site auth request denied" (Origin scheme mismatch), and also drops
# the Secure flag / max-age on session cookies. Falls back to $scheme when nginx
# is itself the TLS edge (standalone `make dev` / Docker), so default
# deployments are unchanged.
map $http_x_forwarded_proto $forwarded_proto {
default $scheme;
"~*^https" https;
}
# Only mark frontend requests as connection upgrades when the browser
# actually requested one. Next.js dev HMR also uses long-lived HTTP
# streams, and forcing "Connection: upgrade" on those requests makes
# nginx treat the upstream response as an invalid upgrade handshake.
map $http_upgrade $connection_upgrade {
default upgrade;
'' '';
}
# ── Main server (path-based routing) ─────────────────────────────────
server {
listen 2026 default_server;
listen [::]:2026 default_server;
server_name _;
# Resolve Docker service names at request time to avoid stale upstream
# IPs when containers restart and receive new addresses.
set $gateway_upstream gateway:8001;
set $frontend_upstream frontend:3000;
# Default proxy settings for all locations (streaming/SSE support)
proxy_buffering off;
proxy_cache off;
# Keep the unified nginx endpoint same-origin by default. When split
# frontend/backend or port-forwarded deployments need browser CORS,
# configure the Gateway allowlist with GATEWAY_CORS_ORIGINS so CORS and
# CSRF origin checks stay aligned instead of approving every origin at
# the proxy layer.
# LangGraph-compatible API routes served by Gateway.
# Rewrites /api/langgraph/* to /api/* before proxying to Gateway.
location /api/langgraph/ {
rewrite ^/api/langgraph/(.*) /api/$1 break;
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
# Headers
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
proxy_set_header Connection '';
# SSE/Streaming support
proxy_set_header X-Accel-Buffering no;
# Long chat/text-prompt support (issue #3952): a sufficiently long
# pasted prompt otherwise exceeds nginx's default 1m
# client_max_body_size, or gets spooled to a temp file via
# proxy_request_buffering before reaching Gateway -- on a non-root
# local run that temp directory may not be writable, producing a
# raw nginx 500 instead of a graceful application error. Mirrors
# the uploads location's fix below (same mechanism, sized for text
# prompts rather than binary file uploads).
client_max_body_size 20M;
proxy_request_buffering off;
# Timeouts for long-running requests
proxy_connect_timeout 600s;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
# Chunked transfer encoding
chunked_transfer_encoding on;
}
# Custom API: Models endpoint
location /api/models {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# Custom API: Memory endpoint
location /api/memory {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# Custom API: MCP configuration endpoint
location /api/mcp {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# Custom API: Skills configuration endpoint
location /api/skills {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# Custom API: Agents endpoint
location /api/agents {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# Custom API: Uploads endpoint
location ~ ^/api/threads/[^/]+/uploads {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
# Large file upload support
client_max_body_size 100M;
proxy_request_buffering off;
# Disable response buffering to avoid permission errors
}
# Live browser stream is a WebSocket upgrade. It must be matched before
# the generic /api/threads regex below (which omits Upgrade/Connection
# forwarding and would downgrade it to plain HTTP).
location ~ ^/api/threads/[^/]+/browser/stream {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_cache_bypass $http_upgrade;
# Long-lived Live stream: keep the upgraded connection open.
proxy_connect_timeout 600s;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
}
# Custom API: Other endpoints under /api/threads
location ~ ^/api/threads {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# API Documentation: Swagger UI
location /docs {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# API Documentation: ReDoc
location /redoc {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# API Documentation: OpenAPI Schema
location /openapi.json {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# Health check endpoint (gateway)
location /health {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# ── Provisioner API (sandbox management) ────────────────────────
# Use a variable so nginx resolves provisioner at request time (not startup).
# This allows nginx to start even when provisioner container is not running.
location /api/sandboxes {
set $provisioner_upstream provisioner:8002;
proxy_pass http://$provisioner_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
}
# Catch-all for /api/ routes not covered above (e.g. /api/v1/auth/*).
# More specific prefix and regex locations above still take precedence.
location /api/ {
proxy_pass http://$gateway_upstream;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
# Disable buffering to avoid permission errors when nginx
# runs as a non-root user (e.g. local development).
}
# All other requests go to frontend
location / {
proxy_pass http://$frontend_upstream;
proxy_http_version 1.1;
# Headers
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $forwarded_proto;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_cache_bypass $http_upgrade;
# Timeouts
proxy_connect_timeout 600s;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
}
}
}