mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
33 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Hyeonsang Cho
|
f8b419ab38
|
fix(scripts): bind local make dev and make start to loopback (#6587)
* fix(scripts): bind local make dev and make start to loopback serve.sh and backend/Makefile started the Gateway with --host 0.0.0.0, nginx.local.conf listened on every interface, and Next.js kept its all-interfaces default outside Windows, so a local stack on a LAN or VPN exposed ports 2026, 8001 and 3000, including /setup before the first admin existed. The Docker stack and the README already document a loopback-only default. Bind the Gateway and frontend to 127.0.0.1 and keep nginx on loopback unless BIND_HOST is set, the variable the Docker stack already honors. nginx-local-conf.sh renders a copy listening on BIND_HOST, and serve.sh resolves it before stopping anything so an invalid value cannot tear down a running stack. * docs(changelog): reference #6587 * docs(changelog): separate the loopback entry from the next bullet --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Yufeng He
|
37598c45ee
|
fix(nginx): canonicalize the loopback origin onto localhost (#6158)
* fix(nginx): canonicalize the loopback origin onto localhost localhost and 127.0.0.1 (or [::1]) are the same loopback listener but separate cookie jars to a browser, so an admin who sets up on one spelling lands on /login when they later open the other, with no hint why (#6156). Cookies cannot bridge the gap: no Domain attribute spans both hosts. All three nginx configs now map the numeric spellings to "localhost", the name every banner and doc already prints, and the main server answers a 301 to the same URL on the canonical host. The published port survives because the map keys on $http_host and reattaches the captured port; Docker's arbitrary host-port mapping is unaffected. Safe methods only: GET/HEAD redirect, while API POSTs pass through untouched, since a redirected POST strands clients that do not re-send bodies. Non-loopback Hosts (LAN names, ingress domains) never match, and the canonical target itself is not a match, so there is no redirect loop. The new suite parses both maps out of each config and evaluates the full method x host matrix against nginx semantics, including the suffix-spoof and redirect-loop cases, and pins the maps identical across the Docker, local-dev, and Helm configs so they cannot drift. Fixes #6156 Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com> * docs: keep the loopback note in the nginx configs The root AGENTS.md paragraph pushed the middlewares and sandbox instruction chains over the 96 KiB hard limit. The nginx configs already carry the full explanation next to the maps, so drop the duplicated paragraph instead of shrinking the budgets. * fix(nginx): let WebSocket upgrades bypass the loopback canonical redirect willem-bd caught this running the stack locally: the canonicalization map keyed on method:host alone, so a GET with an Upgrade header got the same 301 as a plain GET. WebSocket clients do not follow redirects, so every WS handshake against a 127.0.0.1/[::1] spelling died at the redirect. Key the origin map on method:upgrade:host instead. Handshakes carry a non-empty Upgrade value, fall off the GET::/HEAD:: patterns, and proxy through untouched. Plain GET/HEAD behavior is unchanged. --------- Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com> |
||
|
Hyeonsang Cho
|
a253faefdf
|
fix(docker): give project-document uploads their own nginx location so they are not capped at 1 MB (#5934)
* fix(docker): give project-document uploads their own nginx location
POST /api/projects/{id}/documents is a multipart upload that Gateway
accepts up to uploads.max_file_size (50 MiB by default), but no nginx
location matched it. It fell through to the /api/ catch-all, which has no
client_max_body_size, so nginx's 1m default answered a 2 MB PDF with a
bare 413 before Gateway saw the request. Reproduced on nginx:alpine with
the shipped config; the same body to /api/threads/{id}/uploads passed.
Add a `~ ^/api/projects/[^/]+/documents` location to the Docker, make dev
and Helm configs with the thread-uploads settings (100M, streamed body)
and the 600s read timeout the catch-all already granted these routes. The
catch-all keeps nginx's defaults. The shared location-block helpers move
to tests/support/nginx_conf.py so the new test and the existing body-size
test use one implementation.
* docs(changelog): reference #5934 in the project-documents nginx entries
|
||
|
Hyeonsang Cho
|
9f79ddf9b6
|
fix(nginx): allow model-bound /api/ and /api/skills requests past 60 seconds (#5524)
* fix(nginx): allow model-bound /api/ and /api/skills requests past 60 seconds Two locations were left on nginx's 60s default while the routes behind them wait on Gateway. The /api/ catch-all carries the stateless POST /api/runs/wait, which blocks on wait_for_run_completion and cancels its run when the client disconnects, so a caller waiting on a longer run got a 504 and lost the run; it also carries POST /api/input-polish, which waits for a one-shot model call. /api/skills carries POST /api/skills/install, which runs one LLM security scan per file in the archive, and the custom-skill edit and rollback routes, which run one more each. None of them sets an application-level timeout, and only the sibling /api/skills/install/upload endpoint had been given the longer timeout, so the same archive failed at 60s depending on which endpoint installed it. Allow 600s on both locations, matching /api/langgraph/ and /api/threads, in all three copies of the nginx config. Each directive is pinned by its own test that parses the active directive per config. * docs(changelog): link the nginx /api/ and /api/skills timeout entry to #5524 |
||
|
Hyeonsang Cho
|
3cfc9c58fd
|
fix(nginx): allow model-bound /api/threads requests past 60 seconds (#5505)
* fix(nginx): allow model-bound /api/threads requests past 60 seconds The browser calls /api/threads/* directly rather than through /api/langgraph/, and the generic `location ~ ^/api/threads` block set no proxy_read_timeout, so nginx's 60s default applied. /compact and /suggestions hold the response open for a whole model call, and /runs/wait for a whole run. Past 60s nginx returned 504 mid-work: the compaction still committed behind the failed request, and /runs/wait cancelled its run on the disconnect (on_disconnect defaults to cancel). Allow 600s on that location, matching /api/langgraph/, in all three copies of the nginx config: Docker, local dev, and the Helm ConfigMap. The regression test parses the active directive per config, so a missing, commented-out, lowered, or misplaced timeout fails. * docs(changelog): link the nginx /api/threads timeout entry to #5505 --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
PeaceMaker-best
|
72ba661b84
|
feat(skills): install local skill archives (#5039)
* feat(skills): install local skill archives Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> * fix(skills): enforce upload limits before parsing * fix(nginx): scope skill upload limit to upload route * fix(nginx): harden skill upload proxy handling * fix(skills): improve archive upload feedback * fix(skills): address upload review polish --------- Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> |
||
|
DanielWalnut
|
459dd78707
|
perf(frontend): bound delivery, bundles, and long-running UI work (#4622)
* docs: design frontend performance remediation * docs: plan frontend performance remediation * test(frontend): add route asset performance budgets * perf(nginx): compress textual responses safely * perf(frontend): lazy load case study media * perf(frontend): bound static demo file tracing * perf(frontend): restore static locale boundaries * perf(frontend): defer closed workspace panels * perf(frontend): split editors and deduplicate highlighting * perf(frontend): index incremental message derivation * perf(frontend): stabilize paged history cache policy * perf(frontend): bound streaming markdown renders * perf(frontend): virtualize message history * perf(frontend): bound and virtualize chat lists * perf(frontend): suspend inactive decorative animation * perf(browser): stream latest frames as binary * perf(artifacts): stream bounded text previews * docs: finalize performance runtime boundaries * style(backend): apply test formatting * fix(frontend): keep translation functions client-side * perf(frontend): defer decorative animation bundles * test(frontend): lock optimized route budgets * fix: harden frontend performance boundaries * test(frontend): update i18n provider fixture * fix(frontend): preserve sidebar pagination position * style(backend): format artifact range test |
||
|
Ryker_Feng
|
fa496c0c8d
|
feat(browser): add agentic browser control (#4187)
* feat(browser): add agentic browser control
* fix(frontend): format browser view changes
* fix(browser): keep browser optional and isolate sidecar layout
* fix(browser): address PR review security and IME findings
- Nginx: add a browser-stream WebSocket location before the generic
/api/threads regex so Live upgrades instead of downgrading to HTTP
(both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
navigate, and tear down the browser session on thread deletion so a
later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
context-level route guard covering redirects, popups, iframes, and
subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
CJK candidate with Enter no longer submits the remote page form.
Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.
* fix(frontend): smooth streaming in long tool threads
* Revert "fix(frontend): smooth streaming in long tool threads"
This reverts commit f0462516eabe77f138d4027ea1c714fb226683cf.
* fix(browser): address review security and lifecycle findings
- Reject cross-origin WebSocket upgrades on the live browser stream
(Origin allow-list reuse of CORS/same-origin helpers) to close a
WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
(browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
can't permanently stall after the cumulative attempt cap.
* fix(frontend): reduce long tool thread render stalls
Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.
* fix(browser): keep live control responsive during continuous input
Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").
What:
- Frontend forwards one `click` per physical click instead of also emitting
`down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
start a rate-limited background refresh loop (leading frame + bounded cadence)
that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
inputs coalesce, and continuous input keeps refreshing before it stops.
Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.
* fix(browser): harden worker and session lifecycle
* fix(browser): address latest review feedback
* fix(frontend): preserve optimistic new-chat message
* test(e2e): preserve mocked message run ids
* fix(browser): address capability review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
Zhengcy05
|
5994fdf38c
|
fix: avoid forcing frontend nginx upgrades (#4250)
* fix: avoid forcing frontend nginx upgrades * fix: scope nginx upgrade assertion to frontend location |
||
|
Daoyuan Li
|
7757e38b7f
|
fix(nginx): allow long chat prompts through /api/langgraph/ without a raw 500 (#4277)
nginx's default client_max_body_size (1m) and proxy_request_buffering (on) were never overridden for the chat/LangGraph route, only for the upload route. A long pasted prompt either exceeded the default size limit (413) or got spooled to a temp file before reaching Gateway, which can fail with a raw nginx 500 on a non-root local run if that temp directory isn't writable -- reproduced independently (real nginx 1.28.3, byte-identical error page and matching error-log Permission denied line) while confirming fancyboi999's diagnosis in issue #3952. Mirrors the upload route's existing client_max_body_size / proxy_request_ buffering fix onto /api/langgraph/ in all three places this nginx config is maintained (Docker prod, local make dev, and the Kubernetes/Helm ConfigMap), sized smaller (20M) since this route only ever carries JSON chat text, never binary file uploads. |
||
|
ajayr
|
71c5c4a072
|
fix(nginx): preserve upstream X-Forwarded-Proto when behind another TLS proxy (#3793)
When DeerFlow's nginx runs behind another TLS-terminating reverse proxy (Pangolin/Traefik, Cloudflare, Caddy), every location block overwrote the already-correct X-Forwarded-Proto with $scheme (= http on the private hop). The Gateway then treated HTTPS browser traffic as HTTP: the auth-origin check rejected the login POST with 403 "Cross-site auth request denied", and session cookies lost the Secure flag and max-age. Preserve an upstream X-Forwarded-Proto via a map that falls back to $scheme when nginx is itself the TLS edge, so standalone `make dev` / Docker is unchanged. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
zhernrong92
|
e97d93503d
|
fix: make local-dev (make dev) work on non-root / NFS hosts (#3590)
* fix(scripts): avoid lsof hang during make dev cleanup on NFS
`_is_deerflow_pid` and `_report_reclaimed_ports` call `lsof -p <pid>` to
enumerate a process's open files. On hosts whose working tree or home is
on a network filesystem (NFS/autofs), `lsof -p` blocks indefinitely on the
kernel stat calls, so `make dev` / `make stop` hang forever at
"Stopping all services...".
Add `-b` (avoid kernel blocking functions) and `-w` (suppress the
resulting warnings) to both calls. The network-only `lsof -nP -iTCP`
probes are unaffected and already returned quickly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(nginx): set global error_log so local-dev nginx starts as non-root
nginx.local.conf only declared `error_log` inside the `http {}` block.
nginx opens its compiled-in default error log (on Debian/Ubuntu builds,
the absolute /var/log/nginx/error.log) at startup, before it reaches the
http-block directive. When `make dev` launches nginx as a non-root user
that path is not writable, so startup fails with:
[emerg] open() "/var/log/nginx/error.log" failed (13: Permission denied)
Declare a global (main-context) `error_log logs/nginx-error.log warn;`.
Combined with the existing `-p $REPO_ROOT`, logging resolves to the
repo-local logs/ directory and nginx starts without elevated privileges.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Yi Tang
|
48e038f752
|
feat(channels): enhance Discord with mention-only mode, thread routing, and typing indicators (#2842)
* feat(channels): enhance Discord with mention-only mode, thread routing, and typing indicators
Add mention_only config to only respond when bot is mentioned, with
allowed_channels override. Add thread_mode for Hermes-style auto-thread
creation. Add periodic typing indicators while bot is processing.
* fix(discord): include allowed_channels in mention_only skip condition (line 274)
* docs: fix Discord config example to match boolean thread_mode implementation
* style: format with ruff
* fix(discord): apply Copilot review fixes and resolve lint errors
- Remove unused Optional import
- Fix thread_ts type hints to str | None
- Fix has_mention logic for None values
- Implement thread_mode fallback to channel replies on thread creation failure
- Fix thread_mode docstring alignment
- Fix allowed_channels comment formatting in config.example.yaml
* fix(discord): reset context for orphaned threads in mention_only mode
When a message arrives in a thread not tracked by _active_threads,
clear thread_id and typing_target so the message falls through to
the standard channel handling pipeline, which creates a fresh thread
instead of incorrectly routing to the stale thread.
* fix(discord): create new thread on @ when channel has existing tracked thread
When mention_only is enabled and a user @-s the bot in a channel
that already has a tracked thread, create a new thread instead of
incorrectly routing to the old one.
* fix(discord): allow no-@ thread replies while skipping no-@ channel messages
The skip block for no-@ messages was too aggressive — it blocked
continuation replies within tracked threads AND incorrectly routed
no-@ channel messages to the existing thread.
Now:
- Thread message, no @ → routed to existing tracked thread
- Channel message, no @ → skipped
- Channel message, with @ → creates new thread
* feat(discord): add checkmark reaction to acknowledge received messages
* Move discord.py to optional dependency and auto-detect from config.yaml
- Add discord extra to [project.optional-dependencies] in pyproject.toml
- Update detect_uv_extras.py to map channels.discord.enabled: true -> --extra discord
- Set UV_EXTRAS=discord in docker-compose-dev.yaml gateway env
* fix(discord): persist thread-channel mappings to store for recovery after restart
Discord's _active_threads dict was purely in-memory, so all channel-to-thread
mappings were lost on server restart. This fix bridges ChannelStore into
DiscordChannel:
- Save thread mappings to store.json after every thread creation
- Restore active threads from store on DiscordChannel startup
- Pass channel_store to all channels via service.py config injection
Store keys follow the pattern: discord:<channel_id>:<thread_id>
* fix(discord): address Copilot review — fix types, typing targets, cross-thread safety, and config comments
* fix(tests): add multitask_strategy param to mock for clarification follow-up test
* fix(tests): explicitly set model_name=None for title middleware test isolation
* fix(discord): use trigger_typing() instead of typing() for typing indicators
discord.py 2.x TextChannel.typing() and Thread.typing() are async context
managers, not one-shot coroutines. Use trigger_typing() for periodic
typing indicator pings.
* fix(discord): cancel typing tasks on channel shutdown
Prevents 'Task was destroyed but it is pending' warnings when the
Discord client stops while typing indicator loops are still running.
* fix(scripts): detect nested YAML config for discord extra
section_value() only matched top-level YAML sections. Added
nested_section_value() that handles two-level nesting (e.g.,
channels.discord.enabled), so auto-detection of the discord
extra works when config uses the standard nested format.
* fix(docker): remove hard-coded UV_EXTRAS=discord from dev compose
Relies on auto-detection via detect_uv_extras.py instead of forcing
discord.py install even when channels.discord.enabled is false.
Matches production docker-compose.yaml behavior (UV_EXTRAS:-).
* refactor(nginx): move proxy_buffering/proxy_cache to server level
DRY cleanup — these directives were repeated in 14 location blocks.
Set at server level once, reducing duplication and risk of drift.
* fix(discord): use dedicated JSON file for thread persistence
Replace ChannelStore usage for Discord thread-ID persistence with a
dedicated discord_threads.json file. ChannelStore is designed to map
IM conversations to DeerFlow thread IDs — using it to persist Discord
thread IDs was semantically wrong and confusing.
Changes:
- _save_thread() now reads/writes a simple {channel_id: thread_id} JSON dict
- _load_active_threads() reads directly from the JSON file
- File path derived from ChannelStore directory (when available) or
defaults to ~/.deer-flow/channels/discord_threads.json
- Removed unused ChannelStore import
* fix(discord): address WillemJiang's code review comments on PR #2842
1. Remove semantically incorrect message_in_thread variable. At this code
point (after the Thread case is handled above), we're guaranteed to be in
a channel, not a thread. Always apply mention_only check here.
2. Add _active_thread_ids reverse-lookup set for O(1) thread ID membership
checks instead of O(n) scan of _active_threads.values(). Keep the set
in sync with _active_threads in _load_active_threads() and _save_thread().
3. Add _thread_store_lock (threading.Lock) to protect _active_threads and
the JSON file from concurrent access between the Discord loop thread
(_run_client) and the main thread (_load_active_threads, _save_thread).
|
||
|
AochenShen99
|
c3bc6c7cd5
|
fix(nginx): defer CORS to gateway allowlist (#2861)
* fix(nginx): defer cors to gateway allowlist Remove proxy-level wildcard CORS handling so browser origins are controlled by the Gateway allowlist and stay aligned with CSRF origin checks. * docs: document gateway cors allowlist Clarify that same-origin nginx access needs no CORS headers while split-origin or port-forwarded browser clients must opt in with GATEWAY_CORS_ORIGINS. * docs(gateway): record cors source of truth Document that Gateway CORSMiddleware and CSRFMiddleware share GATEWAY_CORS_ORIGINS as the split-origin source of truth. * fix(gateway): align cors origin normalization * docs: clarify gateway langgraph routing * docs(gateway): update runtime routing note |
||
|
Willem Jiang
|
70737af7cd
|
fix(nignx):resolve CSRF auth failure on non-standard ports (#2796) | ||
|
Willem Jiang
|
028493bfd8
|
fix(docker):force ngix to resolve upstream names at request time (#2717)
* fix(docker):force ngix to resolve upstream names at request time * fix(docker): set resolver valid=0s to eliminate DNS cache window for request-time re-resolution Agent-Logs-Url: https://github.com/bytedance/deer-flow/sessions/07bdb872-022f-4fd2-9fa8-d800a4ce34a7 Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com> * Update DNS resolver valid time and add upstreams * fix the unit test error * Remove upstream server configurations from nginx.conf Removed upstream server configurations for gateway and frontend. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> |
||
|
Chincherry93
|
88d47f677f
|
fix(nginx): add catch-all /api/ location for auth routes (#2657)
The recent refactor (7bf618de) removed the langgraph upstream and added individual /api/* prefix locations for models, memory, mcp, skills, agents, threads, and sandboxes. However, /api/v1/auth/* routes (login, register, setup-status, etc.) were not covered by any explicit location block, causing them to fall through to the frontend catch-all. In Docker production builds, Next.js rewrites are baked at build time with http://127.0.0.1:8001 (the gateway is unreachable from the frontend container's localhost), resulting in ECONNREFUSED errors for all auth operations. Adding a catch-all `location /api/` block after the more specific prefix/regex locations ensures all gateway API routes are properly proxied. nginx's location matching priority means the more specific locations above still take precedence for /api/langgraph/, /api/models, /api/memory, /api/mcp, /api/skills, /api/agents, /api/threads/*, and /api/sandboxes. Co-authored-by: Chincherry93 <Chincherry93@users.noreply.github.com> |
||
|
JeffJiang
|
7bf618de67 |
Refactor DeerFlow to use Gateway's LangGraph-compatible API
- Updated documentation and comments to reflect the transition from LangGraph Server to Gateway. - Changed default URLs in ChannelManager and tests to point to Gateway. - Removed references to LangGraph Server in deployment scripts and configurations. - Updated Nginx configuration to route API traffic to Gateway. - Adjusted frontend configurations to utilize Gateway's API. - Removed LangGraph service from Docker Compose files, consolidating services under Gateway. - Added regression tests to ensure Gateway integration works as expected. Co-authored-by: Copilot <copilot@github.com> |
||
|
greatmengqi
|
94eee95fe0 |
feat(auth): release-validation pass for 2.0-rc — 12 blockers + simplify follow-ups (#2008)
* feat(auth): introduce backend auth module Port RFC-001 authentication core from PR #1728: - JWT token handling (create_access_token, decode_token, TokenPayload) - Password hashing (bcrypt) with verify_password - SQLite UserRepository with base interface - Provider Factory pattern (LocalAuthProvider) - CLI reset_admin tool - Auth-specific errors (AuthErrorCode, TokenError, AuthErrorResponse) Deps: - bcrypt>=4.0.0 - pyjwt>=2.9.0 - email-validator>=2.0.0 - backend/uv.toml pins public PyPI index Tests: 12 pure unit tests (test_auth_config.py, test_auth_errors.py). Scope note: authz.py, test_auth.py, and test_auth_type_system.py are deferred to commit 2 because they depend on middleware and deps wiring that is not yet in place. Commit 1 stays "pure new files only" as the spec mandates. * feat(auth): wire auth end-to-end (middleware + frontend replacement) Backend: - Port auth_middleware, csrf_middleware, langgraph_auth, routers/auth - Port authz decorator (owner_filter_key defaults to 'owner_id') - Merge app.py: register AuthMiddleware + CSRFMiddleware + CORS, add _ensure_admin_user lifespan hook, _migrate_orphaned_threads helper, register auth router - Merge deps.py: add get_local_provider, get_current_user_from_request, get_optional_user_from_request; keep get_current_user as thin str|None adapter for feedback router - langgraph.json: add auth path pointing to langgraph_auth.py:auth - Rename metadata['user_id'] -> metadata['owner_id'] in langgraph_auth (both metadata write and LangGraph filter dict) + test fixtures Frontend: - Delete better-auth library and api catch-all route - Remove better-auth npm dependency and env vars (BETTER_AUTH_SECRET, BETTER_AUTH_GITHUB_*) from env.js - Port frontend/src/core/auth/* (AuthProvider, gateway-config, proxy-policy, server-side getServerSideUser, types) - Port frontend/src/core/api/fetcher.ts - Port (auth)/layout, (auth)/login, (auth)/setup pages - Rewrite workspace/layout.tsx as server component that calls getServerSideUser and wraps in AuthProvider - Port workspace/workspace-content.tsx for the client-side sidebar logic Tests: - Port 5 auth test files (test_auth, test_auth_middleware, test_auth_type_system, test_ensure_admin, test_langgraph_auth) - 176 auth tests PASS After this commit: login/logout/registration flow works, but persistence layer does not yet filter by owner_id. Commit 4 closes that gap. * feat(auth): account settings page + i18n - Port account-settings-page.tsx (change password, change email, logout) - Wire into settings-dialog.tsx as new "account" section with UserIcon, rendered first in the section list - Add i18n keys: - en-US/zh-CN: settings.sections.account ("Account" / "账号") - en-US/zh-CN: button.logout ("Log out" / "退出登录") - types.ts: matching type declarations * feat(auth): enforce owner_id across 2.0-rc persistence layer Add request-scoped contextvar-based owner filtering to threads_meta, runs, run_events, and feedback repositories. Router code is unchanged — isolation is enforced at the storage layer so that any caller that forgets to pass owner_id still gets filtered results, and new routes cannot accidentally leak data. Core infrastructure ------------------- - deerflow/runtime/user_context.py (new): - ContextVar[CurrentUser | None] with default None - runtime_checkable CurrentUser Protocol (structural subtype with .id) - set/reset/get/require helpers - AUTO sentinel + resolve_owner_id(value, method_name) for sentinel three-state resolution: AUTO reads contextvar, explicit str overrides, explicit None bypasses the filter (for migration/CLI) Repository changes ------------------ - ThreadMetaRepository: create/get/search/update_*/delete gain owner_id=AUTO kwarg; read paths filter by owner, writes stamp it, mutations check ownership before applying - RunRepository: put/get/list_by_thread/delete gain owner_id=AUTO kwarg - FeedbackRepository: create/get/list_by_run/list_by_thread/delete gain owner_id=AUTO kwarg - DbRunEventStore: list_messages/list_events/list_messages_by_run/ count_messages/delete_by_thread/delete_by_run gain owner_id=AUTO kwarg. Write paths (put/put_batch) read contextvar softly: when a request-scoped user is available, owner_id is stamped; background worker writes without a user context pass None which is valid (orphan row to be bound by migration) Schema ------ - persistence/models/run_event.py: RunEventRow.owner_id = Mapped[ str | None] = mapped_column(String(64), nullable=True, index=True) - No alembic migration needed: 2.0 ships fresh, Base.metadata.create_all picks up the new column automatically Middleware ---------- - auth_middleware.py: after cookie check, call get_optional_user_from_ request to load the real User, stamp it into request.state.user AND the contextvar via set_current_user, reset in a try/finally. Public paths and unauthenticated requests continue without contextvar, and @require_auth handles the strict 401 path Test infrastructure ------------------- - tests/conftest.py: @pytest.fixture(autouse=True) _auto_user_context sets a default SimpleNamespace(id="test-user-autouse") on every test unless marked @pytest.mark.no_auto_user. Keeps existing 20+ persistence tests passing without modification - pyproject.toml [tool.pytest.ini_options]: register no_auto_user marker so pytest does not emit warnings for opt-out tests - tests/test_user_context.py: 6 tests covering three-state semantics, Protocol duck typing, and require/optional APIs - tests/test_thread_meta_repo.py: one test updated to pass owner_id= None explicitly where it was previously relying on the old default Test results ------------ - test_user_context.py: 6 passed - test_auth*.py + test_langgraph_auth.py + test_ensure_admin.py: 127 - test_run_event_store / test_run_repository / test_thread_meta_repo / test_feedback: 92 passed - Full backend suite: 1905 passed, 2 failed (both @requires_llm flaky integration tests unrelated to auth), 1 skipped * feat(auth): extend orphan migration to 2.0-rc persistence tables _ensure_admin_user now runs a three-step pipeline on every boot: Step 1 (fatal): admin user exists / is created / password is reset Step 2 (non-fatal): LangGraph store orphan threads → admin Step 3 (non-fatal): SQL persistence tables → admin - threads_meta - runs - run_events - feedback Each step is idempotent. The fatal/non-fatal split mirrors PR #1728's original philosophy: admin creation failure blocks startup (the system is unusable without an admin), whereas migration failures log a warning and let the service proceed (a partial migration is recoverable; a missing admin is not). Key helpers ----------- - _iter_store_items(store, namespace, *, page_size=500): async generator that cursor-paginates across LangGraph store pages. Fixes PR #1728's hardcoded limit=1000 bug that would silently lose orphans beyond the first page. - _migrate_orphaned_threads(store, admin_user_id): Rewritten to use _iter_store_items. Returns the migrated count so the caller can log it; raises only on unhandled exceptions. - _migrate_orphan_sql_tables(admin_user_id): Imports the 4 ORM models lazily, grabs the shared session factory, runs one UPDATE per table in a single transaction, commits once. No-op when no persistence backend is configured (in-memory dev). Tests: test_ensure_admin.py (8 passed) * test(auth): port AUTH test plan docs + lint/format pass - Port backend/docs/AUTH_TEST_PLAN.md and AUTH_UPGRADE.md from PR #1728 - Rename metadata.user_id → metadata.owner_id in AUTH_TEST_PLAN.md (4 occurrences from the original PR doc) - ruff auto-fix UP037 in sentinel type annotations: drop quotes around "str | None | _AutoSentinel" now that from __future__ import annotations makes them implicit string forms - ruff format: 2 files (app/gateway/app.py, runtime/user_context.py) Note on test coverage additions: - conftest.py autouse fixture was already added in commit 4 (had to be co-located with the repository changes to keep pre-existing persistence tests passing) - cross-user isolation E2E tests (test_owner_isolation.py) deferred — enforcement is already proven by the 98-test repository suite via the autouse fixture + explicit _AUTO sentinel exercises - New test cases (TC-API-17..20, TC-ATK-13, TC-MIG-01..07) listed in AUTH_TEST_PLAN.md are deferred to a follow-up PR — they are manual-QA test cases rather than pytest code, and the spec-level coverage is already met by test_user_context.py + the 98-test repository suite. Final test results: - Auth suite (test_auth*, test_langgraph_auth, test_ensure_admin, test_user_context): 186 passed - Persistence suite (test_run_event_store, test_run_repository, test_thread_meta_repo, test_feedback): 98 passed - Lint: ruff check + ruff format both clean * test(auth): add cross-user isolation test suite 10 tests exercising the storage-layer owner filter by manually switching the user_context contextvar between two users. Verifies the safety invariant: After a repository write with owner_id=A, a subsequent read with owner_id=B must not return the row, and vice versa. Covers all 4 tables that own user-scoped data: TC-API-17 threads_meta — read, search, update, delete cross-user TC-API-18 runs — get, list_by_thread, delete cross-user TC-API-19 run_events — list_messages, list_events, count_messages, delete_by_thread (CRITICAL: raw conversation content leak vector) TC-API-20 feedback — get, list_by_run, delete cross-user Plus two meta-tests verifying the sentinel pattern itself: - AUTO + unset contextvar raises RuntimeError - explicit owner_id=None bypasses the filter (migration escape hatch) Architecture note ----------------- These tests bypass the HTTP layer by design. The full chain (cookie → middleware → contextvar → repository) is covered piecewise: - test_auth_middleware.py: middleware sets contextvar from cookies - test_owner_isolation.py: repositories enforce isolation when contextvar is set to different users Together they prove the end-to-end safety property without the ceremony of spinning up a full TestClient + in-memory DB for every router endpoint. Tests pass: 231 (full auth + persistence + isolation suite) Lint: clean * refactor(auth): migrate user repository to SQLAlchemy ORM Move the users table into the shared persistence engine so auth matches the pattern of threads_meta, runs, run_events, and feedback — one engine, one session factory, one schema init codepath. New files --------- - persistence/user/__init__.py, persistence/user/model.py: UserRow ORM class with partial unique index on (oauth_provider, oauth_id) - Registered in persistence/models/__init__.py so Base.metadata.create_all() picks it up Modified -------- - auth/repositories/sqlite.py: rewritten as async SQLAlchemy, identical constructor pattern to the other four repositories (def __init__(self, session_factory) + self._sf = session_factory) - auth/config.py: drop users_db_path field — storage is configured through config.database like every other table - deps.py/get_local_provider: construct SQLiteUserRepository with the shared session factory, fail fast if engine is not initialised - tests/test_auth.py: rewrite test_sqlite_round_trip_new_fields to use the shared engine (init_engine + close_engine in a tempdir) - tests/test_auth_type_system.py: add per-test autouse fixture that spins up a scratch engine and resets deps._cached_* singletons * refactor(auth): remove SQL orphan migration (unused in supported scenarios) The _migrate_orphan_sql_tables helper existed to bind NULL owner_id rows in threads_meta, runs, run_events, and feedback to the admin on first boot. But in every supported upgrade path, it's a no-op: 1. Fresh install: create_all builds fresh tables, no legacy rows 2. No-auth → with-auth (no existing persistence DB): persistence tables are created fresh by create_all, no legacy rows 3. No-auth → with-auth (has existing persistence DB from #1930): NOT a supported upgrade path — "有 DB 到有 DB" schema evolution is out of scope; users wipe DB or run manual ALTER So the SQL orphan migration never has anything to do in the supported matrix. Delete the function, simplify _ensure_admin_user from a 3-step pipeline to a 2-step one (admin creation + LangGraph store orphan migration only). LangGraph store orphan migration stays: it serves the real "no-auth → with-auth" upgrade path where a user's existing LangGraph thread metadata has no owner_id field and needs to be stamped with the newly-created admin's id. Tests: 284 passed (auth + persistence + isolation) Lint: clean * security(auth): write initial admin password to 0600 file instead of logs CodeQL py/clear-text-logging-sensitive-data flagged 3 call sites that logged the auto-generated admin password to stdout via logger.info(). Production log aggregators (ELK/Splunk/etc) would have captured those cleartext secrets. Replace with a shared helper that writes to .deer-flow/admin_initial_credentials.txt with mode 0600, and log only the path. New file -------- - app/gateway/auth/credential_file.py: write_initial_credentials() helper. Takes email, password, and a "initial"/"reset" label. Creates .deer-flow/ if missing, writes a header comment plus the email+password, chmods 0o600, returns the absolute Path. Modified -------- - app/gateway/app.py: both _ensure_admin_user paths (fresh creation + needs_setup password reset) now write to file and log the path - app/gateway/auth/reset_admin.py: rewritten to use the shared ORM repo (SQLiteUserRepository with session_factory) and the credential_file helper. The previous implementation was broken after the earlier ORM refactor — it still imported _get_users_conn and constructed SQLiteUserRepository() without a session factory. No tests changed — the three password-log sites are all exercised via existing test_ensure_admin.py which checks that startup succeeds, not that a specific string appears in logs. CodeQL alerts 272, 283, 284: all resolved. * security(auth): strict JWT validation in middleware (fix junk cookie bypass) AUTH_TEST_PLAN test 7.5.8 expects junk cookies to be rejected with 401. The previous middleware behaviour was "presence-only": check that some access_token cookie exists, then pass through. In combination with my Task-12 decision to skip @require_auth decorators on routes, this created a gap where a request with any cookie-shaped string (e.g. access_token=not-a-jwt) would bypass authentication on routes that do not touch the repository (/api/models, /api/mcp/config, /api/memory, /api/skills, …). Fix: middleware now calls get_current_user_from_request() strictly and catches the resulting HTTPException to render a 401 with the proper fine-grained error code (token_invalid, token_expired, user_not_found, …). On success it stamps request.state.user and the contextvar so repository-layer owner filters work downstream. The 4 old "_with_cookie_passes" tests in test_auth_middleware.py were written for the presence-only behaviour; they asserted that a junk cookie would make the handler return 200. They are renamed to "_with_junk_cookie_rejected" and their assertions flipped to 401. The negative path (no cookie → 401 not_authenticated) is unchanged. Verified: no cookie → 401 not_authenticated junk cookie → 401 token_invalid (the fixed bug) expired cookie → 401 token_expired Tests: 284 passed (auth + persistence + isolation) Lint: clean * security(auth): wire @require_permission(owner_check=True) on isolation routes Apply the require_permission decorator to all 28 routes that take a {thread_id} path parameter. Combined with the strict middleware (previous commit), this gives the double-layer protection that AUTH_TEST_PLAN test 7.5.9 documents: Layer 1 (AuthMiddleware): cookie + JWT validation, rejects junk cookies and stamps request.state.user Layer 2 (@require_permission with owner_check=True): per-resource ownership verification via ThreadMetaStore.check_access — returns 404 if a different user owns the thread The decorator's owner_check branch is rewritten to use the SQL thread_meta_repo (the 2.0-rc persistence layer) instead of the LangGraph store path that PR #1728 used (_store_get / get_store in routers/threads.py). The inject_record convenience is dropped — no caller in 2.0 needs the LangGraph blob, and the SQL repo has a different shape. Routes decorated (28 total): - threads.py: delete, patch, get, get-state, post-state, post-history - thread_runs.py: post-runs, post-runs-stream, post-runs-wait, list_runs, get_run, cancel_run, join_run, stream_existing_run, list_thread_messages, list_run_messages, list_run_events, thread_token_usage - feedback.py: create, list, stats, delete - uploads.py: upload (added Request param), list, delete - artifacts.py: get_artifact - suggestions.py: generate (renamed body parameter to avoid conflict with FastAPI Request) Test fixes: - test_suggestions_router.py: bypass the decorator via __wrapped__ (the unit tests cover parsing logic, not auth — no point spinning up a thread_meta_repo just to test JSON unwrapping) - test_auth_middleware.py 4 fake-cookie tests: already updated in the previous commit (745bf432) Tests: 293 passed (auth + persistence + isolation + suggestions) Lint: clean * security(auth): defense-in-depth fixes from release validation pass Eight findings caught while running the AUTH_TEST_PLAN end-to-end against the deployed sg_dev stack. Each is a pre-condition for shipping release/2.0-rc that the previous PRs missed. Backend hardening - routers/auth.py: rate limiter X-Real-IP now requires AUTH_TRUSTED_PROXIES whitelist (CIDR/IP allowlist). Without nginx in front, the previous code honored arbitrary X-Real-IP, letting an attacker rotate the header to fully bypass the per-IP login lockout. - routers/auth.py: 36-entry common-password blocklist via Pydantic field_validator on RegisterRequest + ChangePasswordRequest. The shared _validate_strong_password helper keeps the constraint in one place. - routers/threads.py: ThreadCreateRequest + ThreadPatchRequest strip server-reserved metadata keys (owner_id, user_id) via Pydantic field_validator so a forged value can never round-trip back to other clients reading the same thread. The actual ownership invariant stays on the threads_meta row; this closes the metadata-blob echo gap. - authz.py + thread_meta/sql.py: require_permission gains a require_existing flag plumbed through check_access(require_existing=True). Destructive routes (DELETE/PATCH/state-update/runs/feedback) now treat a missing thread_meta row as 404 instead of "untracked legacy thread, allow", closing the cross-user delete-idempotence gap where any user could successfully DELETE another user's deleted thread. - repositories/sqlite.py + base.py: update_user raises UserNotFoundError on a vanished row instead of silently returning the input. Concurrent delete during password reset can no longer look like a successful update. - runtime/user_context.py: resolve_owner_id() coerces User.id (UUID) to str at the contextvar boundary so SQLAlchemy String(64) columns can bind it. The whole 2.0-rc isolation pipeline was previously broken end-to-end (POST /api/threads → 500 "type 'UUID' is not supported"). - persistence/engine.py: SQLAlchemy listener enables PRAGMA journal_mode=WAL, synchronous=NORMAL, foreign_keys=ON on every new SQLite connection. TC-UPG-06 in the test plan expects WAL; previous code shipped with the default 'delete' journal. - auth_middleware.py: stamp request.state.auth = AuthContext(...) so @require_permission's short-circuit fires; previously every isolation request did a duplicate JWT decode + users SELECT. Also unifies the 401 payload through AuthErrorResponse(...).model_dump(). - app.py: _ensure_admin_user restructure removes the noqa F821 scoping bug where 'password' was referenced outside the branch that defined it. New _announce_credentials helper absorbs the duplicate log block in the fresh-admin and reset-admin branches. * fix(frontend+nginx): rollout CSRF on every state-changing client path The frontend was 100% broken in gateway-pro mode for any user trying to open a specific chat thread. Three cumulative bugs each silently masked the next. LangGraph SDK CSRF gap (api-client.ts) - The Client constructor took only apiUrl, no defaultHeaders, no fetch interceptor. The SDK's internal fetch never sent X-CSRF-Token, so every state-changing /api/langgraph-compat/* call (runs/stream, threads/search, threads/{tid}/history, ...) hit CSRFMiddleware and got 403 before reaching the auth check. UI symptom: empty thread page with no error message; the SPA's hooks swallowed the rejection. - Fix: pass an onRequest hook that injects X-CSRF-Token from the csrf_token cookie per request. Reading the cookie per call (not at construction time) handles login / logout / password-change cookie rotation transparently. The SDK's prepareFetchOptions calls onRequest for both regular requests AND streaming/SSE/reconnect, so the same hook covers runs.stream and runs.joinStream. Raw fetch CSRF gap (7 files) - Audit: 11 frontend fetch sites, only 2 included CSRF (login/setup + account-settings change-password). The other 7 routed through raw fetch() with no header — suggestions, memory, agents, mcp, skills, uploads, and the local thread cleanup hook all 403'd silently. - Fix: enhance fetcher.ts:fetchWithAuth to auto-inject X-CSRF-Token on POST/PUT/DELETE/PATCH from a single shared readCsrfCookie() helper. Convert all 7 raw fetch() callers to fetchWithAuth so the contract is centrally enforced. api-client.ts and fetcher.ts share readCsrfCookie + STATE_CHANGING_METHODS to avoid drift. nginx routing + buffering (nginx.local.conf) - The auth feature shipped without updating the nginx config: per-API explicit location blocks but no /api/v1/auth/, /api/feedback, /api/runs. The frontend's client-side fetches to /api/v1/auth/login/local 404'd from the Next.js side because nginx routed /api/* to the frontend. - Fix: add catch-all `location /api/` that proxies to the gateway. nginx longest-prefix matching keeps the explicit blocks (/api/models, /api/threads regex, /api/langgraph/, ...) winning for their paths. - Fix: disable proxy_buffering + proxy_request_buffering for the frontend `location /` block. Without it, nginx tries to spool large Next.js chunks into /var/lib/nginx/proxy (root-owned) and fails with Permission denied → ERR_INCOMPLETE_CHUNKED_ENCODING → ChunkLoadError. * test(auth): release-validation test infra and new coverage Test fixtures and unit tests added during the validation pass. Router test helpers (NEW: tests/_router_auth_helpers.py) - make_authed_test_app(): builds a FastAPI test app with a stub middleware that stamps request.state.user + request.state.auth and a permissive thread_meta_repo mock. TestClient-based router tests (test_artifacts_router, test_threads_router) use it instead of bare FastAPI() so the new @require_permission(owner_check=True) decorators short-circuit cleanly. - call_unwrapped(): walks the __wrapped__ chain to invoke the underlying handler without going through the authz wrappers. Direct-call tests (test_uploads_router) use it. Typed with ParamSpec so the wrapped signature flows through. Backend test additions - test_auth.py: 7 tests for the new _get_client_ip trust model (no proxy / trusted proxy / untrusted peer / XFF rejection / invalid CIDR / no client). 5 tests for the password blocklist (literal, case-insensitive, strong password accepted, change-password binding, short-password length-check still fires before blocklist). test_update_user_raises_when_row_concurrently_deleted: closes a shipped-without-coverage gap on the new UserNotFoundError contract. - test_thread_meta_repo.py: 4 tests for check_access(require_existing=True) — strict missing-row denial, strict owner match, strict owner mismatch, strict null-owner still allowed (shared rows survive the tightening). - test_ensure_admin.py: 3 tests for _migrate_orphaned_threads / _iter_store_items pagination, covering the TC-UPG-02 upgrade story end-to-end via mock store. Closes the gap where the cursor pagination was untested even though the previous PR rewrote it. - test_threads_router.py: 5 tests for _strip_reserved_metadata (owner_id removal, user_id removal, safe-keys passthrough, empty input, both-stripped). - test_auth_type_system.py: replace "password123" fixtures with Tr0ub4dor3a / AnotherStr0ngPwd! so the new password blocklist doesn't reject the test data. * docs(auth): refresh TC-DOCKER-05 + document Docker validation gap - AUTH_TEST_PLAN.md TC-DOCKER-05: the previous expectation ("admin password visible in docker logs") was stale after the simplify pass that moved credentials to a 0600 file. The grep "Password:" check would have silently failed and given a false sense of coverage. New expectation matches the actual file-based path: 0600 file in DEER_FLOW_HOME, log shows the path (not the secret), reverse-grep asserts no leaked password in container logs. - NEW: docs/AUTH_TEST_DOCKER_GAP.md documents the only un-executed block in the test plan (TC-DOCKER-01..06). Reason: sg_dev validation host has no Docker daemon installed. The doc maps each Docker case to an already-validated bare-metal equivalent (TC-1.1, TC-REENT-01, TC-API-02 etc.) so the gap is auditable, and includes pre-flight reproduction steps for whoever has Docker available. --------- Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com> |
||
|
greatmengqi
|
ca2fb95ee6
|
feat: unified serve.sh with gateway mode support (#1847) | ||
|
Octopus
|
fd310582bd
|
fix: remove nginx Plus-only zone/resolve directives from nginx.conf (#1837)
* fix: add missing DEER_FLOW_CONFIG_PATH and DEER_FLOW_EXTENSIONS_CONFIG_PATH env vars to gateway service (fixes #1829) The gateway service was missing these two environment variables that tell it where to find the config files inside the container. Without them, the gateway reads DEER_FLOW_CONFIG_PATH from the host's .env file (set to a host filesystem path), which is not accessible inside the container, causing FileNotFoundError on startup. The langgraph service already had these variables set correctly. * fix: remove nginx Plus-only zone/resolve directives from nginx.conf (fixes #1744) The `zone` and `resolve` parameters in upstream server directives are nginx Plus features not available in the standard `nginx:alpine` image. This caused nginx to fail at startup with: [emerg] invalid parameter "resolve" in /etc/nginx/nginx.conf:25 Remove these directives so the config is compatible with open-source nginx. Docker's internal DNS (127.0.0.11, already configured via `resolver`) handles service name resolution. The `resolver` directive is kept for the provisioner location which uses variable-based proxy_pass for optional-service support. |
||
|
JeffJiang
|
c1366cf559
|
Add documents site (#1767)
* feat: add docs site - Implemented dynamic routing for MDX documentation pages with language support. - Created layout components for documentation with a header and footer. - Added metadata for various documentation sections in English and Chinese. - Developed initial content for the DeerFlow App and Harness documentation. - Introduced i18n hooks and translations for English and Chinese languages. - Enhanced header component to include navigation links for documentation and blog. - Established a structure for tutorials and reference materials. - Created a new translations file to manage locale-specific strings. * feat: enhance documentation structure and content for application and harness sections * feat: update .gitignore to include .playwright-mcp and remove obsolete Playwright YAML file * fix(docs): correct punctuation and formatting in documentation files * feat(docs): remove outdated index.mdx file from documentation * fix(docs): update documentation links and improve Chinese description in index.mdx * fix(docs): update title in Chinese for meta information in _meta.ts |
||
|
rayhpeng
|
34e835bc33
|
feat(gateway): implement LangGraph Platform API in Gateway, replace langgraph-cli (#1403)
* feat(gateway): implement LangGraph Platform API in Gateway, replace langgraph-cli
Implement all core LangGraph Platform API endpoints in the Gateway,
allowing it to fully replace the langgraph-cli dev server for local
development. This eliminates a heavyweight dependency and simplifies
the development stack.
Changes:
- Add runs lifecycle endpoints (create, stream, wait, cancel, join)
- Add threads CRUD and search endpoints
- Add assistants compatibility endpoints (search, get, graph, schemas)
- Add StreamBridge (in-memory pub/sub for SSE) and async provider
- Add RunManager with atomic create_or_reject (eliminates TOCTOU race)
- Add worker with interrupt/rollback cancel actions and runtime context injection
- Route /api/langgraph/* to Gateway in nginx config
- Skip langgraph-cli startup by default (SKIP_LANGGRAPH_SERVER=0 to restore)
- Add unit tests for RunManager, SSE format, and StreamBridge
* fix: drain bridge queue on client disconnect to prevent backpressure
When on_disconnect=continue, keep consuming events from the bridge
without yielding, so the worker is not blocked by a full queue.
Only on_disconnect=cancel breaks out immediately.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: remove pytest import
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: Fix default stream_mode to ["values", "messages-tuple"]
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: Remove unused if_exists field from ThreadCreateRequest
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: address review comments on gateway LangGraph API
- Mount runs.py router in app.py (missing include_router)
- Normalize interrupt_before/after "*" to node list before run_agent()
- Use entry.id for SSE event ID instead of counter
- Drain bridge queue on disconnect when on_disconnect=continue
- Reuse serialization helper in wait_run() for consistent wire format
- Reject unsupported multitask_strategy with 400
- Remove SKIP_LANGGRAPH_SERVER fallback, always use Gateway
* feat: extract app.state access into deps.py
Encapsulate read/write operations for singleton objects (RunManager,
StreamBridge, checkpointer) held in app.state into a shared utility,
reducing repeated access patterns across router modules.
* feat: extract deerflow.runtime.serialization module with tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: replace duplicated serialization with deerflow.runtime.serialization
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: extract app/gateway/services.py with run lifecycle logic
Create a service layer that centralizes SSE formatting, input/config
normalization, and run lifecycle management. Router modules will delegate
to these functions instead of using private cross-imported helpers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: wire routers to use services layer, remove cross-module private imports
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: apply ruff formatting to refactored files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(runtime): support LangGraph dev server and add compat route
- Enable official LangGraph dev server for local development workflow
- Decouple runtime components from agents package for better separation
- Provide gateway-backed fallback route when dev server is skipped
- Simplify lifecycle management using context manager in gateway
* feat(runtime): add Store providers with auto-backend selection
- Add async_provider.py and provider.py under deerflow/runtime/store/
- Support memory, sqlite, postgres backends matching checkpointer config
- Integrate into FastAPI lifespan via AsyncExitStack in deps.py
- Replace hardcoded InMemoryStore with config-driven factory
* refactor(gateway): migrate thread management from checkpointer to Store and resolve multiple endpoint failures
- Add Store-backed CRUD helpers (_store_get, _store_put, _store_upsert)
- Replace checkpoint-scanning search with two-phase strategy:
phase 1 reads Store (O(threads)), phase 2 backfills from checkpointer
for legacy/LangGraph Server threads with lazy migration
- Extend Store record schema with values field for title persistence
- Sync thread title from checkpoint to Store after run completion
- Fix /threads/{id}/runs/{run_id}/stream 405 by accepting both
GET and POST methods; POST handles interrupt/rollback actions
- Fix /threads/{id}/state 500 by separating read_config and
write_config, adding checkpoint_ns to configurable, and
shallow-copying checkpoint/metadata before mutation
- Sync title to Store on state update for immediate search reflection
- Move _upsert_thread_in_store into services.py, remove duplicate logic
- Add _sync_thread_title_after_run: await run task, read final
checkpoint title, write back to Store record
- Spawn title sync as background task from start_run when Store exists
* refactor(runtime): deduplicate store and checkpointer provider logic
Extract _ensure_sqlite_parent_dir() helper into checkpointer/provider.py
and use it in all three places that previously inlined the same mkdir logic.
Consolidate duplicate error constants in store/async_provider.py by importing
from store/provider.py instead of redefining them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(runtime): move SQLite helpers to runtime/store, checkpointer imports from store
_resolve_sqlite_conn_str and _ensure_sqlite_parent_dir now live in
runtime/store/provider.py. agents/checkpointer/provider and
agents/checkpointer/async_provider import from there, reversing the
previous dependency direction (store → checkpointer becomes
checkpointer → store).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(runtime): extract SQLite helpers into runtime/store/_sqlite_utils.py
Move resolve_sqlite_conn_str and ensure_sqlite_parent_dir out of
checkpointer/provider.py into a dedicated _sqlite_utils module.
Functions are now public (no underscore prefix), making cross-module
imports semantically correct. All four provider files import from
the single shared location.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(gateway): use adelete_thread to fully remove thread checkpoints on delete
AsyncSqliteSaver has no adelete method — the previous hasattr check
always evaluated to False, silently leaving all checkpoint rows in the
database. Switch to adelete_thread(thread_id) which deletes every
checkpoint and pending-write row for the thread across all namespaces
(including sub-graph checkpoints).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(gateway): remove dead bridge_cm/ckpt_cm code and fix StrEnum lint
app.py had unreachable code after the async-with lifespan refactor:
bridge_cm and ckpt_cm were referenced but never defined (F821), and
the channel service startup/shutdown was outside the langgraph_runtime
block so it never ran. Move channel service lifecycle inside the
async-with block where it belongs.
Replace str+Enum inheritance in RunStatus and DisconnectMode with
StrEnum as suggested by UP042.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: format with ruff
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: JeffJiang <for-eleven@hotmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|
||
|
passer
|
75c4757f48
|
fix(nginx): re-resolve upstream DNS in Docker (#1517)
Enable runtime DNS re-resolution for docker-compose upstreams (gateway/langgraph/frontend) to avoid stale container IPs causing persistent 502s. |
||
|
JeffJiang
|
3721c82ba8
|
Update Nginx configuration for uploads and improve thread ID handling (#1023)
* fix: update nginx configuration for uploads endpoint and improve thread ID handling in hooks * Update docker/nginx/nginx.local.conf Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update frontend/src/core/threads/hooks.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Xinmin Zeng
|
0c7c96d75e
|
fix(nginx): use cross-platform local paths for pid and logs (#977)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
JeffJiang
|
7de94394d4
|
feat(agent):Supports custom agent and chat experience with refactoring (#957)
* feat: add agent management functionality with creation, editing, and deletion * feat: enhance agent creation and chat experience - Added AgentWelcome component to display agent description on new thread creation. - Improved agent name validation with availability check during agent creation. - Updated NewAgentPage to handle agent creation flow more effectively, including enhanced error handling and user feedback. - Refactored chat components to streamline message handling and improve user experience. - Introduced new bootstrap skill for personalized onboarding conversations, including detailed conversation phases and a structured SOUL.md template. - Updated localization files to reflect new features and error messages. - General code cleanup and optimizations across various components and hooks. * Refactor workspace layout and agent management components - Updated WorkspaceLayout to use useLayoutEffect for sidebar state initialization. - Removed unused AgentFormDialog and related edit functionality from AgentCard. - Introduced ArtifactTrigger component to manage artifact visibility. - Enhanced ChatBox to handle artifact selection and display. - Improved message list rendering logic to avoid loading states. - Updated localization files to remove deprecated keys and add new translations. - Refined hooks for local settings and thread management to improve performance and clarity. - Added temporal awareness guidelines to deep research skill documentation. * feat: refactor chat components and introduce thread management hooks * feat: improve artifact file detail preview logic and clean up console logs * feat: refactor lead agent creation logic and improve logging details * feat: validate agent name format and enhance error handling in agent setup * feat: simplify thread search query by removing unnecessary metadata * feat: update query key in useDeleteThread and useRenameThread for consistency * feat: add isMock parameter to thread and artifact handling for improved testing * fix: reorder import of setup_agent for consistency in builtins module * feat: append mock parameter to thread links in CaseStudySection for testing purposes * fix: update load_agent_soul calls to use cfg.name for improved clarity * fix: update date format in apply_prompt_template for consistency * feat: integrate isMock parameter into artifact content loading for enhanced testing * docs: add license section to SKILL.md for clarity and attribution * feat(agent): enhance model resolution and agent configuration handling * chore: remove unused import of _resolve_model_name from agents * feat(agent): remove unused field * fix(agent): set default value for requested_model_name in _resolve_model_name function * feat(agent): update get_available_tools call to handle optional agent_config and improve middleware function signature --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
JeffJiang
|
3a7251c95e
|
fix(docker): update nginx configuration and simplify docker script (#903) | ||
|
CHANGXUBO
|
67dbb10c2a
|
fix: use /tmp/nginx.pid to avoid permission denied errors (#877)
Set pid directive to /tmp/nginx.pid in nginx.conf and nginx.local.conf to prevent permission denied errors when running nginx as a non-root user. Co-authored-by: Claude <noreply@anthropic.com> |
||
|
JeffJiang
|
300e5a519a
|
Adds Kubernetes sandbox provisioner support (#35)
* Adds Kubernetes sandbox provisioner support * Improves Docker dev setup by standardizing host paths Replaces hardcoded host paths with a configurable root directory, making the development environment more portable and easier to use across different machines. Automatically sets the root path if not already defined, reducing manual setup steps. |
||
|
JeffJiang
|
7b7e32f262
|
Add Kubernetes-based sandbox provider for multi-instance support (#19)
* feat: adds docker-based dev environment * docs: updates Docker command help * fix local dev * feat(sandbox): add Kubernetes-based sandbox provider for multi-instance support * fix: skills path in k8s * feat: add example config for k8s sandbox * fix: docker config * fix: load skills on docker dev * feat: support sandbox execution to Kubernetes Deployment model * chore: rename web service name |
||
|
Henry Li
|
b8c325eb3a | chore: add /api/memory | ||
|
JeffJiang
|
400349c3e0
|
feat: adds docker-based dev environment (#18)
* feat: adds docker-based dev environment * docs: updates Docker command help * fix local dev |