From d8ecaf46c977513c1b6f51954ffffea631966df8 Mon Sep 17 00:00:00 2001 From: rayhpeng Date: Tue, 7 Apr 2026 11:53:52 +0800 Subject: [PATCH 01/38] feat(persistence): add unified persistence layer with event store, token tracking, and feedback (#1930) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(persistence): add SQLAlchemy 2.0 async ORM scaffold Introduce a unified database configuration (DatabaseConfig) that controls both the LangGraph checkpointer and the DeerFlow application persistence layer from a single `database:` config section. New modules: - deerflow.config.database_config — Pydantic config with memory/sqlite/postgres backends - deerflow.persistence — async engine lifecycle, DeclarativeBase with to_dict mixin, Alembic skeleton - deerflow.runtime.runs.store — RunStore ABC + MemoryRunStore implementation Gateway integration initializes/tears down the persistence engine in the existing langgraph_runtime() context manager. Legacy checkpointer config is preserved for backward compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) * feat(persistence): add RunEventStore ABC + MemoryRunEventStore Phase 2-A prerequisite for event storage: adds the unified run event stream interface (RunEventStore) with an in-memory implementation, RunEventsConfig, gateway integration, and comprehensive tests (27 cases). Co-Authored-By: Claude Opus 4.6 (1M context) * feat(persistence): add ORM models, repositories, DB/JSONL event stores, RunJournal, and API endpoints Phase 2-B: run persistence + event storage + token tracking. - ORM models: RunRow (with token fields), ThreadMetaRow, RunEventRow - RunRepository implements RunStore ABC via SQLAlchemy ORM - ThreadMetaRepository with owner access control - DbRunEventStore with trace content truncation and cursor pagination - JsonlRunEventStore with per-run files and seq recovery from disk - RunJournal (BaseCallbackHandler) captures LLM/tool/lifecycle events, accumulates token usage by caller type, buffers and flushes to store - RunManager now accepts optional RunStore for persistent backing - Worker creates RunJournal, writes human_message, injects callbacks - Gateway deps use factory functions (RunRepository when DB available) - New endpoints: messages, run messages, run events, token-usage - ThreadCreateRequest gains assistant_id field - 92 tests pass (33 new), zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) * feat(persistence): add user feedback + follow-up run association Phase 2-C: feedback and follow-up tracking. - FeedbackRow ORM model (rating +1/-1, optional message_id, comment) - FeedbackRepository with CRUD, list_by_run/thread, aggregate stats - Feedback API endpoints: create, list, stats, delete - follow_up_to_run_id in RunCreateRequest (explicit or auto-detected from latest successful run on the thread) - Worker writes follow_up_to_run_id into human_message event metadata - Gateway deps: feedback_repo factory + getter - 17 new tests (14 FeedbackRepository + 3 follow-up association) - 109 total tests pass, zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) * test+config: comprehensive Phase 2 test coverage + deprecate checkpointer config - config.example.yaml: deprecate standalone checkpointer section, activate unified database:sqlite as default (drives both checkpointer + app data) - New: test_thread_meta_repo.py (14 tests) — full ThreadMetaRepository coverage including check_access owner logic, list_by_owner pagination - Extended test_run_repository.py (+4 tests) — completion preserves fields, list ordering desc, limit, owner_none returns all - Extended test_run_journal.py (+8 tests) — on_chain_error, track_tokens=false, middleware no ai_message, unknown caller tokens, convenience fields, tool_error, non-summarization custom event - Extended test_run_event_store.py (+7 tests) — DB batch seq continuity, make_run_event_store factory (memory/db/jsonl/fallback/unknown) - Extended test_phase2b_integration.py (+4 tests) — create_or_reject persists, follow-up metadata, summarization in history, full DB-backed lifecycle - Fixed DB integration test to use proper fake objects (not MagicMock) for JSON-serializable metadata - 157 total Phase 2 tests pass, zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) * config: move default sqlite_dir to .deer-flow/data Keep SQLite databases alongside other DeerFlow-managed data (threads, memory) under the .deer-flow/ directory instead of a top-level ./data folder. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(persistence): remove UTFJSON, use engine-level json_serializer + datetime.now() - Replace custom UTFJSON type with standard sqlalchemy.JSON in all ORM models. Add json_serializer=json.dumps(ensure_ascii=False) to all create_async_engine calls so non-ASCII text (Chinese etc.) is stored as-is in both SQLite and Postgres. - Change ORM datetime defaults from datetime.now(UTC) to datetime.now(), remove UTC imports. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(gateway): simplify deps.py with getter factory + inline repos - Replace 6 identical getter functions with _require() factory. - Inline 3 _make_*_repo() factories into langgraph_runtime(), call get_session_factory() once instead of 3 times. - Add thread_meta upsert in start_run (services.py). Co-Authored-By: Claude Opus 4.6 (1M context) * feat(docker): add UV_EXTRAS build arg for optional dependencies Support installing optional dependency groups (e.g. postgres) at Docker build time via UV_EXTRAS build arg: UV_EXTRAS=postgres docker compose build Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(journal): fix flush, token tracking, and consolidate tests RunJournal fixes: - _flush_sync: retain events in buffer when no event loop instead of dropping them; worker's finally block flushes via async flush(). - on_llm_end: add tool_calls filter and caller=="lead_agent" guard for ai_message events; mark message IDs for dedup with record_llm_usage. - worker.py: persist completion data (tokens, message count) to RunStore in finally block. Model factory: - Auto-inject stream_usage=True for BaseChatOpenAI subclasses with custom api_base, so usage_metadata is populated in streaming responses. Test consolidation: - Delete test_phase2b_integration.py (redundant with existing tests). - Move DB-backed lifecycle test into test_run_journal.py. - Add tests for stream_usage injection in test_model_factory.py. - Clean up executor/task_tool dead journal references. Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): widen content type to str|dict in all store backends Allow event content to be a dict (for structured OpenAI-format messages) in addition to plain strings. Dict values are JSON-serialized for the DB backend and deserialized on read; memory and JSONL backends handle dicts natively. Trace truncation now serializes dicts to JSON before measuring. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(events): use metadata flag instead of heuristic for dict content detection Co-Authored-By: Claude Opus 4.6 (1M context) * feat(converters): add LangChain-to-OpenAI message format converters Pure functions langchain_to_openai_message, langchain_to_openai_completion, langchain_messages_to_openai, and _infer_finish_reason for converting LangChain BaseMessage objects to OpenAI Chat Completions format, used by RunJournal for event storage. 15 unit tests added. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(converters): handle empty list content as null, clean up test Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): human_message content uses OpenAI user message format Co-Authored-By: Claude Sonnet 4.6 * feat(events): ai_message uses OpenAI format, add ai_tool_call message event - ai_message content now uses {"role": "assistant", "content": "..."} format - New ai_tool_call message event emitted when lead_agent LLM responds with tool_calls - ai_tool_call uses langchain_to_openai_message converter for consistent format - Both events include finish_reason in metadata ("stop" or "tool_calls") Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): add tool_result message event with OpenAI tool message format Cache tool_call_id from on_tool_start keyed by run_id as fallback for on_tool_end, then emit a tool_result message event (role=tool, tool_call_id, content) after each successful tool completion. Co-Authored-By: Claude Sonnet 4.6 * feat(events): summary content uses OpenAI system message format Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): replace llm_start/llm_end with llm_request/llm_response in OpenAI format Add on_chat_model_start to capture structured prompt messages as llm_request events. Replace llm_end trace events with llm_response using OpenAI Chat Completions format. Track llm_call_index to pair request/response events. Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): add record_middleware method for middleware trace events Co-Authored-By: Claude Opus 4.6 (1M context) * test(events): add full run sequence integration test for OpenAI content format Co-Authored-By: Claude Sonnet 4.6 * feat(events): align message events with checkpoint format and add middleware tag injection - Message events (ai_message, ai_tool_call, tool_result, human_message) now use BaseMessage.model_dump() format, matching LangGraph checkpoint values.messages - on_tool_end extracts tool_call_id/name/status from ToolMessage objects - on_tool_error now emits tool_result message events with error status - record_middleware uses middleware:{tag} event_type and middleware category - Summarization custom events use middleware:summarize category - TitleMiddleware injects middleware:title tag via get_config() inheritance - SummarizationMiddleware model bound with middleware:summarize tag - Worker writes human_message using HumanMessage.model_dump() Co-Authored-By: Claude Opus 4.6 (1M context) * feat(threads): switch search endpoint to threads_meta table and sync title - POST /api/threads/search now queries threads_meta table directly, removing the two-phase Store + Checkpointer scan approach - Add ThreadMetaRepository.search() with metadata/status filters - Add ThreadMetaRepository.update_display_name() for title sync - Worker syncs checkpoint title to threads_meta.display_name on run completion - Map display_name to values.title in search response for API compatibility Co-Authored-By: Claude Opus 4.6 (1M context) * feat(threads): history endpoint reads messages from event store - POST /api/threads/{thread_id}/history now combines two data sources: checkpointer for checkpoint_id, metadata, title, thread_data; event store for messages (complete history, not truncated by summarization) - Strip internal LangGraph metadata keys from response - Remove full channel_values serialization in favor of selective fields Co-Authored-By: Claude Opus 4.6 (1M context) * fix: remove duplicate optional-dependencies header in pyproject.toml Co-Authored-By: Claude Opus 4.6 (1M context) * fix(middleware): pass tagged config to TitleMiddleware ainvoke call Without the config, the middleware:title tag was not injected, causing the LLM response to be recorded as a lead_agent ai_message in run_events. Co-Authored-By: Claude Opus 4.6 (1M context) * fix: resolve merge conflict in .env.example Keep both DATABASE_URL (from persistence-scaffold) and WECOM credentials (from main) after the merge. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(persistence): address review feedback on PR #1851 - Fix naive datetime.now() → datetime.now(UTC) in all ORM models - Fix seq race condition in DbRunEventStore.put() with FOR UPDATE and UNIQUE(thread_id, seq) constraint - Encapsulate _store access in RunManager.update_run_completion() - Deduplicate _store.put() logic in RunManager via _persist_to_store() - Add update_run_completion to RunStore ABC + MemoryRunStore - Wire follow_up_to_run_id through the full create path - Add error recovery to RunJournal._flush_sync() lost-event scenario - Add migration note for search_threads breaking change - Fix test_checkpointer_none_fix mock to set database=None Co-Authored-By: Claude Opus 4.6 (1M context) * chore: update uv.lock Co-Authored-By: Claude Opus 4.6 (1M context) * fix(persistence): address 22 review comments from CodeQL, Copilot, and Code Quality Bug fixes: - Sanitize log params to prevent log injection (CodeQL) - Reset threads_meta.status to idle/error when run completes - Attach messages only to latest checkpoint in /history response - Write threads_meta on POST /threads so new threads appear in search Lint fixes: - Remove unused imports (journal.py, migrations/env.py, test_converters.py) - Convert lambda to named function (engine.py, Ruff E731) - Remove unused logger definitions in repos (Ruff F841) - Add logging to JSONL decode errors and empty except blocks - Separate assert side-effects in tests (CodeQL) - Remove unused local variables in tests (Ruff F841) - Fix max_trace_content truncation to use byte length, not char length Co-Authored-By: Claude Opus 4.6 (1M context) * style: apply ruff format to persistence and runtime files Co-Authored-By: Claude Opus 4.6 (1M context) * Potential fix for pull request finding 'Statement has no effect' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * refactor(runtime): introduce RunContext to reduce run_agent parameter bloat Extract checkpointer, store, event_store, run_events_config, thread_meta_repo, and follow_up_to_run_id into a frozen RunContext dataclass. Add get_run_context() in deps.py to build the base context from app.state singletons. start_run() uses dataclasses.replace() to enrich per-run fields before passing ctx to run_agent. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(gateway): move sanitize_log_param to app/gateway/utils.py Extract the log-injection sanitizer from routers/threads.py into a shared utils module and rename to sanitize_log_param (public API). Eliminates the reverse service → router import in services.py. Co-Authored-By: Claude Opus 4.6 (1M context) * perf: use SQL aggregation for feedback stats and thread token usage Replace Python-side counting in FeedbackRepository.aggregate_by_run with a single SELECT COUNT/SUM query. Add RunStore.aggregate_tokens_by_thread abstract method with SQL GROUP BY implementation in RunRepository and Python fallback in MemoryRunStore. Simplify the thread_token_usage endpoint to delegate to the new method, eliminating the limit=10000 truncation risk. Co-Authored-By: Claude Opus 4.6 (1M context) * docs: annotate DbRunEventStore.put() as low-frequency path Add docstring clarifying that put() opens a per-call transaction with FOR UPDATE and should only be used for infrequent writes (currently just the initial human_message event). High-throughput callers should use put_batch() instead. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(threads): fall back to Store search when ThreadMetaRepository is unavailable When database.backend=memory (default) or no SQL session factory is configured, search_threads now queries the LangGraph Store instead of returning 503. Returns empty list if neither Store nor repo is available. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(persistence): introduce ThreadMetaStore ABC for backend-agnostic thread metadata Add ThreadMetaStore abstract base class with create/get/search/update/delete interface. ThreadMetaRepository (SQL) now inherits from it. New MemoryThreadMetaStore wraps LangGraph BaseStore for memory-mode deployments. deps.py now always provides a non-None thread_meta_repo, eliminating all `if thread_meta_repo is not None` guards in services.py, worker.py, and routers/threads.py. search_threads no longer needs a Store fallback branch. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(history): read messages from checkpointer instead of RunEventStore The /history endpoint now reads messages directly from the checkpointer's channel_values (the authoritative source) instead of querying RunEventStore.list_messages(). The RunEventStore API is preserved for other consumers. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(persistence): address new Copilot review comments - feedback.py: validate thread_id/run_id before deleting feedback - jsonl.py: add path traversal protection with ID validation - run_repo.py: parse `before` to datetime for PostgreSQL compat - thread_meta_repo.py: fix pagination when metadata filter is active - database_config.py: use resolve_path for sqlite_dir consistency Co-Authored-By: Claude Opus 4.6 (1M context) * Implement skill self-evolution and skill_manage flow (#1874) * chore: ignore .worktrees directory * Add skill_manage self-evolution flow * Fix CI regressions for skill_manage * Address PR review feedback for skill evolution * fix(skill-evolution): preserve history on delete * fix(skill-evolution): tighten scanner fallbacks * docs: add skill_manage e2e evidence screenshot * fix(skill-manage): avoid blocking fs ops in session runtime --------- Co-authored-by: Willem Jiang * fix(config): resolve sqlite_dir relative to CWD, not Paths.base_dir resolve_path() resolves relative to Paths.base_dir (.deer-flow), which double-nested the path to .deer-flow/.deer-flow/data/app.db. Use Path.resolve() (CWD-relative) instead. Co-Authored-By: Claude Opus 4.6 (1M context) * Feature/feishu receive file (#1608) * feat(feishu): add channel file materialization hook for inbound messages - Introduce Channel.receive_file(msg, thread_id) as a base method for file materialization; default is no-op. - Implement FeishuChannel.receive_file to download files/images from Feishu messages, save to sandbox, and inject virtual paths into msg.text. - Update ChannelManager to call receive_file for any channel if msg.files is present, enabling downstream model access to user-uploaded files. - No impact on Slack/Telegram or other channels (they inherit the default no-op). * style(backend): format code with ruff for lint compliance - Auto-formatted packages/harness/deerflow/agents/factory.py and tests/test_create_deerflow_agent.py using `ruff format` - Ensured both files conform to project linting standards - Fixes CI lint check failures caused by code style issues * fix(feishu): handle file write operation asynchronously to prevent blocking * fix(feishu): rename GetMessageResourceRequest to _GetMessageResourceRequest and remove redundant code * test(feishu): add tests for receive_file method and placeholder replacement * fix(manager): remove unnecessary type casting for channel retrieval * fix(feishu): update logging messages to reflect resource handling instead of image * fix(feishu): sanitize filename by replacing invalid characters in file uploads * fix(feishu): improve filename sanitization and reorder image key handling in message processing * fix(feishu): add thread lock to prevent filename conflicts during file downloads * fix(test): correct bad merge in test_feishu_parser.py * chore: run ruff and apply formatting cleanup fix(feishu): preserve rich-text attachment order and improve fallback filename handling * fix(docker): restore gateway env vars and fix langgraph empty arg issue (#1915) Two production docker-compose.yaml bugs prevent `make up` from working: 1. Gateway missing DEER_FLOW_CONFIG_PATH and DEER_FLOW_EXTENSIONS_CONFIG_PATH environment overrides. Added in fb2d99f (#1836) but accidentally reverted by ca2fb95 (#1847). Without them, gateway reads host paths from .env via env_file, causing FileNotFoundError inside the container. 2. Langgraph command fails when LANGGRAPH_ALLOW_BLOCKING is unset (default). Empty $${allow_blocking} inserts a bare space between flags, causing ' --no-reload' to be parsed as unexpected extra argument. Fix by building args string first and conditionally appending --allow-blocking. Co-authored-by: cooper * fix(frontend): resolve invalid HTML nesting and tabnabbing vulnerabilities (#1904) * fix(frontend): resolve invalid HTML nesting and tabnabbing vulnerabilities Fix ` + + +
+ +
+ +
+ + ← Back to home + +
+ + + ); +} diff --git a/frontend/src/app/(auth)/setup/page.tsx b/frontend/src/app/(auth)/setup/page.tsx new file mode 100644 index 000000000..e70d1efc6 --- /dev/null +++ b/frontend/src/app/(auth)/setup/page.tsx @@ -0,0 +1,115 @@ +"use client"; + +import { useRouter } from "next/navigation"; +import { useState } from "react"; + +import { Button } from "@/components/ui/button"; +import { Input } from "@/components/ui/input"; +import { getCsrfHeaders } from "@/core/api/fetcher"; +import { parseAuthError } from "@/core/auth/types"; + +export default function SetupPage() { + const router = useRouter(); + const [email, setEmail] = useState(""); + const [newPassword, setNewPassword] = useState(""); + const [confirmPassword, setConfirmPassword] = useState(""); + const [currentPassword, setCurrentPassword] = useState(""); + const [error, setError] = useState(""); + const [loading, setLoading] = useState(false); + + const handleSetup = async (e: React.FormEvent) => { + e.preventDefault(); + setError(""); + + if (newPassword !== confirmPassword) { + setError("Passwords do not match"); + return; + } + if (newPassword.length < 8) { + setError("Password must be at least 8 characters"); + return; + } + + setLoading(true); + try { + const res = await fetch("/api/v1/auth/change-password", { + method: "POST", + headers: { + "Content-Type": "application/json", + ...getCsrfHeaders(), + }, + credentials: "include", + body: JSON.stringify({ + current_password: currentPassword, + new_password: newPassword, + new_email: email || undefined, + }), + }); + + if (!res.ok) { + const data = await res.json(); + const authError = parseAuthError(data); + setError(authError.message); + return; + } + + router.push("/workspace"); + } catch { + setError("Network error. Please try again."); + } finally { + setLoading(false); + } + }; + + return ( +
+
+
+

DeerFlow

+

+ Complete admin account setup +

+

+ Set your real email and a new password. +

+
+
+ setEmail(e.target.value)} + required + /> + setCurrentPassword(e.target.value)} + required + /> + setNewPassword(e.target.value)} + required + minLength={8} + /> + setConfirmPassword(e.target.value)} + required + minLength={8} + /> + {error &&

{error}

} + +
+
+
+ ); +} diff --git a/frontend/src/app/api/auth/[...all]/route.ts b/frontend/src/app/api/auth/[...all]/route.ts deleted file mode 100644 index cde6018a8..000000000 --- a/frontend/src/app/api/auth/[...all]/route.ts +++ /dev/null @@ -1,5 +0,0 @@ -import { toNextJsHandler } from "better-auth/next-js"; - -import { auth } from "@/server/better-auth"; - -export const { GET, POST } = toNextJsHandler(auth.handler); diff --git a/frontend/src/app/workspace/layout.tsx b/frontend/src/app/workspace/layout.tsx index 4c1dd2036..fa19025a0 100644 --- a/frontend/src/app/workspace/layout.tsx +++ b/frontend/src/app/workspace/layout.tsx @@ -1,35 +1,58 @@ -import { cookies } from "next/headers"; -import { Toaster } from "sonner"; +import Link from "next/link"; +import { redirect } from "next/navigation"; -import { QueryClientProvider } from "@/components/query-client-provider"; -import { SidebarInset, SidebarProvider } from "@/components/ui/sidebar"; -import { CommandPalette } from "@/components/workspace/command-palette"; -import { WorkspaceSidebar } from "@/components/workspace/workspace-sidebar"; +import { AuthProvider } from "@/core/auth/AuthProvider"; +import { getServerSideUser } from "@/core/auth/server"; +import { assertNever } from "@/core/auth/types"; -function parseSidebarOpenCookie( - value: string | undefined, -): boolean | undefined { - if (value === "true") return true; - if (value === "false") return false; - return undefined; -} +import { WorkspaceContent } from "./workspace-content"; + +export const dynamic = "force-dynamic"; export default async function WorkspaceLayout({ children, }: Readonly<{ children: React.ReactNode }>) { - const cookieStore = await cookies(); - const initialSidebarOpen = parseSidebarOpenCookie( - cookieStore.get("sidebar_state")?.value, - ); + const result = await getServerSideUser(); - return ( - - - - {children} - - - - - ); + switch (result.tag) { + case "authenticated": + return ( + + {children} + + ); + case "needs_setup": + redirect("/setup"); + case "unauthenticated": + redirect("/login"); + case "gateway_unavailable": + return ( +
+

+ Service temporarily unavailable. +

+

+ The backend may be restarting. Please wait a moment and try again. +

+
+ + Retry + + + Logout & Reset + +
+
+ ); + case "config_error": + throw new Error(result.message); + default: + assertNever(result); + } } diff --git a/frontend/src/app/workspace/workspace-content.tsx b/frontend/src/app/workspace/workspace-content.tsx new file mode 100644 index 000000000..85c20b2ca --- /dev/null +++ b/frontend/src/app/workspace/workspace-content.tsx @@ -0,0 +1,35 @@ +import { cookies } from "next/headers"; +import { Toaster } from "sonner"; + +import { QueryClientProvider } from "@/components/query-client-provider"; +import { SidebarInset, SidebarProvider } from "@/components/ui/sidebar"; +import { CommandPalette } from "@/components/workspace/command-palette"; +import { WorkspaceSidebar } from "@/components/workspace/workspace-sidebar"; + +function parseSidebarOpenCookie( + value: string | undefined, +): boolean | undefined { + if (value === "true") return true; + if (value === "false") return false; + return undefined; +} + +export async function WorkspaceContent({ + children, +}: Readonly<{ children: React.ReactNode }>) { + const cookieStore = await cookies(); + const initialSidebarOpen = parseSidebarOpenCookie( + cookieStore.get("sidebar_state")?.value, + ); + + return ( + + + + {children} + + + + + ); +} diff --git a/frontend/src/components/workspace/input-box.tsx b/frontend/src/components/workspace/input-box.tsx index ddc682744..83bd75952 100644 --- a/frontend/src/components/workspace/input-box.tsx +++ b/frontend/src/components/workspace/input-box.tsx @@ -55,6 +55,7 @@ import { DropdownMenuLabel, DropdownMenuSeparator, } from "@/components/ui/dropdown-menu"; +import { fetchWithAuth } from "@/core/api/fetcher"; import { getBackendBaseURL } from "@/core/config"; import { useI18n } from "@/core/i18n/hooks"; import { useModels } from "@/core/models/hooks"; @@ -395,16 +396,19 @@ export function InputBox({ setFollowupsLoading(true); setFollowups([]); - fetch(`${getBackendBaseURL()}/api/threads/${threadId}/suggestions`, { - method: "POST", - headers: { "Content-Type": "application/json" }, - body: JSON.stringify({ - messages: recent, - n: 3, - model_name: context.model_name ?? undefined, - }), - signal: controller.signal, - }) + fetchWithAuth( + `${getBackendBaseURL()}/api/threads/${threadId}/suggestions`, + { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + messages: recent, + n: 3, + model_name: context.model_name ?? undefined, + }), + signal: controller.signal, + }, + ) .then(async (res) => { if (!res.ok) { return { suggestions: [] as string[] }; diff --git a/frontend/src/components/workspace/settings/account-settings-page.tsx b/frontend/src/components/workspace/settings/account-settings-page.tsx new file mode 100644 index 000000000..6382b8859 --- /dev/null +++ b/frontend/src/components/workspace/settings/account-settings-page.tsx @@ -0,0 +1,132 @@ +"use client"; + +import { LogOutIcon } from "lucide-react"; +import { useState } from "react"; + +import { Button } from "@/components/ui/button"; +import { Input } from "@/components/ui/input"; +import { fetchWithAuth, getCsrfHeaders } from "@/core/api/fetcher"; +import { useAuth } from "@/core/auth/AuthProvider"; +import { parseAuthError } from "@/core/auth/types"; + +import { SettingsSection } from "./settings-section"; + +export function AccountSettingsPage() { + const { user, logout } = useAuth(); + const [currentPassword, setCurrentPassword] = useState(""); + const [newPassword, setNewPassword] = useState(""); + const [confirmPassword, setConfirmPassword] = useState(""); + const [message, setMessage] = useState(""); + const [error, setError] = useState(""); + const [loading, setLoading] = useState(false); + + const handleChangePassword = async (e: React.FormEvent) => { + e.preventDefault(); + setError(""); + setMessage(""); + + if (newPassword !== confirmPassword) { + setError("New passwords do not match"); + return; + } + if (newPassword.length < 8) { + setError("Password must be at least 8 characters"); + return; + } + + setLoading(true); + try { + const res = await fetchWithAuth("/api/v1/auth/change-password", { + method: "POST", + headers: { + "Content-Type": "application/json", + ...getCsrfHeaders(), + }, + body: JSON.stringify({ + current_password: currentPassword, + new_password: newPassword, + }), + }); + + if (!res.ok) { + const data = await res.json(); + const authError = parseAuthError(data); + setError(authError.message); + return; + } + + setMessage("Password changed successfully"); + setCurrentPassword(""); + setNewPassword(""); + setConfirmPassword(""); + } catch { + setError("Network error. Please try again."); + } finally { + setLoading(false); + } + }; + + return ( +
+ +
+
+ Email + {user?.email ?? "—"} +
+
+ Role + + {user?.system_role ?? "—"} + +
+
+ + + +
+ setCurrentPassword(e.target.value)} + required + /> + setNewPassword(e.target.value)} + required + minLength={8} + /> + setConfirmPassword(e.target.value)} + required + minLength={8} + /> + {error &&

{error}

} + {message &&

{message}

} + +
+ + + + + +
+ ); +} diff --git a/frontend/src/components/workspace/settings/settings-dialog.tsx b/frontend/src/components/workspace/settings/settings-dialog.tsx index fadc25fa6..6e9fa5ddf 100644 --- a/frontend/src/components/workspace/settings/settings-dialog.tsx +++ b/frontend/src/components/workspace/settings/settings-dialog.tsx @@ -6,6 +6,7 @@ import { BrainIcon, PaletteIcon, SparklesIcon, + UserIcon, WrenchIcon, } from "lucide-react"; import { useEffect, useMemo, useState } from "react"; @@ -18,6 +19,7 @@ import { } from "@/components/ui/dialog"; import { ScrollArea } from "@/components/ui/scroll-area"; import { AboutSettingsPage } from "@/components/workspace/settings/about-settings-page"; +import { AccountSettingsPage } from "@/components/workspace/settings/account-settings-page"; import { AppearanceSettingsPage } from "@/components/workspace/settings/appearance-settings-page"; import { MemorySettingsPage } from "@/components/workspace/settings/memory-settings-page"; import { NotificationSettingsPage } from "@/components/workspace/settings/notification-settings-page"; @@ -27,6 +29,7 @@ import { useI18n } from "@/core/i18n/hooks"; import { cn } from "@/lib/utils"; type SettingsSection = + | "account" | "appearance" | "memory" | "tools" @@ -54,6 +57,11 @@ export function SettingsDialog(props: SettingsDialogProps) { const sections = useMemo( () => [ + { + id: "account", + label: t.settings.sections.account, + icon: UserIcon, + }, { id: "appearance", label: t.settings.sections.appearance, @@ -74,6 +82,7 @@ export function SettingsDialog(props: SettingsDialogProps) { { id: "about", label: t.settings.sections.about, icon: InfoIcon }, ], [ + t.settings.sections.account, t.settings.sections.appearance, t.settings.sections.memory, t.settings.sections.tools, @@ -122,8 +131,9 @@ export function SettingsDialog(props: SettingsDialogProps) { })} - -
+ +
+ {activeSection === "account" && } {activeSection === "appearance" && } {activeSection === "memory" && } {activeSection === "tools" && } diff --git a/frontend/src/core/agents/api.ts b/frontend/src/core/agents/api.ts index 927b5f20b..984ee8832 100644 --- a/frontend/src/core/agents/api.ts +++ b/frontend/src/core/agents/api.ts @@ -1,3 +1,4 @@ +import { fetchWithAuth } from "@/core/api/fetcher"; import { getBackendBaseURL } from "@/core/config"; import type { Agent, CreateAgentRequest, UpdateAgentRequest } from "./types"; @@ -28,7 +29,7 @@ export async function getAgent(name: string): Promise { } export async function createAgent(request: CreateAgentRequest): Promise { - const res = await fetch(`${getBackendBaseURL()}/api/agents`, { + const res = await fetchWithAuth(`${getBackendBaseURL()}/api/agents`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(request), @@ -44,7 +45,7 @@ export async function updateAgent( name: string, request: UpdateAgentRequest, ): Promise { - const res = await fetch(`${getBackendBaseURL()}/api/agents/${name}`, { + const res = await fetchWithAuth(`${getBackendBaseURL()}/api/agents/${name}`, { method: "PUT", headers: { "Content-Type": "application/json" }, body: JSON.stringify(request), @@ -57,7 +58,7 @@ export async function updateAgent( } export async function deleteAgent(name: string): Promise { - const res = await fetch(`${getBackendBaseURL()}/api/agents/${name}`, { + const res = await fetchWithAuth(`${getBackendBaseURL()}/api/agents/${name}`, { method: "DELETE", }); if (!res.ok) throw new Error(`Failed to delete agent: ${res.statusText}`); diff --git a/frontend/src/core/api/api-client.ts b/frontend/src/core/api/api-client.ts index b86251513..5e71730e7 100644 --- a/frontend/src/core/api/api-client.ts +++ b/frontend/src/core/api/api-client.ts @@ -4,11 +4,37 @@ import { Client as LangGraphClient } from "@langchain/langgraph-sdk/client"; import { getLangGraphBaseURL } from "../config"; +import { isStateChangingMethod, readCsrfCookie } from "./fetcher"; import { sanitizeRunStreamOptions } from "./stream-mode"; +/** + * SDK ``onRequest`` hook that mints the ``X-CSRF-Token`` header from the + * live ``csrf_token`` cookie just before each outbound fetch. + * + * Reading the cookie per-request (rather than baking it into the SDK's + * ``defaultHeaders`` at construction) handles login / logout / password + * change cookie rotation transparently. Both the ``/langgraph-compat/*`` + * SDK path and the direct REST endpoints in ``fetcher.ts:fetchWithAuth`` + * share :func:`readCsrfCookie` and :const:`STATE_CHANGING_METHODS` so + * the contract stays in lockstep. + */ +function injectCsrfHeader(_url: URL, init: RequestInit): RequestInit { + if (!isStateChangingMethod(init.method ?? "GET")) { + return init; + } + const token = readCsrfCookie(); + if (!token) return init; + const headers = new Headers(init.headers); + if (!headers.has("X-CSRF-Token")) { + headers.set("X-CSRF-Token", token); + } + return { ...init, headers }; +} + function createCompatibleClient(isMock?: boolean): LangGraphClient { const client = new LangGraphClient({ apiUrl: getLangGraphBaseURL(isMock), + onRequest: injectCsrfHeader, }); const originalRunStream = client.runs.stream.bind(client.runs); diff --git a/frontend/src/core/api/fetcher.ts b/frontend/src/core/api/fetcher.ts new file mode 100644 index 000000000..d5f30404c --- /dev/null +++ b/frontend/src/core/api/fetcher.ts @@ -0,0 +1,104 @@ +import { buildLoginUrl } from "@/core/auth/types"; + +/** HTTP methods that the gateway's CSRFMiddleware checks. */ +export type StateChangingMethod = "POST" | "PUT" | "DELETE" | "PATCH"; + +export const STATE_CHANGING_METHODS: ReadonlySet = new Set( + ["POST", "PUT", "DELETE", "PATCH"], +); + +/** Mirror of the gateway's ``should_check_csrf`` decision. */ +export function isStateChangingMethod(method: string): boolean { + return (STATE_CHANGING_METHODS as ReadonlySet).has( + method.toUpperCase(), + ); +} + +const CSRF_COOKIE_PREFIX = "csrf_token="; + +/** + * Read the ``csrf_token`` cookie set by the gateway at login. + * + * SSR-safe: returns ``null`` when ``document`` is undefined so the same + * helper can be imported from server components without a guard. + * + * Uses `String.split` instead of a regex to side-step ESLint's + * `prefer-regexp-exec` rule and the cookie value's reliable `; ` + * separator (set by the gateway, not the browser, so format is stable). + */ +export function readCsrfCookie(): string | null { + if (typeof document === "undefined") return null; + for (const pair of document.cookie.split("; ")) { + if (pair.startsWith(CSRF_COOKIE_PREFIX)) { + return decodeURIComponent(pair.slice(CSRF_COOKIE_PREFIX.length)); + } + } + return null; +} + +/** + * Fetch with credentials and automatic CSRF protection. + * + * Two centralized contracts every API call needs: + * + * 1. ``credentials: "include"`` so the HttpOnly access_token cookie + * accompanies cross-origin SSR-routed requests. + * 2. ``X-CSRF-Token`` header on state-changing methods (POST/PUT/ + * DELETE/PATCH), echoed from the ``csrf_token`` cookie. The gateway's + * CSRFMiddleware enforces Double Submit Cookie comparison and returns + * 403 if the header is missing — silently breaking every call site + * that uses raw ``fetch()`` instead of this wrapper. + * + * Auto-redirects to ``/login`` on 401. Caller-supplied headers are + * preserved; the helper only ADDS the CSRF header when it isn't already + * present, so explicit overrides win. + */ +export async function fetchWithAuth( + input: RequestInfo | string, + init?: RequestInit, +): Promise { + const url = typeof input === "string" ? input : input.url; + + // Inject CSRF for state-changing methods. GET/HEAD/OPTIONS/TRACE skip + // it to mirror the gateway's ``should_check_csrf`` logic exactly. + let headers = init?.headers; + if (isStateChangingMethod(init?.method ?? "GET")) { + const token = readCsrfCookie(); + if (token) { + // Fresh Headers instance so we don't mutate caller-supplied objects. + const merged = new Headers(headers); + if (!merged.has("X-CSRF-Token")) { + merged.set("X-CSRF-Token", token); + } + headers = merged; + } + } + + const res = await fetch(url, { + ...init, + headers, + credentials: "include", + }); + + if (res.status === 401) { + window.location.href = buildLoginUrl(window.location.pathname); + throw new Error("Unauthorized"); + } + + return res; +} + +/** + * Build headers for CSRF-protected requests. + * + * **Prefer :func:`fetchWithAuth`** for new code — it injects the header + * automatically on state-changing methods. This helper exists for legacy + * call sites that need to compose headers manually (e.g. inside + * `next/server` route handlers that build their own ``Headers`` object). + * + * Per RFC-001: Double Submit Cookie pattern. + */ +export function getCsrfHeaders(): HeadersInit { + const token = readCsrfCookie(); + return token ? { "X-CSRF-Token": token } : {}; +} diff --git a/frontend/src/core/auth/AuthProvider.tsx b/frontend/src/core/auth/AuthProvider.tsx new file mode 100644 index 000000000..652cc49b8 --- /dev/null +++ b/frontend/src/core/auth/AuthProvider.tsx @@ -0,0 +1,165 @@ +"use client"; + +import { useRouter, usePathname } from "next/navigation"; +import React, { + createContext, + useContext, + useState, + useCallback, + useEffect, + type ReactNode, +} from "react"; + +import { type User, buildLoginUrl } from "./types"; + +// Re-export for consumers +export type { User }; + +/** + * Authentication context provided to consuming components + */ +interface AuthContextType { + user: User | null; + isAuthenticated: boolean; + isLoading: boolean; + logout: () => Promise; + refreshUser: () => Promise; +} + +const AuthContext = createContext(undefined); + +interface AuthProviderProps { + children: ReactNode; + initialUser: User | null; +} + +/** + * AuthProvider - Unified authentication context for the application + * + * Per RFC-001: + * - Only holds display information (user), never JWT or tokens + * - initialUser comes from server-side guard, avoiding client flicker + * - Provides logout and refresh capabilities + */ +export function AuthProvider({ children, initialUser }: AuthProviderProps) { + const [user, setUser] = useState(initialUser); + const [isLoading, setIsLoading] = useState(false); + const router = useRouter(); + const pathname = usePathname(); + + const isAuthenticated = user !== null; + + /** + * Fetch current user from FastAPI + * Used when initialUser might be stale (e.g., after tab was inactive) + */ + const refreshUser = useCallback(async () => { + try { + setIsLoading(true); + const res = await fetch("/api/v1/auth/me", { + credentials: "include", + }); + + if (res.ok) { + const data = await res.json(); + setUser(data); + } else if (res.status === 401) { + // Session expired or invalid + setUser(null); + // Redirect to login if on a protected route + if (pathname?.startsWith("/workspace")) { + router.push(buildLoginUrl(pathname)); + } + } + } catch (err) { + console.error("Failed to refresh user:", err); + setUser(null); + } finally { + setIsLoading(false); + } + }, [pathname, router]); + + /** + * Logout - call FastAPI logout endpoint and clear local state + * Per RFC-001: Immediately clear local state, don't wait for server confirmation + */ + const logout = useCallback(async () => { + // Immediately clear local state to prevent UI flicker + setUser(null); + + try { + await fetch("/api/v1/auth/logout", { + method: "POST", + credentials: "include", + }); + } catch (err) { + console.error("Logout request failed:", err); + // Still redirect even if logout request fails + } + + // Redirect to home page + router.push("/"); + }, [router]); + + /** + * Handle visibility change - refresh user when tab becomes visible again. + * Throttled to at most once per 60 s to avoid spamming the backend on rapid tab switches. + */ + const lastCheckRef = React.useRef(0); + + useEffect(() => { + const handleVisibilityChange = () => { + if (document.visibilityState !== "visible" || user === null) return; + const now = Date.now(); + if (now - lastCheckRef.current < 60_000) return; + lastCheckRef.current = now; + void refreshUser(); + }; + + document.addEventListener("visibilitychange", handleVisibilityChange); + return () => { + document.removeEventListener("visibilitychange", handleVisibilityChange); + }; + }, [user, refreshUser]); + + const value: AuthContextType = { + user, + isAuthenticated, + isLoading, + logout, + refreshUser, + }; + + return {children}; +} + +/** + * Hook to access authentication context + * Throws if used outside AuthProvider - this is intentional for proper usage + */ +export function useAuth(): AuthContextType { + const context = useContext(AuthContext); + if (context === undefined) { + throw new Error("useAuth must be used within an AuthProvider"); + } + return context; +} + +/** + * Hook to require authentication - redirects to login if not authenticated + * Useful for client-side checks in addition to server-side guards + */ +export function useRequireAuth(): AuthContextType { + const auth = useAuth(); + const router = useRouter(); + const pathname = usePathname(); + + useEffect(() => { + // Only redirect if we're sure user is not authenticated (not just loading) + if (!auth.isLoading && !auth.isAuthenticated) { + router.push(buildLoginUrl(pathname || "/workspace")); + } + }, [auth.isAuthenticated, auth.isLoading, router, pathname]); + + return auth; +} diff --git a/frontend/src/core/auth/gateway-config.ts b/frontend/src/core/auth/gateway-config.ts new file mode 100644 index 000000000..61c6ae850 --- /dev/null +++ b/frontend/src/core/auth/gateway-config.ts @@ -0,0 +1,34 @@ +import { z } from "zod"; + +const gatewayConfigSchema = z.object({ + internalGatewayUrl: z.string().url(), + trustedOrigins: z.array(z.string()).min(1), +}); + +export type GatewayConfig = z.infer; + +let _cached: GatewayConfig | null = null; + +export function getGatewayConfig(): GatewayConfig { + if (_cached) return _cached; + + const isDev = process.env.NODE_ENV === "development"; + + const rawUrl = process.env.DEER_FLOW_INTERNAL_GATEWAY_BASE_URL?.trim(); + const internalGatewayUrl = + rawUrl?.replace(/\/+$/, "") ?? + (isDev ? "http://localhost:8001" : undefined); + + const rawOrigins = process.env.DEER_FLOW_TRUSTED_ORIGINS?.trim(); + const trustedOrigins = rawOrigins + ? rawOrigins + .split(",") + .map((s) => s.trim()) + .filter(Boolean) + : isDev + ? ["http://localhost:3000"] + : undefined; + + _cached = gatewayConfigSchema.parse({ internalGatewayUrl, trustedOrigins }); + return _cached; +} diff --git a/frontend/src/core/auth/proxy-policy.ts b/frontend/src/core/auth/proxy-policy.ts new file mode 100644 index 000000000..9e6f1f424 --- /dev/null +++ b/frontend/src/core/auth/proxy-policy.ts @@ -0,0 +1,55 @@ +export interface ProxyPolicy { + /** Allowed upstream path prefixes */ + readonly allowedPaths: readonly string[]; + /** Request headers to strip before forwarding */ + readonly strippedRequestHeaders: ReadonlySet; + /** Response headers to strip before returning */ + readonly strippedResponseHeaders: ReadonlySet; + /** Credential mode: which cookie to forward */ + readonly credential: { readonly type: "cookie"; readonly name: string }; + /** Timeout in ms */ + readonly timeoutMs: number; + /** CSRF: required for non-GET/HEAD */ + readonly csrf: boolean; +} + +export const LANGGRAPH_COMPAT_POLICY: ProxyPolicy = { + allowedPaths: [ + "threads", + "runs", + "assistants", + "store", + "models", + "mcp", + "skills", + "memory", + ], + strippedRequestHeaders: new Set([ + "host", + "connection", + "keep-alive", + "transfer-encoding", + "te", + "trailer", + "upgrade", + "authorization", + "x-api-key", + "origin", + "referer", + "proxy-authorization", + "proxy-authenticate", + ]), + strippedResponseHeaders: new Set([ + "connection", + "keep-alive", + "transfer-encoding", + "te", + "trailer", + "upgrade", + "content-length", + "set-cookie", + ]), + credential: { type: "cookie", name: "access_token" }, + timeoutMs: 120_000, + csrf: true, +}; diff --git a/frontend/src/core/auth/server.ts b/frontend/src/core/auth/server.ts new file mode 100644 index 000000000..4229143aa --- /dev/null +++ b/frontend/src/core/auth/server.ts @@ -0,0 +1,57 @@ +import { cookies } from "next/headers"; + +import { getGatewayConfig } from "./gateway-config"; +import { type AuthResult, userSchema } from "./types"; + +const SSR_AUTH_TIMEOUT_MS = 5_000; + +/** + * Fetch the authenticated user from the gateway using the request's cookies. + * Returns a tagged AuthResult — callers use exhaustive switch, no try/catch. + */ +export async function getServerSideUser(): Promise { + const cookieStore = await cookies(); + const sessionCookie = cookieStore.get("access_token"); + + let internalGatewayUrl: string; + try { + internalGatewayUrl = getGatewayConfig().internalGatewayUrl; + } catch (err) { + return { tag: "config_error", message: String(err) }; + } + + if (!sessionCookie) return { tag: "unauthenticated" }; + + const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), SSR_AUTH_TIMEOUT_MS); + + try { + const res = await fetch(`${internalGatewayUrl}/api/v1/auth/me`, { + headers: { Cookie: `access_token=${sessionCookie.value}` }, + cache: "no-store", + signal: controller.signal, + }); + clearTimeout(timeout); // Clear immediately — covers all response branches + + if (res.ok) { + const parsed = userSchema.safeParse(await res.json()); + if (!parsed.success) { + console.error("[SSR auth] Malformed /auth/me response:", parsed.error); + return { tag: "gateway_unavailable" }; + } + if (parsed.data.needs_setup) { + return { tag: "needs_setup", user: parsed.data }; + } + return { tag: "authenticated", user: parsed.data }; + } + if (res.status === 401 || res.status === 403) { + return { tag: "unauthenticated" }; + } + console.error(`[SSR auth] /api/v1/auth/me responded ${res.status}`); + return { tag: "gateway_unavailable" }; + } catch (err) { + clearTimeout(timeout); + console.error("[SSR auth] Failed to reach gateway:", err); + return { tag: "gateway_unavailable" }; + } +} diff --git a/frontend/src/core/auth/types.ts b/frontend/src/core/auth/types.ts new file mode 100644 index 000000000..4cf42583e --- /dev/null +++ b/frontend/src/core/auth/types.ts @@ -0,0 +1,72 @@ +import { z } from "zod"; + +// ── User schema (single source of truth) ────────────────────────── + +export const userSchema = z.object({ + id: z.string(), + email: z.string().email(), + system_role: z.enum(["admin", "user"]), + needs_setup: z.boolean().optional().default(false), +}); + +export type User = z.infer; + +// ── SSR auth result (tagged union) ──────────────────────────────── + +export type AuthResult = + | { tag: "authenticated"; user: User } + | { tag: "needs_setup"; user: User } + | { tag: "unauthenticated" } + | { tag: "gateway_unavailable" } + | { tag: "config_error"; message: string }; + +export function assertNever(x: never): never { + throw new Error(`Unexpected auth result: ${JSON.stringify(x)}`); +} + +export function buildLoginUrl(returnPath: string): string { + return `/login?next=${encodeURIComponent(returnPath)}`; +} + +// ── Backend error response parsing ──────────────────────────────── + +const AUTH_ERROR_CODES = [ + "invalid_credentials", + "token_expired", + "token_invalid", + "user_not_found", + "email_already_exists", + "provider_not_found", + "not_authenticated", +] as const; + +export type AuthErrorCode = (typeof AUTH_ERROR_CODES)[number]; + +export interface AuthErrorResponse { + code: AuthErrorCode; + message: string; +} + +const authErrorSchema = z.object({ + code: z.enum(AUTH_ERROR_CODES), + message: z.string(), +}); + +export function parseAuthError(data: unknown): AuthErrorResponse { + // Try top-level {code, message} first + const parsed = authErrorSchema.safeParse(data); + if (parsed.success) return parsed.data; + + // Unwrap FastAPI's {detail: {code, message}} envelope + if (typeof data === "object" && data !== null && "detail" in data) { + const detail = (data as Record).detail; + const nested = authErrorSchema.safeParse(detail); + if (nested.success) return nested.data; + // Legacy string-detail responses + if (typeof detail === "string") { + return { code: "invalid_credentials", message: detail }; + } + } + + return { code: "invalid_credentials", message: "Authentication failed" }; +} diff --git a/frontend/src/core/i18n/locales/en-US.ts b/frontend/src/core/i18n/locales/en-US.ts index de94e0c98..7de0d78ec 100644 --- a/frontend/src/core/i18n/locales/en-US.ts +++ b/frontend/src/core/i18n/locales/en-US.ts @@ -236,6 +236,7 @@ export const enUS: Translations = { reportIssue: "Report a issue", contactUs: "Contact us", about: "About DeerFlow", + logout: "Log out", }, // Conversation @@ -324,6 +325,7 @@ export const enUS: Translations = { title: "Settings", description: "Adjust how DeerFlow looks and behaves for you.", sections: { + account: "Account", appearance: "Appearance", memory: "Memory", tools: "Tools", diff --git a/frontend/src/core/i18n/locales/types.ts b/frontend/src/core/i18n/locales/types.ts index a8d99e4c7..e42c681f2 100644 --- a/frontend/src/core/i18n/locales/types.ts +++ b/frontend/src/core/i18n/locales/types.ts @@ -168,6 +168,7 @@ export interface Translations { reportIssue: string; contactUs: string; about: string; + logout: string; }; // Conversation @@ -253,6 +254,7 @@ export interface Translations { title: string; description: string; sections: { + account: string; appearance: string; memory: string; tools: string; diff --git a/frontend/src/core/i18n/locales/zh-CN.ts b/frontend/src/core/i18n/locales/zh-CN.ts index 600cb8f07..b6416c288 100644 --- a/frontend/src/core/i18n/locales/zh-CN.ts +++ b/frontend/src/core/i18n/locales/zh-CN.ts @@ -224,6 +224,7 @@ export const zhCN: Translations = { reportIssue: "报告问题", contactUs: "联系我们", about: "关于 DeerFlow", + logout: "退出登录", }, // Conversation @@ -309,6 +310,7 @@ export const zhCN: Translations = { title: "设置", description: "根据你的偏好调整 DeerFlow 的界面和行为。", sections: { + account: "账号", appearance: "外观", memory: "记忆", tools: "工具", diff --git a/frontend/src/core/mcp/api.ts b/frontend/src/core/mcp/api.ts index 003303238..61e681d34 100644 --- a/frontend/src/core/mcp/api.ts +++ b/frontend/src/core/mcp/api.ts @@ -1,3 +1,4 @@ +import { fetchWithAuth } from "@/core/api/fetcher"; import { getBackendBaseURL } from "@/core/config"; import type { MCPConfig } from "./types"; @@ -8,12 +9,15 @@ export async function loadMCPConfig() { } export async function updateMCPConfig(config: MCPConfig) { - const response = await fetch(`${getBackendBaseURL()}/api/mcp/config`, { - method: "PUT", - headers: { - "Content-Type": "application/json", + const response = await fetchWithAuth( + `${getBackendBaseURL()}/api/mcp/config`, + { + method: "PUT", + headers: { + "Content-Type": "application/json", + }, + body: JSON.stringify(config), }, - body: JSON.stringify(config), - }); + ); return response.json(); } diff --git a/frontend/src/core/memory/api.ts b/frontend/src/core/memory/api.ts index 5fcf8e4c0..073328808 100644 --- a/frontend/src/core/memory/api.ts +++ b/frontend/src/core/memory/api.ts @@ -1,3 +1,4 @@ +import { fetchWithAuth } from "../api/fetcher"; import { getBackendBaseURL } from "../config"; import type { @@ -85,14 +86,14 @@ export async function loadMemory(): Promise { } export async function clearMemory(): Promise { - const response = await fetch(`${getBackendBaseURL()}/api/memory`, { + const response = await fetchWithAuth(`${getBackendBaseURL()}/api/memory`, { method: "DELETE", }); return readMemoryResponse(response, "Failed to clear memory"); } export async function deleteMemoryFact(factId: string): Promise { - const response = await fetch( + const response = await fetchWithAuth( `${getBackendBaseURL()}/api/memory/facts/${encodeURIComponent(factId)}`, { method: "DELETE", @@ -107,26 +108,32 @@ export async function exportMemory(): Promise { } export async function importMemory(memory: UserMemory): Promise { - const response = await fetch(`${getBackendBaseURL()}/api/memory/import`, { - method: "POST", - headers: { - "Content-Type": "application/json", + const response = await fetchWithAuth( + `${getBackendBaseURL()}/api/memory/import`, + { + method: "POST", + headers: { + "Content-Type": "application/json", + }, + body: JSON.stringify(memory), }, - body: JSON.stringify(memory), - }); + ); return readMemoryResponse(response, "Failed to import memory"); } export async function createMemoryFact( input: MemoryFactInput, ): Promise { - const response = await fetch(`${getBackendBaseURL()}/api/memory/facts`, { - method: "POST", - headers: { - "Content-Type": "application/json", + const response = await fetchWithAuth( + `${getBackendBaseURL()}/api/memory/facts`, + { + method: "POST", + headers: { + "Content-Type": "application/json", + }, + body: JSON.stringify(input), }, - body: JSON.stringify(input), - }); + ); return readMemoryResponse(response, "Failed to create memory fact"); } @@ -134,7 +141,7 @@ export async function updateMemoryFact( factId: string, input: MemoryFactPatchInput, ): Promise { - const response = await fetch( + const response = await fetchWithAuth( `${getBackendBaseURL()}/api/memory/facts/${encodeURIComponent(factId)}`, { method: "PATCH", diff --git a/frontend/src/core/skills/api.ts b/frontend/src/core/skills/api.ts index b6a358f03..03a713d92 100644 --- a/frontend/src/core/skills/api.ts +++ b/frontend/src/core/skills/api.ts @@ -1,3 +1,4 @@ +import { fetchWithAuth } from "@/core/api/fetcher"; import { getBackendBaseURL } from "@/core/config"; import type { Skill } from "./type"; @@ -9,7 +10,7 @@ export async function loadSkills() { } export async function enableSkill(skillName: string, enabled: boolean) { - const response = await fetch( + const response = await fetchWithAuth( `${getBackendBaseURL()}/api/skills/${skillName}`, { method: "PUT", @@ -38,13 +39,16 @@ export interface InstallSkillResponse { export async function installSkill( request: InstallSkillRequest, ): Promise { - const response = await fetch(`${getBackendBaseURL()}/api/skills/install`, { - method: "POST", - headers: { - "Content-Type": "application/json", + const response = await fetchWithAuth( + `${getBackendBaseURL()}/api/skills/install`, + { + method: "POST", + headers: { + "Content-Type": "application/json", + }, + body: JSON.stringify(request), }, - body: JSON.stringify(request), - }); + ); if (!response.ok) { // Handle HTTP error responses (4xx, 5xx) diff --git a/frontend/src/core/threads/hooks.ts b/frontend/src/core/threads/hooks.ts index 9292ac12b..33424aeee 100644 --- a/frontend/src/core/threads/hooks.ts +++ b/frontend/src/core/threads/hooks.ts @@ -8,6 +8,7 @@ import { toast } from "sonner"; import type { PromptInputMessage } from "@/components/ai-elements/prompt-input"; import { getAPIClient } from "../api"; +import { fetchWithAuth } from "../api/fetcher"; import { getBackendBaseURL } from "../config"; import { useI18n } from "../i18n/hooks"; import type { FileInMessage } from "../messages/utils"; @@ -604,7 +605,7 @@ export function useDeleteThread() { mutationFn: async ({ threadId }: { threadId: string }) => { await apiClient.threads.delete(threadId); - const response = await fetch( + const response = await fetchWithAuth( `${getBackendBaseURL()}/api/threads/${encodeURIComponent(threadId)}`, { method: "DELETE", diff --git a/frontend/src/core/uploads/api.ts b/frontend/src/core/uploads/api.ts index 23d463c2d..a00a259cb 100644 --- a/frontend/src/core/uploads/api.ts +++ b/frontend/src/core/uploads/api.ts @@ -2,6 +2,7 @@ * API functions for file uploads */ +import { fetchWithAuth } from "../api/fetcher"; import { getBackendBaseURL } from "../config"; export interface UploadedFileInfo { @@ -50,7 +51,7 @@ export async function uploadFiles( formData.append("files", file); }); - const response = await fetch( + const response = await fetchWithAuth( `${getBackendBaseURL()}/api/threads/${threadId}/uploads`, { method: "POST", @@ -91,7 +92,7 @@ export async function deleteUploadedFile( threadId: string, filename: string, ): Promise<{ success: boolean; message: string }> { - const response = await fetch( + const response = await fetchWithAuth( `${getBackendBaseURL()}/api/threads/${threadId}/uploads/${filename}`, { method: "DELETE", diff --git a/frontend/src/env.js b/frontend/src/env.js index f00fa7a6c..ea90cac5d 100644 --- a/frontend/src/env.js +++ b/frontend/src/env.js @@ -7,12 +7,6 @@ export const env = createEnv({ * isn't built with invalid env vars. */ server: { - BETTER_AUTH_SECRET: - process.env.NODE_ENV === "production" - ? z.string() - : z.string().optional(), - BETTER_AUTH_GITHUB_CLIENT_ID: z.string().optional(), - BETTER_AUTH_GITHUB_CLIENT_SECRET: z.string().optional(), GITHUB_OAUTH_TOKEN: z.string().optional(), NODE_ENV: z .enum(["development", "test", "production"]) @@ -35,10 +29,6 @@ export const env = createEnv({ * middlewares) or client-side so we need to destruct manually. */ runtimeEnv: { - BETTER_AUTH_SECRET: process.env.BETTER_AUTH_SECRET, - BETTER_AUTH_GITHUB_CLIENT_ID: process.env.BETTER_AUTH_GITHUB_CLIENT_ID, - BETTER_AUTH_GITHUB_CLIENT_SECRET: - process.env.BETTER_AUTH_GITHUB_CLIENT_SECRET, NODE_ENV: process.env.NODE_ENV, NEXT_PUBLIC_BACKEND_BASE_URL: process.env.NEXT_PUBLIC_BACKEND_BASE_URL, diff --git a/frontend/src/server/better-auth/client.ts b/frontend/src/server/better-auth/client.ts deleted file mode 100644 index 493f84993..000000000 --- a/frontend/src/server/better-auth/client.ts +++ /dev/null @@ -1,5 +0,0 @@ -import { createAuthClient } from "better-auth/react"; - -export const authClient = createAuthClient(); - -export type Session = typeof authClient.$Infer.Session; diff --git a/frontend/src/server/better-auth/config.ts b/frontend/src/server/better-auth/config.ts deleted file mode 100644 index abf50faca..000000000 --- a/frontend/src/server/better-auth/config.ts +++ /dev/null @@ -1,9 +0,0 @@ -import { betterAuth } from "better-auth"; - -export const auth = betterAuth({ - emailAndPassword: { - enabled: true, - }, -}); - -export type Session = typeof auth.$Infer.Session; diff --git a/frontend/src/server/better-auth/index.ts b/frontend/src/server/better-auth/index.ts deleted file mode 100644 index d705e873e..000000000 --- a/frontend/src/server/better-auth/index.ts +++ /dev/null @@ -1 +0,0 @@ -export { auth } from "./config"; diff --git a/frontend/src/server/better-auth/server.ts b/frontend/src/server/better-auth/server.ts deleted file mode 100644 index 064cd349c..000000000 --- a/frontend/src/server/better-auth/server.ts +++ /dev/null @@ -1,8 +0,0 @@ -import { headers } from "next/headers"; -import { cache } from "react"; - -import { auth } from "."; - -export const getSession = cache(async () => - auth.api.getSession({ headers: await headers() }), -); From 848ace98cb0ca54735f6003b711d0bbb21eecab8 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Sat, 11 Apr 2026 16:39:12 +0800 Subject: [PATCH 03/38] feat: replace auto-admin creation with secure interactive first-boot setup (#2063) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(persistence): add unified persistence layer with event store, token tracking, and feedback (#1930) * feat(persistence): add SQLAlchemy 2.0 async ORM scaffold Introduce a unified database configuration (DatabaseConfig) that controls both the LangGraph checkpointer and the DeerFlow application persistence layer from a single `database:` config section. New modules: - deerflow.config.database_config — Pydantic config with memory/sqlite/postgres backends - deerflow.persistence — async engine lifecycle, DeclarativeBase with to_dict mixin, Alembic skeleton - deerflow.runtime.runs.store — RunStore ABC + MemoryRunStore implementation Gateway integration initializes/tears down the persistence engine in the existing langgraph_runtime() context manager. Legacy checkpointer config is preserved for backward compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) * feat(persistence): add RunEventStore ABC + MemoryRunEventStore Phase 2-A prerequisite for event storage: adds the unified run event stream interface (RunEventStore) with an in-memory implementation, RunEventsConfig, gateway integration, and comprehensive tests (27 cases). Co-Authored-By: Claude Opus 4.6 (1M context) * feat(persistence): add ORM models, repositories, DB/JSONL event stores, RunJournal, and API endpoints Phase 2-B: run persistence + event storage + token tracking. - ORM models: RunRow (with token fields), ThreadMetaRow, RunEventRow - RunRepository implements RunStore ABC via SQLAlchemy ORM - ThreadMetaRepository with owner access control - DbRunEventStore with trace content truncation and cursor pagination - JsonlRunEventStore with per-run files and seq recovery from disk - RunJournal (BaseCallbackHandler) captures LLM/tool/lifecycle events, accumulates token usage by caller type, buffers and flushes to store - RunManager now accepts optional RunStore for persistent backing - Worker creates RunJournal, writes human_message, injects callbacks - Gateway deps use factory functions (RunRepository when DB available) - New endpoints: messages, run messages, run events, token-usage - ThreadCreateRequest gains assistant_id field - 92 tests pass (33 new), zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) * feat(persistence): add user feedback + follow-up run association Phase 2-C: feedback and follow-up tracking. - FeedbackRow ORM model (rating +1/-1, optional message_id, comment) - FeedbackRepository with CRUD, list_by_run/thread, aggregate stats - Feedback API endpoints: create, list, stats, delete - follow_up_to_run_id in RunCreateRequest (explicit or auto-detected from latest successful run on the thread) - Worker writes follow_up_to_run_id into human_message event metadata - Gateway deps: feedback_repo factory + getter - 17 new tests (14 FeedbackRepository + 3 follow-up association) - 109 total tests pass, zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) * test+config: comprehensive Phase 2 test coverage + deprecate checkpointer config - config.example.yaml: deprecate standalone checkpointer section, activate unified database:sqlite as default (drives both checkpointer + app data) - New: test_thread_meta_repo.py (14 tests) — full ThreadMetaRepository coverage including check_access owner logic, list_by_owner pagination - Extended test_run_repository.py (+4 tests) — completion preserves fields, list ordering desc, limit, owner_none returns all - Extended test_run_journal.py (+8 tests) — on_chain_error, track_tokens=false, middleware no ai_message, unknown caller tokens, convenience fields, tool_error, non-summarization custom event - Extended test_run_event_store.py (+7 tests) — DB batch seq continuity, make_run_event_store factory (memory/db/jsonl/fallback/unknown) - Extended test_phase2b_integration.py (+4 tests) — create_or_reject persists, follow-up metadata, summarization in history, full DB-backed lifecycle - Fixed DB integration test to use proper fake objects (not MagicMock) for JSON-serializable metadata - 157 total Phase 2 tests pass, zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) * config: move default sqlite_dir to .deer-flow/data Keep SQLite databases alongside other DeerFlow-managed data (threads, memory) under the .deer-flow/ directory instead of a top-level ./data folder. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(persistence): remove UTFJSON, use engine-level json_serializer + datetime.now() - Replace custom UTFJSON type with standard sqlalchemy.JSON in all ORM models. Add json_serializer=json.dumps(ensure_ascii=False) to all create_async_engine calls so non-ASCII text (Chinese etc.) is stored as-is in both SQLite and Postgres. - Change ORM datetime defaults from datetime.now(UTC) to datetime.now(), remove UTC imports. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(gateway): simplify deps.py with getter factory + inline repos - Replace 6 identical getter functions with _require() factory. - Inline 3 _make_*_repo() factories into langgraph_runtime(), call get_session_factory() once instead of 3 times. - Add thread_meta upsert in start_run (services.py). Co-Authored-By: Claude Opus 4.6 (1M context) * feat(docker): add UV_EXTRAS build arg for optional dependencies Support installing optional dependency groups (e.g. postgres) at Docker build time via UV_EXTRAS build arg: UV_EXTRAS=postgres docker compose build Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(journal): fix flush, token tracking, and consolidate tests RunJournal fixes: - _flush_sync: retain events in buffer when no event loop instead of dropping them; worker's finally block flushes via async flush(). - on_llm_end: add tool_calls filter and caller=="lead_agent" guard for ai_message events; mark message IDs for dedup with record_llm_usage. - worker.py: persist completion data (tokens, message count) to RunStore in finally block. Model factory: - Auto-inject stream_usage=True for BaseChatOpenAI subclasses with custom api_base, so usage_metadata is populated in streaming responses. Test consolidation: - Delete test_phase2b_integration.py (redundant with existing tests). - Move DB-backed lifecycle test into test_run_journal.py. - Add tests for stream_usage injection in test_model_factory.py. - Clean up executor/task_tool dead journal references. Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): widen content type to str|dict in all store backends Allow event content to be a dict (for structured OpenAI-format messages) in addition to plain strings. Dict values are JSON-serialized for the DB backend and deserialized on read; memory and JSONL backends handle dicts natively. Trace truncation now serializes dicts to JSON before measuring. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(events): use metadata flag instead of heuristic for dict content detection Co-Authored-By: Claude Opus 4.6 (1M context) * feat(converters): add LangChain-to-OpenAI message format converters Pure functions langchain_to_openai_message, langchain_to_openai_completion, langchain_messages_to_openai, and _infer_finish_reason for converting LangChain BaseMessage objects to OpenAI Chat Completions format, used by RunJournal for event storage. 15 unit tests added. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(converters): handle empty list content as null, clean up test Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): human_message content uses OpenAI user message format Co-Authored-By: Claude Sonnet 4.6 * feat(events): ai_message uses OpenAI format, add ai_tool_call message event - ai_message content now uses {"role": "assistant", "content": "..."} format - New ai_tool_call message event emitted when lead_agent LLM responds with tool_calls - ai_tool_call uses langchain_to_openai_message converter for consistent format - Both events include finish_reason in metadata ("stop" or "tool_calls") Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): add tool_result message event with OpenAI tool message format Cache tool_call_id from on_tool_start keyed by run_id as fallback for on_tool_end, then emit a tool_result message event (role=tool, tool_call_id, content) after each successful tool completion. Co-Authored-By: Claude Sonnet 4.6 * feat(events): summary content uses OpenAI system message format Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): replace llm_start/llm_end with llm_request/llm_response in OpenAI format Add on_chat_model_start to capture structured prompt messages as llm_request events. Replace llm_end trace events with llm_response using OpenAI Chat Completions format. Track llm_call_index to pair request/response events. Co-Authored-By: Claude Opus 4.6 (1M context) * feat(events): add record_middleware method for middleware trace events Co-Authored-By: Claude Opus 4.6 (1M context) * test(events): add full run sequence integration test for OpenAI content format Co-Authored-By: Claude Sonnet 4.6 * feat(events): align message events with checkpoint format and add middleware tag injection - Message events (ai_message, ai_tool_call, tool_result, human_message) now use BaseMessage.model_dump() format, matching LangGraph checkpoint values.messages - on_tool_end extracts tool_call_id/name/status from ToolMessage objects - on_tool_error now emits tool_result message events with error status - record_middleware uses middleware:{tag} event_type and middleware category - Summarization custom events use middleware:summarize category - TitleMiddleware injects middleware:title tag via get_config() inheritance - SummarizationMiddleware model bound with middleware:summarize tag - Worker writes human_message using HumanMessage.model_dump() Co-Authored-By: Claude Opus 4.6 (1M context) * feat(threads): switch search endpoint to threads_meta table and sync title - POST /api/threads/search now queries threads_meta table directly, removing the two-phase Store + Checkpointer scan approach - Add ThreadMetaRepository.search() with metadata/status filters - Add ThreadMetaRepository.update_display_name() for title sync - Worker syncs checkpoint title to threads_meta.display_name on run completion - Map display_name to values.title in search response for API compatibility Co-Authored-By: Claude Opus 4.6 (1M context) * feat(threads): history endpoint reads messages from event store - POST /api/threads/{thread_id}/history now combines two data sources: checkpointer for checkpoint_id, metadata, title, thread_data; event store for messages (complete history, not truncated by summarization) - Strip internal LangGraph metadata keys from response - Remove full channel_values serialization in favor of selective fields Co-Authored-By: Claude Opus 4.6 (1M context) * fix: remove duplicate optional-dependencies header in pyproject.toml Co-Authored-By: Claude Opus 4.6 (1M context) * fix(middleware): pass tagged config to TitleMiddleware ainvoke call Without the config, the middleware:title tag was not injected, causing the LLM response to be recorded as a lead_agent ai_message in run_events. Co-Authored-By: Claude Opus 4.6 (1M context) * fix: resolve merge conflict in .env.example Keep both DATABASE_URL (from persistence-scaffold) and WECOM credentials (from main) after the merge. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(persistence): address review feedback on PR #1851 - Fix naive datetime.now() → datetime.now(UTC) in all ORM models - Fix seq race condition in DbRunEventStore.put() with FOR UPDATE and UNIQUE(thread_id, seq) constraint - Encapsulate _store access in RunManager.update_run_completion() - Deduplicate _store.put() logic in RunManager via _persist_to_store() - Add update_run_completion to RunStore ABC + MemoryRunStore - Wire follow_up_to_run_id through the full create path - Add error recovery to RunJournal._flush_sync() lost-event scenario - Add migration note for search_threads breaking change - Fix test_checkpointer_none_fix mock to set database=None Co-Authored-By: Claude Opus 4.6 (1M context) * chore: update uv.lock Co-Authored-By: Claude Opus 4.6 (1M context) * fix(persistence): address 22 review comments from CodeQL, Copilot, and Code Quality Bug fixes: - Sanitize log params to prevent log injection (CodeQL) - Reset threads_meta.status to idle/error when run completes - Attach messages only to latest checkpoint in /history response - Write threads_meta on POST /threads so new threads appear in search Lint fixes: - Remove unused imports (journal.py, migrations/env.py, test_converters.py) - Convert lambda to named function (engine.py, Ruff E731) - Remove unused logger definitions in repos (Ruff F841) - Add logging to JSONL decode errors and empty except blocks - Separate assert side-effects in tests (CodeQL) - Remove unused local variables in tests (Ruff F841) - Fix max_trace_content truncation to use byte length, not char length Co-Authored-By: Claude Opus 4.6 (1M context) * style: apply ruff format to persistence and runtime files Co-Authored-By: Claude Opus 4.6 (1M context) * Potential fix for pull request finding 'Statement has no effect' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * refactor(runtime): introduce RunContext to reduce run_agent parameter bloat Extract checkpointer, store, event_store, run_events_config, thread_meta_repo, and follow_up_to_run_id into a frozen RunContext dataclass. Add get_run_context() in deps.py to build the base context from app.state singletons. start_run() uses dataclasses.replace() to enrich per-run fields before passing ctx to run_agent. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(gateway): move sanitize_log_param to app/gateway/utils.py Extract the log-injection sanitizer from routers/threads.py into a shared utils module and rename to sanitize_log_param (public API). Eliminates the reverse service → router import in services.py. Co-Authored-By: Claude Opus 4.6 (1M context) * perf: use SQL aggregation for feedback stats and thread token usage Replace Python-side counting in FeedbackRepository.aggregate_by_run with a single SELECT COUNT/SUM query. Add RunStore.aggregate_tokens_by_thread abstract method with SQL GROUP BY implementation in RunRepository and Python fallback in MemoryRunStore. Simplify the thread_token_usage endpoint to delegate to the new method, eliminating the limit=10000 truncation risk. Co-Authored-By: Claude Opus 4.6 (1M context) * docs: annotate DbRunEventStore.put() as low-frequency path Add docstring clarifying that put() opens a per-call transaction with FOR UPDATE and should only be used for infrequent writes (currently just the initial human_message event). High-throughput callers should use put_batch() instead. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(threads): fall back to Store search when ThreadMetaRepository is unavailable When database.backend=memory (default) or no SQL session factory is configured, search_threads now queries the LangGraph Store instead of returning 503. Returns empty list if neither Store nor repo is available. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(persistence): introduce ThreadMetaStore ABC for backend-agnostic thread metadata Add ThreadMetaStore abstract base class with create/get/search/update/delete interface. ThreadMetaRepository (SQL) now inherits from it. New MemoryThreadMetaStore wraps LangGraph BaseStore for memory-mode deployments. deps.py now always provides a non-None thread_meta_repo, eliminating all `if thread_meta_repo is not None` guards in services.py, worker.py, and routers/threads.py. search_threads no longer needs a Store fallback branch. Co-Authored-By: Claude Opus 4.6 (1M context) * refactor(history): read messages from checkpointer instead of RunEventStore The /history endpoint now reads messages directly from the checkpointer's channel_values (the authoritative source) instead of querying RunEventStore.list_messages(). The RunEventStore API is preserved for other consumers. Co-Authored-By: Claude Opus 4.6 (1M context) * fix(persistence): address new Copilot review comments - feedback.py: validate thread_id/run_id before deleting feedback - jsonl.py: add path traversal protection with ID validation - run_repo.py: parse `before` to datetime for PostgreSQL compat - thread_meta_repo.py: fix pagination when metadata filter is active - database_config.py: use resolve_path for sqlite_dir consistency Co-Authored-By: Claude Opus 4.6 (1M context) * Implement skill self-evolution and skill_manage flow (#1874) * chore: ignore .worktrees directory * Add skill_manage self-evolution flow * Fix CI regressions for skill_manage * Address PR review feedback for skill evolution * fix(skill-evolution): preserve history on delete * fix(skill-evolution): tighten scanner fallbacks * docs: add skill_manage e2e evidence screenshot * fix(skill-manage): avoid blocking fs ops in session runtime --------- Co-authored-by: Willem Jiang * fix(config): resolve sqlite_dir relative to CWD, not Paths.base_dir resolve_path() resolves relative to Paths.base_dir (.deer-flow), which double-nested the path to .deer-flow/.deer-flow/data/app.db. Use Path.resolve() (CWD-relative) instead. Co-Authored-By: Claude Opus 4.6 (1M context) * Feature/feishu receive file (#1608) * feat(feishu): add channel file materialization hook for inbound messages - Introduce Channel.receive_file(msg, thread_id) as a base method for file materialization; default is no-op. - Implement FeishuChannel.receive_file to download files/images from Feishu messages, save to sandbox, and inject virtual paths into msg.text. - Update ChannelManager to call receive_file for any channel if msg.files is present, enabling downstream model access to user-uploaded files. - No impact on Slack/Telegram or other channels (they inherit the default no-op). * style(backend): format code with ruff for lint compliance - Auto-formatted packages/harness/deerflow/agents/factory.py and tests/test_create_deerflow_agent.py using `ruff format` - Ensured both files conform to project linting standards - Fixes CI lint check failures caused by code style issues * fix(feishu): handle file write operation asynchronously to prevent blocking * fix(feishu): rename GetMessageResourceRequest to _GetMessageResourceRequest and remove redundant code * test(feishu): add tests for receive_file method and placeholder replacement * fix(manager): remove unnecessary type casting for channel retrieval * fix(feishu): update logging messages to reflect resource handling instead of image * fix(feishu): sanitize filename by replacing invalid characters in file uploads * fix(feishu): improve filename sanitization and reorder image key handling in message processing * fix(feishu): add thread lock to prevent filename conflicts during file downloads * fix(test): correct bad merge in test_feishu_parser.py * chore: run ruff and apply formatting cleanup fix(feishu): preserve rich-text attachment order and improve fallback filename handling * fix(docker): restore gateway env vars and fix langgraph empty arg issue (#1915) Two production docker-compose.yaml bugs prevent `make up` from working: 1. Gateway missing DEER_FLOW_CONFIG_PATH and DEER_FLOW_EXTENSIONS_CONFIG_PATH environment overrides. Added in fb2d99f (#1836) but accidentally reverted by ca2fb95 (#1847). Without them, gateway reads host paths from .env via env_file, causing FileNotFoundError inside the container. 2. Langgraph command fails when LANGGRAPH_ALLOW_BLOCKING is unset (default). Empty $${allow_blocking} inserts a bare space between flags, causing ' --no-reload' to be parsed as unexpected extra argument. Fix by building args string first and conditionally appending --allow-blocking. Co-authored-by: cooper * fix(frontend): resolve invalid HTML nesting and tabnabbing vulnerabilities (#1904) * fix(frontend): resolve invalid HTML nesting and tabnabbing vulnerabilities Fix ` + +
+
+ ); + } + + // ── Change-password form (needs_setup after login) ───────────────── return ( -
-
+
+ +

DeerFlow

@@ -73,7 +266,7 @@ export default function SetupPage() { Set your real email and a new password.

-
+ setCurrentPassword(e.target.value)} required @@ -106,7 +299,7 @@ export default function SetupPage() { /> {error &&

{error}

}
diff --git a/frontend/src/app/workspace/layout.tsx b/frontend/src/app/workspace/layout.tsx index fa19025a0..c2d567339 100644 --- a/frontend/src/app/workspace/layout.tsx +++ b/frontend/src/app/workspace/layout.tsx @@ -23,6 +23,8 @@ export default async function WorkspaceLayout({ ); case "needs_setup": redirect("/setup"); + case "system_setup_required": + redirect("/setup"); case "unauthenticated": redirect("/login"); case "gateway_unavailable": diff --git a/frontend/src/components/workspace/settings/account-settings-page.tsx b/frontend/src/components/workspace/settings/account-settings-page.tsx index 6382b8859..c00d6961e 100644 --- a/frontend/src/components/workspace/settings/account-settings-page.tsx +++ b/frontend/src/components/workspace/settings/account-settings-page.tsx @@ -69,12 +69,10 @@ export function AccountSettingsPage() { return (
-
-
+
+
Email {user?.email ?? "—"} -
-
Role {user?.system_role ?? "—"} @@ -83,7 +81,10 @@ export function AccountSettingsPage() {
- +
- + + +
+ ); +} + export function MessageListItem({ className, + threadId, message, isLoading, - threadId, tokenUsageEnabled = false, + feedback, + runId, }: { className?: string; message: Message; isLoading?: boolean; threadId: string; tokenUsageEnabled?: boolean; + feedback?: FeedbackData | null; + runId?: string; }) { const isHuman = message.type === "human"; return ( @@ -70,7 +153,7 @@ export function MessageListItem({
@@ -81,6 +164,13 @@ export function MessageListItem({ "" } /> + {feedback !== undefined && runId && threadId && ( + + )}
)} diff --git a/frontend/src/components/workspace/messages/message-list.tsx b/frontend/src/components/workspace/messages/message-list.tsx index d1d02c6d0..238046ae6 100644 --- a/frontend/src/components/workspace/messages/message-list.tsx +++ b/frontend/src/components/workspace/messages/message-list.tsx @@ -19,6 +19,7 @@ import { useRehypeSplitWordsIntoSpans } from "@/core/rehype"; import type { Subtask } from "@/core/tasks"; import { useUpdateSubtask } from "@/core/tasks/context"; import type { AgentThreadState } from "@/core/threads"; +import { useThreadMessageEnrichment } from "@/core/threads/hooks"; import { cn } from "@/lib/utils"; import { ArtifactFileList } from "../artifacts/artifact-file-list"; @@ -51,6 +52,8 @@ export function MessageList({ const rehypePlugins = useRehypeSplitWordsIntoSpans(thread.isLoading); const updateSubtask = useUpdateSubtask(); const messages = thread.messages; + const { data: enrichment } = useThreadMessageEnrichment(threadId); + if (thread.isThreadLoading && messages.length === 0) { return ; } @@ -62,13 +65,16 @@ export function MessageList({ {groupMessages(messages, (group) => { if (group.type === "human" || group.type === "assistant") { return group.messages.map((msg) => { + const entry = msg.id ? enrichment?.get(msg.id) : undefined; return ( ); }); @@ -183,7 +189,7 @@ export function MessageList({ results.push(
{t.subtasks.executing(tasks.size)}
, diff --git a/frontend/src/core/api/feedback.ts b/frontend/src/core/api/feedback.ts new file mode 100644 index 000000000..5af3f02c8 --- /dev/null +++ b/frontend/src/core/api/feedback.ts @@ -0,0 +1,42 @@ +import { getBackendBaseURL } from "../config"; + +import { fetchWithAuth } from "./fetcher"; + +export interface FeedbackData { + feedback_id: string; + rating: number; + comment: string | null; +} + +export async function upsertFeedback( + threadId: string, + runId: string, + rating: number, + comment?: string, +): Promise { + const res = await fetchWithAuth( + `${getBackendBaseURL()}/api/threads/${encodeURIComponent(threadId)}/runs/${encodeURIComponent(runId)}/feedback`, + { + method: "PUT", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ rating, comment: comment ?? null }), + }, + ); + if (!res.ok) { + throw new Error(`Failed to submit feedback: ${res.status}`); + } + return res.json(); +} + +export async function deleteFeedback( + threadId: string, + runId: string, +): Promise { + const res = await fetchWithAuth( + `${getBackendBaseURL()}/api/threads/${encodeURIComponent(threadId)}/runs/${encodeURIComponent(runId)}/feedback`, + { method: "DELETE" }, + ); + if (!res.ok && res.status !== 404) { + throw new Error(`Failed to delete feedback: ${res.status}`); + } +} diff --git a/frontend/src/core/threads/hooks.ts b/frontend/src/core/threads/hooks.ts index 33424aeee..3b0aafda2 100644 --- a/frontend/src/core/threads/hooks.ts +++ b/frontend/src/core/threads/hooks.ts @@ -8,6 +8,7 @@ import { toast } from "sonner"; import type { PromptInputMessage } from "@/components/ai-elements/prompt-input"; import { getAPIClient } from "../api"; +import type { FeedbackData } from "../api/feedback"; import { fetchWithAuth } from "../api/fetcher"; import { getBackendBaseURL } from "../config"; import { useI18n } from "../i18n/hooks"; @@ -294,6 +295,9 @@ export function useThreadStream({ onFinish(state) { listeners.current.onFinish?.(state.values); void queryClient.invalidateQueries({ queryKey: ["threads", "search"] }); + void queryClient.invalidateQueries({ + queryKey: ["thread-message-enrichment"], + }); }, }); @@ -678,3 +682,65 @@ export function useRenameThread() { }, }); } + +/** Per-message enrichment data attached by the backend ``/history`` helper. */ +export interface MessageEnrichment { + run_id: string; + /** ``undefined`` = not feedback-eligible; ``null`` = eligible but unrated. */ + feedback?: FeedbackData | null; +} + +/** + * Fetch ``/history`` once and index feedback + run_id by message id. + * + * Replaces the old ``useThreadFeedback`` hook which keyed by AI-message + * ordinal position — an inherently fragile mapping that broke whenever + * ``ai_tool_call`` messages were interleaved with ``ai_message`` messages. + * Keying by ``message.id`` is stable regardless of run count, tool-call + * chains, or summarization. + * + * The ``/history`` response is refreshed on every stream completion via + * ``invalidateQueries(["thread-message-enrichment"])`` in ``onFinish``. + */ +export function useThreadMessageEnrichment( + threadId: string | null | undefined, +) { + return useQuery({ + queryKey: ["thread-message-enrichment", threadId], + queryFn: async (): Promise> => { + const empty = new Map(); + if (!threadId) return empty; + const res = await fetchWithAuth( + `${getBackendBaseURL()}/api/threads/${encodeURIComponent(threadId)}/history`, + { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ limit: 1 }), + }, + ); + if (!res.ok) return empty; + const entries = (await res.json()) as Array<{ + values?: { + messages?: Array<{ + id?: string; + run_id?: string; + feedback?: FeedbackData | null; + }>; + }; + }>; + const messages = entries[0]?.values?.messages ?? []; + const map = new Map(); + for (const m of messages) { + if (!m.id || !m.run_id) continue; + const entry: MessageEnrichment = { run_id: m.run_id }; + // Preserve presence: "feedback" key absent → ineligible; present with + // null → eligible but unrated; present with object → rated. + if ("feedback" in m) entry.feedback = m.feedback; + map.set(m.id, entry); + } + return map; + }, + enabled: !!threadId, + staleTime: 30_000, + }); +} From 44d9953e2e2f4f2993660ecb191be86ed89e608a Mon Sep 17 00:00:00 2001 From: JeffJiang Date: Sun, 12 Apr 2026 11:16:08 +0800 Subject: [PATCH 09/38] feat: Add metadata and descriptions to various documentation pages in Chinese - Added titles and descriptions to workspace usage, configuration, customization, design principles, installation, integration guide, lead agent, MCP integration, memory system, middleware, quick start, sandbox, skills, subagents, and tools documentation. - Removed outdated API/Gateway reference and concepts glossary pages. - Updated configuration reference to reflect current structure and removed unnecessary sections. - Introduced new model provider documentation for Ark and updated the index page for model providers. - Enhanced tutorials with titles and descriptions for better clarity and navigation. --- deer-flow.code-workspace | 4 +- frontend/src/app/[lang]/docs/layout.tsx | 4 +- frontend/src/components/landing/footer.tsx | 15 +- frontend/src/content/en/_meta.ts | 9 + .../en/application/agents-and-threads.mdx | 5 + .../content/en/application/configuration.mdx | 5 + .../en/application/deployment-guide.mdx | 56 +++--- frontend/src/content/en/application/index.mdx | 5 + .../operations-and-troubleshooting.mdx | 5 + .../content/en/application/quick-start.mdx | 5 + .../en/application/workspace-usage.mdx | 5 + .../src/content/en/harness/configuration.mdx | 5 + .../src/content/en/harness/customization.mdx | 5 + .../content/en/harness/design-principles.mdx | 5 + frontend/src/content/en/harness/index.mdx | 5 + .../content/en/harness/integration-guide.mdx | 5 + .../src/content/en/harness/lead-agent.mdx | 5 + frontend/src/content/en/harness/mcp.mdx | 5 + frontend/src/content/en/harness/memory.mdx | 5 + .../src/content/en/harness/middlewares.mdx | 5 + .../src/content/en/harness/quick-start.mdx | 168 +++++++----------- frontend/src/content/en/harness/sandbox.mdx | 5 + frontend/src/content/en/harness/skills.mdx | 5 + frontend/src/content/en/harness/subagents.mdx | 5 + frontend/src/content/en/harness/tools.mdx | 5 + .../content/en/introduction/core-concepts.mdx | 5 + .../en/introduction/harness-vs-app.mdx | 5 + .../content/en/introduction/why-deerflow.mdx | 5 + frontend/src/content/en/reference/_meta.ts | 16 +- .../en/reference/api-gateway-reference.mdx | 69 ------- .../en/reference/concepts-glossary.mdx | 67 ------- .../en/reference/configuration-reference.mdx | 123 ------------- .../en/reference/model-providers/_meta.ts | 9 + .../en/reference/model-providers/ark.mdx | 8 + .../en/reference/model-providers/index.mdx | 7 + .../en/reference/runtime-flags-and-modes.mdx | 36 ---- .../src/content/en/reference/source-map.mdx | 88 --------- .../tutorials/create-your-first-harness.mdx | 5 + .../en/tutorials/deploy-your-own-deerflow.mdx | 5 + .../en/tutorials/first-conversation.mdx | 5 + .../en/tutorials/use-tools-and-skills.mdx | 5 + .../content/en/tutorials/work-with-memory.mdx | 5 + frontend/src/content/zh/_meta.ts | 9 + .../zh/application/agents-and-threads.mdx | 5 + .../content/zh/application/configuration.mdx | 5 + .../zh/application/deployment-guide.mdx | 5 + frontend/src/content/zh/application/index.mdx | 5 + .../operations-and-troubleshooting.mdx | 5 + .../content/zh/application/quick-start.mdx | 5 + .../zh/application/workspace-usage.mdx | 5 + .../src/content/zh/harness/configuration.mdx | 5 + .../src/content/zh/harness/customization.mdx | 5 + .../content/zh/harness/design-principles.mdx | 5 + frontend/src/content/zh/harness/index.mdx | 5 + .../content/zh/harness/integration-guide.mdx | 5 + .../src/content/zh/harness/lead-agent.mdx | 5 + frontend/src/content/zh/harness/mcp.mdx | 10 +- frontend/src/content/zh/harness/memory.mdx | 5 + .../src/content/zh/harness/middlewares.mdx | 5 + .../src/content/zh/harness/quick-start.mdx | 166 +++++++---------- frontend/src/content/zh/harness/sandbox.mdx | 5 + frontend/src/content/zh/harness/skills.mdx | 5 + frontend/src/content/zh/harness/subagents.mdx | 5 + frontend/src/content/zh/harness/tools.mdx | 5 + .../content/zh/introduction/core-concepts.mdx | 5 + .../zh/introduction/harness-vs-app.mdx | 5 + .../content/zh/introduction/why-deerflow.mdx | 5 + frontend/src/content/zh/reference/_meta.ts | 16 +- .../zh/reference/api-gateway-reference.mdx | 68 ------- .../zh/reference/concepts-glossary.mdx | 67 ------- .../zh/reference/configuration-reference.mdx | 122 ------------- .../zh/reference/model-providers/_meta.ts | 9 + .../zh/reference/model-providers/ark.mdx | 8 + .../zh/reference/model-providers/index.mdx | 7 + .../zh/reference/runtime-flags-and-modes.mdx | 36 ---- .../src/content/zh/reference/source-map.mdx | 88 --------- .../tutorials/create-your-first-harness.mdx | 5 + .../zh/tutorials/deploy-your-own-deerflow.mdx | 5 + .../zh/tutorials/first-conversation.mdx | 5 + .../zh/tutorials/use-tools-and-skills.mdx | 5 + .../content/zh/tutorials/work-with-memory.mdx | 5 + 81 files changed, 528 insertions(+), 1027 deletions(-) delete mode 100644 frontend/src/content/en/reference/api-gateway-reference.mdx delete mode 100644 frontend/src/content/en/reference/concepts-glossary.mdx delete mode 100644 frontend/src/content/en/reference/configuration-reference.mdx create mode 100644 frontend/src/content/en/reference/model-providers/_meta.ts create mode 100644 frontend/src/content/en/reference/model-providers/ark.mdx create mode 100644 frontend/src/content/en/reference/model-providers/index.mdx delete mode 100644 frontend/src/content/en/reference/runtime-flags-and-modes.mdx delete mode 100644 frontend/src/content/en/reference/source-map.mdx delete mode 100644 frontend/src/content/zh/reference/api-gateway-reference.mdx delete mode 100644 frontend/src/content/zh/reference/concepts-glossary.mdx delete mode 100644 frontend/src/content/zh/reference/configuration-reference.mdx create mode 100644 frontend/src/content/zh/reference/model-providers/_meta.ts create mode 100644 frontend/src/content/zh/reference/model-providers/ark.mdx create mode 100644 frontend/src/content/zh/reference/model-providers/index.mdx delete mode 100644 frontend/src/content/zh/reference/runtime-flags-and-modes.mdx delete mode 100644 frontend/src/content/zh/reference/source-map.mdx diff --git a/deer-flow.code-workspace b/deer-flow.code-workspace index ef2863302..a4f4cb240 100644 --- a/deer-flow.code-workspace +++ b/deer-flow.code-workspace @@ -5,7 +5,7 @@ } ], "settings": { - "typescript.tsdk": "frontend/node_modules/typescript/lib", + "js/ts.tsdk.path": "frontend/node_modules/typescript/lib", "python-envs.pythonProjects": [ { "path": "backend", @@ -44,4 +44,4 @@ } ] } -} +} \ No newline at end of file diff --git a/frontend/src/app/[lang]/docs/layout.tsx b/frontend/src/app/[lang]/docs/layout.tsx index f63d6ae7b..895da1da8 100644 --- a/frontend/src/app/[lang]/docs/layout.tsx +++ b/frontend/src/app/[lang]/docs/layout.tsx @@ -34,14 +34,14 @@ export default async function DocLayout({ children, params }) { } pageMap={pageMap} docsRepositoryBase="https://github.com/bytedance/deerflow/tree/main/frontend/src/content" - footer={
@@ -144,7 +158,7 @@ export default function ChatPage() { />
- {mounted ? ( + {mountedRef.current ? (