DanielWalnut 25ea6970a6
feat(runtime): implement goal continuations (#3858)
* implement goal continuations

* fix(goal): address review findings for goal continuations

- goal: key the no-progress breaker on a signature of the latest visible
  assistant evidence instead of the evaluator's volatile free-text, so it
  actually fires on stalled turns; thread the signature through every
  worker persist / no-progress call site
- goal: align _stand_down_reason default caps with should_continue_goal
  (8 / 2) so the two gate functions agree on goals missing the fields
- runtime: offload the synchronous checkpointer fallback via
  asyncio.to_thread (goal.py + worker.py) to keep blocking IO off the loop
- frontend: i18n the GoalStatus "Goal" label (goalLabel in en/zh/types)
- frontend: extract pure composer helpers into input-box-helpers.ts with
  unit tests (parseGoalCommand, readGoalResponseError, skill suggestions)
- tests: cover the evidence-based no-progress and default-cap behavior
- docs: align backend/AGENTS.md goal paragraph with actual behavior
- e2e: prettier-format chat.spec.ts (fixes the lint-frontend CI failure)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(frontend): hide goal continuation counter until the agent continues

The goal status bar rendered a raw "0/8" before any auto-continuation, which
read as a mysterious score. Now the counter is hidden until
continuation_count > 0, then shows "Continuing N/M" with a tooltip explaining
the auto-continuation cap.

- Extract getGoalContinuationDisplay into a pure helper (hides at 0) + unit tests
- Add goalContinuing / goalContinuationTooltip i18n keys (en/zh/types)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(goal): address review findings for goal continuations

Frontend correctness
- Fix the optimistic /goal result permanently shadowing server goal state:
  the streamed continuation counter never surfaced for a goal set in-session.
  Extract a shared useActiveGoal hook (used by both chat pages) that reconciles
  the optimistic copy with server state via a goalReconciliationKey, de-duping
  the copy-pasted goal block across the two pages.
- Stop /goal status|clear failures from escaping handleSubmit as unhandled
  rejections (handleGoalCommand now returns success; the run only starts when a
  goal was actually saved).
- Use a function replacer for the goal-status toast so an objective containing
  $&/$1 isn't treated as a replacement pattern.

Backend cleanliness / correctness
- De-duplicate four byte-identical helpers (_call_checkpointer_method,
  _message_type, _additional_kwargs, _is_visible_message) by importing them
  from runtime.goal instead of re-defining them in the run worker.
- Remove the dead `checkpoint_tuple.tasks` durability guard (CheckpointTuple has
  no tasks field) and document that pending_writes is the durability signal.
- Decompose the 176-line _prepare_goal_continuation_input: extract
  _reread_goal_and_checkpoint and a _persist closure so the thread-unchanged
  guard and stand-down persistence aren't open-coded three times. Document the
  last-writer-wins write-window limitation as a follow-up.
- Add a shared parse_goal_command helper and use it from the TUI and IM-channel
  /goal handlers (one place for the status/clear/set semantics).

Tests
- Restore the 11 command-registry tests dropped by the previous goal change
  (filter_commands ranking/description, build_registry builtins/skills, resolve
  cases) alongside the new goal tests.
- Add coverage for the IM-channel _handle_goal_command, the TUI _handle_goal
  handler, parse_goal_command, and goalReconciliationKey.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix goal review feedback

* fix goal continuation checkpoint races

* prioritize goal commands while streaming

Route composer submits through a shared helper so /goal commands can be handled before the streaming stop shortcut, while ordinary streaming submits still stop the active run.

Testing: cd frontend && pnpm exec rstest run tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; cd frontend && pnpm check

* preserve goal status during clarification

Keep omitted stream goal fields distinct from explicit null clears so clarification interrupts do not hide an active thread goal that is still present in the checkpoint.

Testing: pnpm exec rstest run tests/unit/components/workspace/use-active-goal.test.ts tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; pnpm check; git diff --check

* style: format active goal hook

Run Prettier on use-active-goal.ts to satisfy the frontend lint workflow formatting gate.

Testing: pnpm format; pnpm exec rstest run tests/unit/components/workspace/use-active-goal.test.ts tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; pnpm check; git diff --check

* fix goal review followups

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 08:49:33 +08:00

653 lines
19 KiB
TypeScript

/**
* Shared mock helpers for E2E tests.
*
* Intercepts all LangGraph / Backend API endpoints so tests can run without
* a real backend. Each test file imports `mockLangGraphAPI` and
* `handleRunStream` from here.
*/
import type { Page, Route } from "@playwright/test";
// ---------------------------------------------------------------------------
// Constants — deterministic IDs used across tests
// ---------------------------------------------------------------------------
export const MOCK_THREAD_ID = "00000000-0000-0000-0000-000000000001";
export const MOCK_THREAD_ID_2 = "00000000-0000-0000-0000-000000000002";
export const MOCK_RUN_ID = "00000000-0000-0000-0000-000000000099";
const MOCK_AUTH_USER = {
id: "default",
email: "default@test.local",
system_role: "admin",
needs_setup: false,
};
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
export type MockThread = {
thread_id: string;
title?: string;
updated_at?: string;
agent_name?: string;
metadata?: Record<string, unknown>;
messages?: unknown[];
artifacts?: string[];
goal?: Record<string, unknown> | null;
};
export type MockAgent = {
name: string;
description?: string;
system_prompt?: string;
};
export type MockSkill = {
name: string;
description: string;
category?: string;
license?: string | null;
enabled?: boolean;
};
export type MockAPIOptions = {
threads?: MockThread[];
agents?: MockAgent[];
skills?: MockSkill[];
uploadLimits?: {
max_files: number;
max_file_size: number;
max_total_size: number;
};
};
const DEFAULT_SKILLS: MockSkill[] = [
{
name: "data-analysis",
description: "Analyze structured data and produce charts.",
category: "public",
enabled: true,
},
{
name: "frontend-design",
description: "Create polished frontend interfaces.",
category: "public",
enabled: true,
},
{
name: "disabled-skill",
description: "Hidden from slash autocomplete.",
category: "public",
enabled: false,
},
];
function mockStreamMessages() {
return [
{
type: "human",
id: "msg-human-1",
content: [{ type: "text", text: "Hello" }],
},
{
type: "ai",
id: "msg-ai-1",
content: "Hello from DeerFlow!",
},
];
}
// ---------------------------------------------------------------------------
// mockLangGraphAPI
// ---------------------------------------------------------------------------
/**
* Mock all LangGraph API endpoints that the frontend calls on page load and
* during message sending. Without these mocks the pages would hang waiting
* for a real backend.
*/
export function mockLangGraphAPI(page: Page, options?: MockAPIOptions) {
let threads = [...(options?.threads ?? [])];
const agents = options?.agents ?? [];
const skills = options?.skills ?? DEFAULT_SKILLS;
const uploadLimits = options?.uploadLimits ?? {
max_files: 10,
max_file_size: 50 * 1024 * 1024,
max_total_size: 100 * 1024 * 1024,
};
const upsertThread = (thread: MockThread) => {
threads = [
thread,
...threads.filter((existing) => existing.thread_id !== thread.thread_id),
];
};
const threadSearchResult = (thread: MockThread) => ({
thread_id: thread.thread_id,
created_at: "2025-01-01T00:00:00Z",
updated_at: thread.updated_at ?? "2025-01-01T00:00:00Z",
metadata: {
...(thread.metadata ?? {}),
...(thread.agent_name ? { agent_name: thread.agent_name } : {}),
},
status: "idle",
values: { title: thread.title ?? "Untitled", goal: thread.goal ?? null },
});
// Auth — keep workspace tests independent from a real gateway session.
void page.route("**/api/v1/auth/me", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify(MOCK_AUTH_USER),
});
}
return route.fallback();
});
void page.route("**/api/v1/auth/setup-status", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ needs_setup: false }),
});
}
return route.fallback();
});
void page.route("**/api/v1/auth/logout", (route) => {
if (route.request().method() === "POST") {
return route.fulfill({ status: 204 });
}
return route.fallback();
});
void page.route("**/api/channels/providers", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ enabled: false, providers: [] }),
});
}
return route.fallback();
});
void page.route("**/api/channels/connections", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ connections: [] }),
});
}
return route.fallback();
});
// Thread search — sidebar thread list & chats list page
void page.route("**/api/langgraph/threads/search", async (route) => {
const body = threads.map(threadSearchResult);
let limit: number | undefined;
let offset = 0;
try {
const postData = route.request().postDataJSON() as {
limit?: number;
offset?: number;
} | null;
if (postData) {
if (typeof postData.limit === "number") {
limit = postData.limit;
}
if (typeof postData.offset === "number") {
offset = postData.offset;
}
}
} catch {
// No / invalid JSON body — fall back to returning the full list.
}
const sliced =
typeof limit === "number" ? body.slice(offset, offset + limit) : body;
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify(sliced),
});
});
// Thread create — called when user sends first message in a new chat
void page.route("**/api/langgraph/threads", (route) => {
if (route.request().method() === "POST") {
upsertThread({
thread_id: MOCK_THREAD_ID,
title: "New Chat",
updated_at: new Date().toISOString(),
messages: mockStreamMessages(),
});
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
thread_id: MOCK_THREAD_ID,
created_at: new Date().toISOString(),
updated_at: new Date().toISOString(),
metadata: {},
status: "idle",
values: {},
}),
});
}
return route.fallback();
});
// Thread update (PATCH) — metadata update after creation
void page.route("**/api/langgraph/threads/*", (route) => {
const threadId = decodeURIComponent(
new URL(route.request().url()).pathname.split("/").at(-1) ?? "",
);
const matchingThread = threads.find(
(thread) => thread.thread_id === threadId,
);
if (route.request().method() === "GET") {
if (!matchingThread) {
return route.fulfill({
status: 404,
contentType: "application/json",
body: JSON.stringify({ detail: "Thread not found" }),
});
}
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify(threadSearchResult(matchingThread)),
});
}
if (route.request().method() === "PATCH") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ thread_id: MOCK_THREAD_ID }),
});
}
if (route.request().method() === "DELETE") {
threads = threads.filter((thread) => thread.thread_id !== threadId);
return route.fulfill({
status: 204,
});
}
return route.fallback();
});
void page.route(/\/api\/threads\/[^/]+$/, (route) => {
if (route.request().method() === "DELETE") {
return route.fulfill({
status: 204,
});
}
return route.fallback();
});
void page.route(/\/api\/threads\/[^/]+\/goal$/, async (route) => {
const threadId = decodeURIComponent(
new URL(route.request().url()).pathname.split("/").at(-2) ?? "",
);
let matchingThread = threads.find(
(thread) => thread.thread_id === threadId,
);
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ goal: matchingThread?.goal ?? null }),
});
}
if (route.request().method() === "DELETE") {
if (matchingThread) {
matchingThread.goal = null;
}
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ goal: null }),
});
}
if (route.request().method() === "PUT") {
const payload = route.request().postDataJSON() as {
objective?: string;
};
const goal = {
objective: payload.objective ?? "",
status: "active",
created_at: new Date().toISOString(),
updated_at: new Date().toISOString(),
continuation_count: 0,
max_continuations: 8,
no_progress_count: 0,
max_no_progress_continuations: 2,
};
matchingThread ??= {
thread_id: threadId,
title: "New Chat",
updated_at: new Date().toISOString(),
};
upsertThread({ ...matchingThread, goal });
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ goal }),
});
}
return route.fallback();
});
void page.route("**/api/threads/*/uploads/limits", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify(uploadLimits),
});
}
return route.fallback();
});
// Thread history — useStream fetches state history on mount
void page.route("**/api/langgraph/threads/*/history", (route) => {
const url = route.request().url();
// For threads that exist in our mock data, return history with messages
const matchingThread = threads.find((t) => url.includes(t.thread_id));
if (matchingThread) {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify([
{
values: {
title: matchingThread.title ?? "Untitled",
goal: matchingThread.goal ?? null,
messages: matchingThread.messages ?? [
{
type: "human",
id: `msg-human-${matchingThread.thread_id}`,
content: [{ type: "text", text: "Previous question" }],
},
{
type: "ai",
id: `msg-ai-${matchingThread.thread_id}`,
content: `Response in thread ${matchingThread.title ?? matchingThread.thread_id}`,
},
],
artifacts: matchingThread.artifacts ?? [],
},
next: [],
metadata: {},
created_at: "2025-01-01T00:00:00Z",
parent_config: null,
},
]),
});
}
// New threads — empty history
return route.fulfill({
status: 200,
contentType: "application/json",
body: "[]",
});
});
// Thread state — getState for individual thread
void page.route("**/api/langgraph/threads/*/state", (route) => {
if (route.request().method() === "GET") {
const url = route.request().url();
const matchingThread = threads.find((t) => url.includes(t.thread_id));
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
values: {
title: matchingThread?.title ?? "Untitled",
goal: matchingThread?.goal ?? null,
messages: matchingThread
? (matchingThread.messages ?? [
{
type: "human",
id: `msg-human-${matchingThread.thread_id}`,
content: [{ type: "text", text: "Previous question" }],
},
{
type: "ai",
id: `msg-ai-${matchingThread.thread_id}`,
content: `Response in thread ${matchingThread.title ?? matchingThread.thread_id}`,
},
])
: [],
artifacts: matchingThread?.artifacts ?? [],
},
next: [],
metadata: {},
created_at: "2025-01-01T00:00:00Z",
}),
});
}
return route.fallback();
});
// The URL carries a query string (e.g. `?limit=10&offset=0`), which Playwright
// glob `*` does NOT cross, so we match with a regex anchored to `/runs`
// followed by `?` or end-of-string. This must NOT match `/runs/stream`.
void page.route(/\/api\/langgraph\/threads\/[^/]+\/runs(\?|$)/, (route) => {
if (route.request().method() === "GET") {
const url = route.request().url();
const matchingThread = threads.find((t) => url.includes(t.thread_id));
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify(
matchingThread
? [
{
run_id: `run-${matchingThread.thread_id}`,
thread_id: matchingThread.thread_id,
assistant_id: "lead_agent",
status: "success",
metadata: {},
kwargs: {},
created_at: "2025-01-01T00:00:00Z",
updated_at:
matchingThread.updated_at ?? "2025-01-01T00:00:00Z",
},
]
: [],
),
});
}
return route.fallback();
});
void page.route(
/\/api\/threads\/([^/]+)\/runs\/([^/]+)\/messages/,
(route) => {
if (route.request().method() === "GET") {
const url = route.request().url();
const matchingThread = threads.find((t) =>
url.includes(`/api/threads/${t.thread_id}/runs/`),
);
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
data: (matchingThread?.messages ?? []).map((message, index) => ({
run_id: `run-${matchingThread?.thread_id ?? "unknown"}`,
content: message,
metadata: { caller: "lead_agent" },
created_at: `2025-01-01T00:00:${String(index).padStart(2, "0")}Z`,
})),
hasMore: false,
}),
});
}
return route.fallback();
},
);
// Run stream — returns a minimal SSE response with an AI message
const handleMockRunStream = (route: Route) => {
const requestUrl = route.request().url();
const matchingThread = threads.find((thread) =>
requestUrl.includes(encodeURIComponent(thread.thread_id)),
);
const fallbackGoal = threads.find((thread) => thread.goal)?.goal ?? null;
const goal = matchingThread?.goal ?? fallbackGoal;
upsertThread({
thread_id: MOCK_THREAD_ID,
title: "New Chat",
updated_at: new Date().toISOString(),
goal,
messages: mockStreamMessages(),
});
return handleRunStream(route, { goal });
};
void page.route("**/api/langgraph/runs/stream", handleMockRunStream);
void page.route(
"**/api/langgraph/threads/*/runs/stream",
handleMockRunStream,
);
// Models list — model picker dropdown
void page.route("**/api/models", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
models: [],
token_usage: { enabled: false },
}),
});
}
return route.fallback();
});
// Feature flags — frontend gates UI (e.g. agents) on these. Default to
// enabled so existing tests exercise the normal path; tests that need the
// disabled state override this route after calling mockLangGraphAPI.
void page.route("**/api/features", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ agents_api: { enabled: true } }),
});
}
return route.fallback();
});
// Skills list — settings page and slash autocomplete
void page.route("**/api/skills", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ skills }),
});
}
return route.fallback();
});
// Follow-up suggestions — input box auto-suggest after AI response
void page.route("**/api/threads/*/suggestions", (route) => {
if (route.request().method() === "POST") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ suggestions: [] }),
});
}
return route.fallback();
});
// Agents list — sidebar & gallery page
void page.route("**/api/agents", (route) => {
if (route.request().method() === "GET") {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ agents }),
});
}
return route.fallback();
});
// Individual agent — agent chat page
void page.route("**/api/agents/*", (route) => {
if (route.request().method() === "GET") {
const url = route.request().url();
const agent = agents.find((a) => url.endsWith(`/api/agents/${a.name}`));
if (agent) {
return route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify(agent),
});
}
}
return route.fulfill({
status: 404,
contentType: "application/json",
body: JSON.stringify({ detail: "Agent not found" }),
});
});
}
// ---------------------------------------------------------------------------
// handleRunStream
// ---------------------------------------------------------------------------
/**
* Build a minimal SSE stream that the LangGraph SDK can parse.
* The stream returns a single AI message: "Hello from DeerFlow!".
*/
export function handleRunStream(
route: Route,
values: Record<string, unknown> = {},
) {
const events = [
{
event: "metadata",
data: { run_id: MOCK_RUN_ID, thread_id: MOCK_THREAD_ID },
},
{
event: "values",
data: {
...values,
messages: mockStreamMessages(),
},
},
{ event: "end", data: {} },
];
const body = events
.map((e) => `event: ${e.event}\ndata: ${JSON.stringify(e.data)}\n\n`)
.join("");
return route.fulfill({
status: 200,
contentType: "text/event-stream",
body,
});
}