fix(frontend): keep additional_kwargs on the optimistic bubble through an upload (#5982)

* fix(frontend): keep additional_kwargs on the optimistic bubble through an upload

When a send carried files, the upload-complete update replaced the
optimistic human message's additional_kwargs with only the uploaded
files. Quote and conversation reference chips vanished until the server
echoed the message, and project attachments staged with the send were
missing from the bubble both while uploading and after. The submitted
message was always intact.

The optimistic copy (before and after the upload) and the submit now
build additional_kwargs through one helper,
buildHumanMessageAdditionalKwargs, and share a single uploaded-files
value, so the bubble always matches what was sent.

* docs(changelog): reference #5982

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

CHANGELOG.md

@@ -327,6 +327,16 @@ This release closes that milestone with **181 merged pull requests**.
 
 ### Fixed
 
+- **frontend:** The optimistic human bubble keeps its quote and conversation
+  reference chips once a file upload finishes. The upload-complete update
+  replaced the bubble's `additional_kwargs` with only the uploaded files, so
+  those chips disappeared until the server echoed the message; project
+  attachments staged with the send were also missing from the bubble, both
+  while uploading and after. The submitted message was always intact. The
+  optimistic copy (before and after the upload) and the submit now build
+  `additional_kwargs` through one helper,
+  `buildHumanMessageAdditionalKwargs`. ([#5982])
+
 - **scheduler:** Editing an interval task's title or prompt no longer fails with
   a 500. The edit dialog always sends `schedule_spec` beside the changed field,
   and when the cadence is unchanged `PATCH /api/scheduled-tasks/{id}` keeps
@@ -6194,4 +6204,5 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
 [#5945]: https://github.com/bytedance/deer-flow/pull/5945
 [#5964]: https://github.com/bytedance/deer-flow/pull/5964
 [#5981]: https://github.com/bytedance/deer-flow/pull/5981
+[#5982]: https://github.com/bytedance/deer-flow/pull/5982
 

CHANGELOG_zh.md

@@ -332,6 +332,13 @@
 
 ### 修复
 
+- **前端：** 文件上传完成后，乐观显示的用户消息气泡不再丢失引用与对话
+  引用标签。上传完成时的更新会用仅含已上传文件的对象替换气泡的
+  `additional_kwargs`，因此在服务端回传该消息之前这些标签会消失；随本次
+  发送一起暂存的项目附件在上传中和上传后也都不会显示在气泡中。实际提交的
+  消息始终完整。乐观副本（上传前后）与提交现在通过同一个辅助函数
+  `buildHumanMessageAdditionalKwargs` 构建 `additional_kwargs`。([#5982])
+
 - **调度器：** 修改间隔任务的标题或 prompt 不再返回 500。编辑对话框总是把 `schedule_spec` 与被修改的
   字段一起发送；当节奏没有变化时，`PATCH /api/scheduled-tasks/{id}` 会沿用任务原有的 `next_run_at`——
   而这个值是仓储以 ISO 字符串序列化后返回的。`ScheduledTaskRepository.update()` 原样把它赋给
@@ -5258,3 +5265,4 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写，核心包含子
 [#5945]: https://github.com/bytedance/deer-flow/pull/5945
 [#5964]: https://github.com/bytedance/deer-flow/pull/5964
 [#5981]: https://github.com/bytedance/deer-flow/pull/5981
+[#5982]: https://github.com/bytedance/deer-flow/pull/5982

frontend/src/AGENTS.md

@@ -167,7 +167,7 @@ Array previews coalesce consecutive generated markers only at the end into one o
 Model picker, chat entry pages, branch tree, and browser-control ownership are
 detailed in `frontend/docs/conversation-ui-ownership.md`.
 
-- `src/core/threads/hooks.ts` owns pre-submit upload state and thread submission.
+- `src/core/threads/hooks.ts` owns pre-submit upload state and thread submission. The optimistic human bubble (before and after upload) and the submitted message all build `additional_kwargs` through `buildHumanMessageAdditionalKwargs`, so quote/reference metadata and staged project files survive the upload; never replace them with only the uploaded files.
 - `src/components/workspace/chats/chat-box.tsx` owns the desktop right-panel layout, and **all three** right panels (artifacts, sidecar, browser) share one `ResizablePanelGroup` — do not fork a non-resizable branch per panel kind, which is how the artifacts divider silently lost its drag handle (#4465). Open/close is `collapse()` / `resize()` on the side panel's imperative handle, not conditional rendering, so the width can animate. Three constraints hold that together: the size transition is applied from the group as `[&>[data-panel]]:transition-[flex-grow]` because the sized flex item is the library's own `[data-panel]` element rather than the child `className` lands on; it is applied only while an open/close is in flight, so a drag is not interpolated frame by frame; and during the animation the panel content is held at its final width in `cqw` and clipped, because a reflowing message list re-runs its scroll-to-bottom (pinned by `tests/e2e/sidecar-chat.spec.ts`'s no-animated-scroll test) and a re-wrapping composer changes which responsive labels it shows. Because the panel is `collapsible`, the library can also collapse it to `0%` on its own when a drag crosses `minSize`, without going through the state that owns it. `onResize` records the last positive size while the pointer moves, but the owning `sidecar` / `browserView` / `artifactsOpen` state must only mirror a final `0%` layout from `onLayoutChanged`, after pointer release; closing on the first `0%` resize frame breaks a continuous drag that reaches the edge and then reverses before release.
 
 `ask_clarification` ToolMessages delimit runs despite hidden human replies; skill

frontend/src/core/threads/hooks.ts

@@ -32,7 +32,6 @@ import { isSidecarThread, SIDECAR_METADATA_KEY } from "../sidecar/thread";
 import { useSubtaskContext, useUpdateSubtask } from "../tasks/context";
 import { taskEventToSubtaskUpdate } from "../tasks/lifecycle";
 import { messageToStep } from "../tasks/steps";
-import type { UploadedFileInfo } from "../uploads";
 import { promptInputFilePartToFile, uploadFiles } from "../uploads";
 import { uuid } from "../utils/uuid";
 
@@ -172,6 +171,29 @@ export function hasToolResult(messages: Message[], toolName: string): boolean {
   );
 }
 
+/**
+ * `additional_kwargs` of this turn's visible human message. The optimistic
+ * display copy (before and after the upload) and the submitted message all
+ * build it here, so caller metadata such as quotes and conversation
+ * references stays on the bubble through the upload. Files staged out-of-band
+ * (e.g. a project document attached to this thread and carried in
+ * ``additionalKwargs.files``) ride alongside ``files`` instead of being
+ * overwritten by them.
+ */
+export function buildHumanMessageAdditionalKwargs(
+  additionalKwargs: Record<string, unknown> | undefined,
+  files: FileInMessage[],
+): Record<string, unknown> {
+  const stagedFiles = Array.isArray(additionalKwargs?.files)
+    ? (additionalKwargs.files as FileInMessage[])
+    : [];
+  const allFiles = [...stagedFiles, ...files];
+  return {
+    ...additionalKwargs,
+    ...(allFiles.length > 0 ? { files: allFiles } : {}),
+  };
+}
+
 export function buildThreadSubmitMessages({
   text,
   additionalKwargs,
@@ -190,13 +212,6 @@ export function buildThreadSubmitMessages({
    */
   humanMessageId?: string;
 }): Message[] {
-  // Files staged out-of-band (e.g. a project document attached to this
-  // thread and carried in ``additionalKwargs.files``) ride alongside the
-  // freshly uploaded files instead of being overwritten by them.
-  const stagedFiles = Array.isArray(additionalKwargs?.files)
-    ? (additionalKwargs.files as FileInMessage[])
-    : [];
-  const allFiles = [...stagedFiles, ...filesForSubmit];
   return [
     ...additionalInputMessages,
     {
@@ -208,10 +223,10 @@ export function buildThreadSubmitMessages({
           text,
         },
       ],
-      additional_kwargs: {
-        ...additionalKwargs,
-        ...(allFiles.length > 0 ? { files: allFiles } : {}),
-      },
+      additional_kwargs: buildHumanMessageAdditionalKwargs(
+        additionalKwargs,
+        filesForSubmit,
+      ),
     } as Message,
   ];
 }
@@ -2478,18 +2493,16 @@ export function useThreadStream({
         }),
       );
 
-      const optimisticAdditionalKwargs = {
-        ...options?.additionalKwargs,
-        ...(optimisticFiles.length > 0 ? { files: optimisticFiles } : {}),
-      };
-
       const newOptimistic: Message[] = [];
       if (!hideFromUI) {
         newOptimistic.push({
           type: "human",
           id: humanMessageId,
           content: text ? [{ type: "text", text }] : "",
-          additional_kwargs: optimisticAdditionalKwargs,
+          additional_kwargs: buildHumanMessageAdditionalKwargs(
+            options?.additionalKwargs,
+            optimisticFiles,
+          ),
         });
       }
 
@@ -2508,7 +2521,9 @@ export function useThreadStream({
 
       listeners.current.onSend?.(threadId);
 
-      let uploadedFileInfo: UploadedFileInfo[] = [];
+      // Shared by the optimistic bubble and the submit, so the files the
+      // user sees are exactly the files that are sent.
+      let uploadedFiles: FileInMessage[] = [];
 
       try {
         // Upload files first if any
@@ -2537,24 +2552,24 @@ export function useThreadStream({
 
             if (files.length > 0) {
               const uploadResponse = await uploadFiles(threadId, files);
-              uploadedFileInfo = uploadResponse.files;
+              uploadedFiles = uploadResponse.files.map((info) => ({
+                filename: info.filename,
+                size: info.size,
+                path: info.virtual_path,
+                status: "uploaded" as const,
+              }));
 
               // Update optimistic human message with uploaded status + paths
-              const uploadedFiles: FileInMessage[] = uploadedFileInfo.map(
-                (info) => ({
-                  filename: info.filename,
-                  size: info.size,
-                  path: info.virtual_path,
-                  status: "uploaded" as const,
-                }),
-              );
               setOptimisticMessages((messages) => {
                 if (messages.length > 1 && messages[0]) {
                   const humanMessage: Message = messages[0];
                   return [
                     {
                       ...humanMessage,
-                      additional_kwargs: { files: uploadedFiles },
+                      additional_kwargs: buildHumanMessageAdditionalKwargs(
+                        options?.additionalKwargs,
+                        uploadedFiles,
+                      ),
                     },
                     ...messages.slice(1),
                   ];
@@ -2577,23 +2592,13 @@ export function useThreadStream({
           }
         }
 
-        // Build files metadata for submission (included in additional_kwargs)
-        const filesForSubmit: FileInMessage[] = uploadedFileInfo.map(
-          (info) => ({
-            filename: info.filename,
-            size: info.size,
-            path: info.virtual_path,
-            status: "uploaded" as const,
-          }),
-        );
-
         await thread.submit(
           {
             messages: buildThreadSubmitMessages({
               text,
               additionalKwargs: options?.additionalKwargs,
               additionalInputMessages: options?.additionalInputMessages,
-              filesForSubmit,
+              filesForSubmit: uploadedFiles,
               humanMessageId,
             }),
           },

frontend/tests/unit/core/threads/optimistic-upload.dom.test.tsx

@@ -0,0 +1,223 @@
+import type { Message } from "@langchain/langgraph-sdk";
+import { afterEach, beforeEach, expect, rs, test } from "@rstest/core";
+import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
+import { act, renderHook } from "@testing-library/react";
+import { createElement, type ReactNode } from "react";
+
+import {
+  buildConversationReferenceMetadata,
+  readConversationReferences,
+} from "@/core/conversation-references";
+import { I18nContext } from "@/core/i18n/context";
+import { enUS } from "@/core/i18n/locales/en-US";
+import type { FileInMessage } from "@/core/messages/utils";
+import { DEFAULT_LOCAL_SETTINGS } from "@/core/settings/local";
+import {
+  buildReferenceMessageMetadata,
+  readReferenceMessageContexts,
+} from "@/core/sidecar/reference-metadata";
+import type { UploadResponse } from "@/core/uploads/api";
+
+const streamMockState = rs.hoisted(() => ({
+  submit: rs.fn(async () => undefined),
+}));
+
+rs.mock("@langchain/langgraph-sdk/react", () => ({
+  useStream: () => ({
+    isLoading: false,
+    messages: [],
+    stop: rs.fn(async () => undefined),
+    submit: streamMockState.submit,
+    values: { artifacts: [], messages: [], title: "", todos: [] },
+  }),
+}));
+
+const QUOTE = {
+  type: "referenced_message" as const,
+  label: "Earlier answer",
+  messageId: "ai-1",
+  role: "assistant" as const,
+  content: "The quoted text",
+};
+const CONVERSATION_REFERENCE = {
+  threadId: "thread-other",
+  title: "Another conversation",
+};
+const STAGED_FILE: FileInMessage = {
+  filename: "brief.md",
+  size: 12,
+  path: "/mnt/user-data/uploads/brief.md",
+  status: "uploaded",
+};
+const UPLOAD_RESPONSE: UploadResponse = {
+  success: true,
+  files: [
+    {
+      filename: "report.pdf",
+      size: 42,
+      path: "/data/uploads/report.pdf",
+      virtual_path: "/mnt/user-data/uploads/report.pdf",
+      artifact_url: "/api/threads/thread-1/artifacts/report.pdf",
+    },
+  ],
+  message: "ok",
+  skipped_files: [],
+};
+
+/**
+ * Hold the upload POST open until the test resolves it, so the optimistic
+ * bubble can be observed both while uploading and after the upload lands.
+ */
+function stubDeferredUploadFetch() {
+  let resolveUpload: (() => void) | undefined;
+  const uploadResponded = new Promise<void>((resolve) => {
+    resolveUpload = resolve;
+  });
+  rs.stubGlobal("fetch", async (input: unknown) => {
+    const url = String(input);
+    if (/\/api\/threads\/[^/]+\/uploads$/.test(url)) {
+      await uploadResponded;
+      return new Response(JSON.stringify(UPLOAD_RESPONSE), {
+        status: 200,
+        headers: { "Content-Type": "application/json" },
+      });
+    }
+    if (/\/api\/threads\/[^/]+\/messages\/page/.test(url)) {
+      return new Response(
+        JSON.stringify({ data: [], has_more: false, next_before_seq: null }),
+        { status: 200, headers: { "Content-Type": "application/json" } },
+      );
+    }
+    throw new Error(`Unexpected fetch: ${url}`);
+  });
+  return () => resolveUpload?.();
+}
+
+function createWrapper(queryClient: QueryClient) {
+  return function ThreadStreamTestWrapper({
+    children,
+  }: {
+    children: ReactNode;
+  }) {
+    return createElement(
+      QueryClientProvider,
+      { client: queryClient },
+      createElement(
+        I18nContext.Provider,
+        {
+          value: { locale: "en-US", setLocale: () => undefined, t: enUS },
+        },
+        children,
+      ),
+    );
+  };
+}
+
+async function flushFrames() {
+  for (let index = 0; index < 6; index += 1) {
+    await act(async () => undefined);
+  }
+}
+
+function displayedHuman(messages: Message[]): Message | undefined {
+  return messages.filter((message) => message.type === "human").at(-1);
+}
+
+beforeEach(() => {
+  streamMockState.submit.mockClear();
+});
+
+afterEach(() => {
+  rs.unstubAllGlobals();
+});
+
+test("keeps quote and conversation reference chips on the optimistic bubble through the upload", async () => {
+  const resolveUpload = stubDeferredUploadFetch();
+  const { useThreadStream } = await import("@/core/threads/hooks");
+  const { result } = renderHook(
+    () =>
+      useThreadStream({
+        context: DEFAULT_LOCAL_SETTINGS.context,
+        isMock: false,
+        threadId: "thread-1",
+      }),
+    {
+      wrapper: createWrapper(
+        new QueryClient({ defaultOptions: { queries: { retry: false } } }),
+      ),
+    },
+  );
+  await flushFrames();
+
+  let sendPromise: Promise<unknown> | undefined;
+  act(() => {
+    sendPromise = result.current.sendMessage(
+      "thread-1",
+      {
+        text: "Compare these",
+        files: [
+          {
+            type: "file",
+            url: "blob:report",
+            mediaType: "application/pdf",
+            filename: "report.pdf",
+            file: new File(["%PDF"], "report.pdf", {
+              type: "application/pdf",
+            }),
+          },
+        ],
+      },
+      undefined,
+      {
+        additionalKwargs: {
+          ...buildReferenceMessageMetadata([QUOTE]),
+          ...buildConversationReferenceMetadata([CONVERSATION_REFERENCE]),
+          files: [STAGED_FILE],
+        },
+      },
+    );
+  });
+  await flushFrames();
+
+  const uploading = displayedHuman(result.current.thread.messages);
+  expect(readReferenceMessageContexts(uploading?.additional_kwargs)).toEqual([
+    QUOTE,
+  ]);
+  expect(readConversationReferences(uploading?.additional_kwargs)).toEqual([
+    CONVERSATION_REFERENCE,
+  ]);
+  expect(uploading?.additional_kwargs?.files).toEqual([
+    STAGED_FILE,
+    { filename: "report.pdf", size: 0, status: "uploading" },
+  ]);
+
+  resolveUpload();
+  await act(async () => {
+    await sendPromise;
+  });
+
+  // The server has not echoed the human yet, so the optimistic copy is
+  // still the bubble on screen.
+  const uploaded = displayedHuman(result.current.thread.messages);
+  const submitCalls = streamMockState.submit.mock.calls as unknown as Array<
+    [{ messages: Message[] }]
+  >;
+  const submitted = submitCalls.at(-1)?.[0].messages.at(-1);
+  expect(uploaded?.id).toBe(submitted?.id);
+  expect(uploaded?.additional_kwargs).toEqual(submitted?.additional_kwargs);
+  expect(readReferenceMessageContexts(uploaded?.additional_kwargs)).toEqual([
+    QUOTE,
+  ]);
+  expect(readConversationReferences(uploaded?.additional_kwargs)).toEqual([
+    CONVERSATION_REFERENCE,
+  ]);
+  expect(uploaded?.additional_kwargs?.files).toEqual([
+    STAGED_FILE,
+    {
+      filename: "report.pdf",
+      size: 42,
+      path: "/mnt/user-data/uploads/report.pdf",
+      status: "uploaded",
+    },
+  ]);
+});