diff --git a/backend/docs/FILE_UPLOAD.md b/backend/docs/FILE_UPLOAD.md index 3bcd99947..778b57210 100644 --- a/backend/docs/FILE_UPLOAD.md +++ b/backend/docs/FILE_UPLOAD.md @@ -9,7 +9,7 @@ DeerFlow 后端提供了完整的文件上传功能,支持多文件上传, - ✅ 支持多文件同时上传 - ✅ 可选地转换文档为 Markdown(PDF、PPT、Excel、Word) - ✅ 文件存储在线程隔离的目录中 -- ✅ Agent 自动感知已上传的文件 +- ✅ Agent 自动感知当前消息中附带的文件 - ✅ 支持文件列表查询和删除 ## API 端点 @@ -116,24 +116,30 @@ DELETE /api/threads/{thread_id}/uploads/{filename} ## Agent 集成 -### 自动文件列举 +### 当前消息中的文件上下文 -Agent 在每次请求时会自动收到已上传文件的列表,格式如下: +发送消息时,前端会把该消息附带的上传文件元数据放入 +`HumanMessage.additional_kwargs.files`。`UploadsMiddleware` 只把当前消息中的文件 +注入 Agent 上下文,格式如下: ```xml - -The following files have been uploaded and are available for use: + +The following files were uploaded in this message: - document.pdf (1.2 MB) Path: /mnt/user-data/uploads/document.pdf -- document.md (45.3 KB) - Path: /mnt/user-data/uploads/document.md - -You can read these files using the `read_file` tool with the paths shown above. - +To work with these files: +- Read from the file first — use the outline line numbers and `read_file` to locate relevant sections. +- Use `grep` to search for keywords when you are not sure which section to look at. +- Use `glob` to find files by name pattern. + ``` +以前轮次上传的文件不会在每次请求中重复注入。Agent 可按需调用 +`list_uploaded_files` 查询历史上传;如果已知文件名,也可直接使用 +`read_file` 或 `grep` 访问 `/mnt/user-data/uploads/` 下的文件。 + ### 使用上传的文件 Agent 在沙箱中运行,使用虚拟路径访问文件。Agent 可以直接使用 `read_file` 工具读取上传的文件: @@ -240,8 +246,9 @@ backend/.deer-flow/threads/ - 使用 markitdown 转换文档 2. **Uploads Middleware** (`packages/harness/deerflow/agents/middlewares/uploads_middleware.py`) - - 在每次 Agent 请求前注入文件列表 - - 自动生成格式化的文件列表消息 + - 读取当前消息的 `additional_kwargs.files` + - 在 Agent 请求前生成并注入 `` 文件上下文 + - 历史上传由 `list_uploaded_files` 按需查询,不会每轮自动注入 3. **Nginx 配置** (`nginx.conf`) - 路由上传请求到 Gateway API diff --git a/backend/docs/IM_CHANNEL_CONNECTIONS.md b/backend/docs/IM_CHANNEL_CONNECTIONS.md index 06107cd6a..c49803894 100644 --- a/backend/docs/IM_CHANNEL_CONNECTIONS.md +++ b/backend/docs/IM_CHANNEL_CONNECTIONS.md @@ -208,7 +208,15 @@ The cached value is reused across the blocking (`runs.wait`) and streaming (`_ha ## IM File Attachment Pipeline -Inbound files (images, documents) walk through `Channel.receive_file` for materialization, then `_ingest_inbound_files` for owner-bound staging. The agent sees the staged path via the `` block injected into its context. +Inbound files (images, documents) first pass through `Channel.receive_file` for +provider-specific materialization. Attachments that continue through the shared +metadata path are staged by `_ingest_inbound_files`; their metadata is placed in +`HumanMessage.additional_kwargs.files`, and `UploadsMiddleware` injects a +`` block for the current message. Some providers instead consume +their descriptors while downloading and rewrite placeholders or message text with +the resulting virtual path (or a failure notice). Historical uploads are not +automatically injected on later turns; the agent discovers them with +`list_uploaded_files`. ```mermaid sequenceDiagram @@ -218,17 +226,24 @@ sequenceDiagram participant Mgr as ChannelManager participant Ch as Channel impl
.receive_file participant FS as Uploads directory
users/OWNER/.../uploads/ + participant MW as UploadsMiddleware participant Agent as Agent run IM->>Worker: message with file URL/bytes Worker->>Mgr: InboundMessage(files=[...], connection_id, owner_user_id) Mgr->>Mgr: storage_user_id = _channel_storage_user_id(msg) Mgr->>Ch: receive_file(msg, thread_id, user_id=storage_user_id) - Note over Ch: provider-specific download
(WeCom: decrypt_file;
WeChat: read_bytes; others: HTTP GET) - Ch->>FS: write_upload_file_no_symlink(
uploads/OWNER/.../
, safe_name, data) - Ch-->>Mgr: msg with text rewritten to include - Mgr->>Mgr: _ingest_inbound_files(
thread_id, msg, user_id=storage_user_id) - Mgr->>Agent: HumanMessage with block
(paths under /mnt/user-data/uploads/) + Note over Ch: provider-specific download/decrypt/read;
may persist or hand bytes to manager + Ch-->>Mgr: materialized message
(provider may rewrite placeholders/text) + alt attachment continues through shared metadata path + Mgr->>FS: _ingest_inbound_files(
thread_id, msg, user_id=storage_user_id) + FS-->>Mgr: uploaded file metadata + Mgr->>MW: HumanMessage with
additional_kwargs.files + MW->>Agent: prepend
(paths under /mnt/user-data/uploads/) + else provider supplies virtual path in message text + Ch->>FS: persist and/or sync attachment + Mgr->>Agent: HumanMessage with rewritten path
or failure notice + end Agent->>FS: read_file / view_image (sandbox) ```