mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-04 11:58:36 +00:00
docs(uploads): document current upload context (#4632)
This commit is contained in:
parent
540940bac1
commit
cd8825b0f0
@ -9,7 +9,7 @@ DeerFlow 后端提供了完整的文件上传功能,支持多文件上传,
|
|||||||
- ✅ 支持多文件同时上传
|
- ✅ 支持多文件同时上传
|
||||||
- ✅ 可选地转换文档为 Markdown(PDF、PPT、Excel、Word)
|
- ✅ 可选地转换文档为 Markdown(PDF、PPT、Excel、Word)
|
||||||
- ✅ 文件存储在线程隔离的目录中
|
- ✅ 文件存储在线程隔离的目录中
|
||||||
- ✅ Agent 自动感知已上传的文件
|
- ✅ Agent 自动感知当前消息中附带的文件
|
||||||
- ✅ 支持文件列表查询和删除
|
- ✅ 支持文件列表查询和删除
|
||||||
|
|
||||||
## API 端点
|
## API 端点
|
||||||
@ -116,24 +116,30 @@ DELETE /api/threads/{thread_id}/uploads/{filename}
|
|||||||
|
|
||||||
## Agent 集成
|
## Agent 集成
|
||||||
|
|
||||||
### 自动文件列举
|
### 当前消息中的文件上下文
|
||||||
|
|
||||||
Agent 在每次请求时会自动收到已上传文件的列表,格式如下:
|
发送消息时,前端会把该消息附带的上传文件元数据放入
|
||||||
|
`HumanMessage.additional_kwargs.files`。`UploadsMiddleware` 只把当前消息中的文件
|
||||||
|
注入 Agent 上下文,格式如下:
|
||||||
|
|
||||||
```xml
|
```xml
|
||||||
<uploaded_files>
|
<current_uploads>
|
||||||
The following files have been uploaded and are available for use:
|
The following files were uploaded in this message:
|
||||||
|
|
||||||
- document.pdf (1.2 MB)
|
- document.pdf (1.2 MB)
|
||||||
Path: /mnt/user-data/uploads/document.pdf
|
Path: /mnt/user-data/uploads/document.pdf
|
||||||
|
|
||||||
- document.md (45.3 KB)
|
To work with these files:
|
||||||
Path: /mnt/user-data/uploads/document.md
|
- Read from the file first — use the outline line numbers and `read_file` to locate relevant sections.
|
||||||
|
- Use `grep` to search for keywords when you are not sure which section to look at.
|
||||||
You can read these files using the `read_file` tool with the paths shown above.
|
- Use `glob` to find files by name pattern.
|
||||||
</uploaded_files>
|
</current_uploads>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
以前轮次上传的文件不会在每次请求中重复注入。Agent 可按需调用
|
||||||
|
`list_uploaded_files` 查询历史上传;如果已知文件名,也可直接使用
|
||||||
|
`read_file` 或 `grep` 访问 `/mnt/user-data/uploads/` 下的文件。
|
||||||
|
|
||||||
### 使用上传的文件
|
### 使用上传的文件
|
||||||
|
|
||||||
Agent 在沙箱中运行,使用虚拟路径访问文件。Agent 可以直接使用 `read_file` 工具读取上传的文件:
|
Agent 在沙箱中运行,使用虚拟路径访问文件。Agent 可以直接使用 `read_file` 工具读取上传的文件:
|
||||||
@ -240,8 +246,9 @@ backend/.deer-flow/threads/
|
|||||||
- 使用 markitdown 转换文档
|
- 使用 markitdown 转换文档
|
||||||
|
|
||||||
2. **Uploads Middleware** (`packages/harness/deerflow/agents/middlewares/uploads_middleware.py`)
|
2. **Uploads Middleware** (`packages/harness/deerflow/agents/middlewares/uploads_middleware.py`)
|
||||||
- 在每次 Agent 请求前注入文件列表
|
- 读取当前消息的 `additional_kwargs.files`
|
||||||
- 自动生成格式化的文件列表消息
|
- 在 Agent 请求前生成并注入 `<current_uploads>` 文件上下文
|
||||||
|
- 历史上传由 `list_uploaded_files` 按需查询,不会每轮自动注入
|
||||||
|
|
||||||
3. **Nginx 配置** (`nginx.conf`)
|
3. **Nginx 配置** (`nginx.conf`)
|
||||||
- 路由上传请求到 Gateway API
|
- 路由上传请求到 Gateway API
|
||||||
|
|||||||
@ -208,7 +208,15 @@ The cached value is reused across the blocking (`runs.wait`) and streaming (`_ha
|
|||||||
|
|
||||||
## IM File Attachment Pipeline
|
## IM File Attachment Pipeline
|
||||||
|
|
||||||
Inbound files (images, documents) walk through `Channel.receive_file` for materialization, then `_ingest_inbound_files` for owner-bound staging. The agent sees the staged path via the `<uploaded_files>` block injected into its context.
|
Inbound files (images, documents) first pass through `Channel.receive_file` for
|
||||||
|
provider-specific materialization. Attachments that continue through the shared
|
||||||
|
metadata path are staged by `_ingest_inbound_files`; their metadata is placed in
|
||||||
|
`HumanMessage.additional_kwargs.files`, and `UploadsMiddleware` injects a
|
||||||
|
`<current_uploads>` block for the current message. Some providers instead consume
|
||||||
|
their descriptors while downloading and rewrite placeholders or message text with
|
||||||
|
the resulting virtual path (or a failure notice). Historical uploads are not
|
||||||
|
automatically injected on later turns; the agent discovers them with
|
||||||
|
`list_uploaded_files`.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
sequenceDiagram
|
sequenceDiagram
|
||||||
@ -218,17 +226,24 @@ sequenceDiagram
|
|||||||
participant Mgr as ChannelManager
|
participant Mgr as ChannelManager
|
||||||
participant Ch as Channel impl<br/>.receive_file
|
participant Ch as Channel impl<br/>.receive_file
|
||||||
participant FS as Uploads directory<br/>users/OWNER/.../uploads/
|
participant FS as Uploads directory<br/>users/OWNER/.../uploads/
|
||||||
|
participant MW as UploadsMiddleware
|
||||||
participant Agent as Agent run
|
participant Agent as Agent run
|
||||||
|
|
||||||
IM->>Worker: message with file URL/bytes
|
IM->>Worker: message with file URL/bytes
|
||||||
Worker->>Mgr: InboundMessage(files=[...], connection_id, owner_user_id)
|
Worker->>Mgr: InboundMessage(files=[...], connection_id, owner_user_id)
|
||||||
Mgr->>Mgr: storage_user_id = _channel_storage_user_id(msg)
|
Mgr->>Mgr: storage_user_id = _channel_storage_user_id(msg)
|
||||||
Mgr->>Ch: receive_file(msg, thread_id, user_id=storage_user_id)
|
Mgr->>Ch: receive_file(msg, thread_id, user_id=storage_user_id)
|
||||||
Note over Ch: provider-specific download<br/>(WeCom: decrypt_file;<br/>WeChat: read_bytes; others: HTTP GET)
|
Note over Ch: provider-specific download/decrypt/read;<br/>may persist or hand bytes to manager
|
||||||
Ch->>FS: write_upload_file_no_symlink(<br/>uploads/OWNER/.../<br/>, safe_name, data)
|
Ch-->>Mgr: materialized message<br/>(provider may rewrite placeholders/text)
|
||||||
Ch-->>Mgr: msg with text rewritten to include <uploaded_files>
|
alt attachment continues through shared metadata path
|
||||||
Mgr->>Mgr: _ingest_inbound_files(<br/>thread_id, msg, user_id=storage_user_id)
|
Mgr->>FS: _ingest_inbound_files(<br/>thread_id, msg, user_id=storage_user_id)
|
||||||
Mgr->>Agent: HumanMessage with <uploaded_files> block<br/>(paths under /mnt/user-data/uploads/)
|
FS-->>Mgr: uploaded file metadata
|
||||||
|
Mgr->>MW: HumanMessage with<br/>additional_kwargs.files
|
||||||
|
MW->>Agent: prepend <current_uploads><br/>(paths under /mnt/user-data/uploads/)
|
||||||
|
else provider supplies virtual path in message text
|
||||||
|
Ch->>FS: persist and/or sync attachment
|
||||||
|
Mgr->>Agent: HumanMessage with rewritten path<br/>or failure notice
|
||||||
|
end
|
||||||
Agent->>FS: read_file / view_image (sandbox)
|
Agent->>FS: read_file / view_image (sandbox)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user