mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-04 03:49:25 +00:00
docs(uploads): document current upload context (#4632)
This commit is contained in:
parent
540940bac1
commit
cd8825b0f0
@ -9,7 +9,7 @@ DeerFlow 后端提供了完整的文件上传功能,支持多文件上传,
|
||||
- ✅ 支持多文件同时上传
|
||||
- ✅ 可选地转换文档为 Markdown(PDF、PPT、Excel、Word)
|
||||
- ✅ 文件存储在线程隔离的目录中
|
||||
- ✅ Agent 自动感知已上传的文件
|
||||
- ✅ Agent 自动感知当前消息中附带的文件
|
||||
- ✅ 支持文件列表查询和删除
|
||||
|
||||
## API 端点
|
||||
@ -116,24 +116,30 @@ DELETE /api/threads/{thread_id}/uploads/{filename}
|
||||
|
||||
## Agent 集成
|
||||
|
||||
### 自动文件列举
|
||||
### 当前消息中的文件上下文
|
||||
|
||||
Agent 在每次请求时会自动收到已上传文件的列表,格式如下:
|
||||
发送消息时,前端会把该消息附带的上传文件元数据放入
|
||||
`HumanMessage.additional_kwargs.files`。`UploadsMiddleware` 只把当前消息中的文件
|
||||
注入 Agent 上下文,格式如下:
|
||||
|
||||
```xml
|
||||
<uploaded_files>
|
||||
The following files have been uploaded and are available for use:
|
||||
<current_uploads>
|
||||
The following files were uploaded in this message:
|
||||
|
||||
- document.pdf (1.2 MB)
|
||||
Path: /mnt/user-data/uploads/document.pdf
|
||||
|
||||
- document.md (45.3 KB)
|
||||
Path: /mnt/user-data/uploads/document.md
|
||||
|
||||
You can read these files using the `read_file` tool with the paths shown above.
|
||||
</uploaded_files>
|
||||
To work with these files:
|
||||
- Read from the file first — use the outline line numbers and `read_file` to locate relevant sections.
|
||||
- Use `grep` to search for keywords when you are not sure which section to look at.
|
||||
- Use `glob` to find files by name pattern.
|
||||
</current_uploads>
|
||||
```
|
||||
|
||||
以前轮次上传的文件不会在每次请求中重复注入。Agent 可按需调用
|
||||
`list_uploaded_files` 查询历史上传;如果已知文件名,也可直接使用
|
||||
`read_file` 或 `grep` 访问 `/mnt/user-data/uploads/` 下的文件。
|
||||
|
||||
### 使用上传的文件
|
||||
|
||||
Agent 在沙箱中运行,使用虚拟路径访问文件。Agent 可以直接使用 `read_file` 工具读取上传的文件:
|
||||
@ -240,8 +246,9 @@ backend/.deer-flow/threads/
|
||||
- 使用 markitdown 转换文档
|
||||
|
||||
2. **Uploads Middleware** (`packages/harness/deerflow/agents/middlewares/uploads_middleware.py`)
|
||||
- 在每次 Agent 请求前注入文件列表
|
||||
- 自动生成格式化的文件列表消息
|
||||
- 读取当前消息的 `additional_kwargs.files`
|
||||
- 在 Agent 请求前生成并注入 `<current_uploads>` 文件上下文
|
||||
- 历史上传由 `list_uploaded_files` 按需查询,不会每轮自动注入
|
||||
|
||||
3. **Nginx 配置** (`nginx.conf`)
|
||||
- 路由上传请求到 Gateway API
|
||||
|
||||
@ -208,7 +208,15 @@ The cached value is reused across the blocking (`runs.wait`) and streaming (`_ha
|
||||
|
||||
## IM File Attachment Pipeline
|
||||
|
||||
Inbound files (images, documents) walk through `Channel.receive_file` for materialization, then `_ingest_inbound_files` for owner-bound staging. The agent sees the staged path via the `<uploaded_files>` block injected into its context.
|
||||
Inbound files (images, documents) first pass through `Channel.receive_file` for
|
||||
provider-specific materialization. Attachments that continue through the shared
|
||||
metadata path are staged by `_ingest_inbound_files`; their metadata is placed in
|
||||
`HumanMessage.additional_kwargs.files`, and `UploadsMiddleware` injects a
|
||||
`<current_uploads>` block for the current message. Some providers instead consume
|
||||
their descriptors while downloading and rewrite placeholders or message text with
|
||||
the resulting virtual path (or a failure notice). Historical uploads are not
|
||||
automatically injected on later turns; the agent discovers them with
|
||||
`list_uploaded_files`.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
@ -218,17 +226,24 @@ sequenceDiagram
|
||||
participant Mgr as ChannelManager
|
||||
participant Ch as Channel impl<br/>.receive_file
|
||||
participant FS as Uploads directory<br/>users/OWNER/.../uploads/
|
||||
participant MW as UploadsMiddleware
|
||||
participant Agent as Agent run
|
||||
|
||||
IM->>Worker: message with file URL/bytes
|
||||
Worker->>Mgr: InboundMessage(files=[...], connection_id, owner_user_id)
|
||||
Mgr->>Mgr: storage_user_id = _channel_storage_user_id(msg)
|
||||
Mgr->>Ch: receive_file(msg, thread_id, user_id=storage_user_id)
|
||||
Note over Ch: provider-specific download<br/>(WeCom: decrypt_file;<br/>WeChat: read_bytes; others: HTTP GET)
|
||||
Ch->>FS: write_upload_file_no_symlink(<br/>uploads/OWNER/.../<br/>, safe_name, data)
|
||||
Ch-->>Mgr: msg with text rewritten to include <uploaded_files>
|
||||
Mgr->>Mgr: _ingest_inbound_files(<br/>thread_id, msg, user_id=storage_user_id)
|
||||
Mgr->>Agent: HumanMessage with <uploaded_files> block<br/>(paths under /mnt/user-data/uploads/)
|
||||
Note over Ch: provider-specific download/decrypt/read;<br/>may persist or hand bytes to manager
|
||||
Ch-->>Mgr: materialized message<br/>(provider may rewrite placeholders/text)
|
||||
alt attachment continues through shared metadata path
|
||||
Mgr->>FS: _ingest_inbound_files(<br/>thread_id, msg, user_id=storage_user_id)
|
||||
FS-->>Mgr: uploaded file metadata
|
||||
Mgr->>MW: HumanMessage with<br/>additional_kwargs.files
|
||||
MW->>Agent: prepend <current_uploads><br/>(paths under /mnt/user-data/uploads/)
|
||||
else provider supplies virtual path in message text
|
||||
Ch->>FS: persist and/or sync attachment
|
||||
Mgr->>Agent: HumanMessage with rewritten path<br/>or failure notice
|
||||
end
|
||||
Agent->>FS: read_file / view_image (sandbox)
|
||||
```
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user