mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-10 14:08:52 +00:00
fix(feishu): preserve inbound attachment files (#4903)
* fix(feishu): preserve inbound attachment files * fix(feishu): harden inbound attachment handling * fix(dingtalk): reserve symlinked inbound filenames
This commit is contained in:
parent
74d9e6c2e0
commit
e5bf3ccf45
@ -654,6 +654,7 @@ DINGTALK_CLIENT_SECRET=your_client_secret
|
|||||||
2. Add permissions: `im:message`, `im:message.p2p_msg:readonly`, `im:resource`.
|
2. Add permissions: `im:message`, `im:message.p2p_msg:readonly`, `im:resource`.
|
||||||
3. Under **Events**, subscribe to `im.message.receive_v1` and select **Long Connection** mode.
|
3. Under **Events**, subscribe to `im.message.receive_v1` and select **Long Connection** mode.
|
||||||
4. Copy the App ID and App Secret. Set `FEISHU_APP_ID` and `FEISHU_APP_SECRET` in `.env` and enable the channel in `config.yaml`.
|
4. Copy the App ID and App Secret. Set `FEISHU_APP_ID` and `FEISHU_APP_SECRET` in `.env` and enable the channel in `config.yaml`.
|
||||||
|
5. The bot supports inbound text, image, and file messages. Inbound attachment downloads are limited to 20 MB per attachment.
|
||||||
|
|
||||||
**WeChat Setup**
|
**WeChat Setup**
|
||||||
|
|
||||||
|
|||||||
@ -22,6 +22,7 @@ Bridges external messaging platforms (Feishu, Slack, Telegram, Discord, DingTalk
|
|||||||
1. External platform -> Channel impl -> `MessageBus.publish_inbound()`
|
1. External platform -> Channel impl -> `MessageBus.publish_inbound()`
|
||||||
- For GitHub, the webhook router verifies the delivery then calls `fanout_event(bus, ...)`; matching agent bindings publish one `InboundMessage` each instead of a long-polling channel worker.
|
- For GitHub, the webhook router verifies the delivery then calls `fanout_event(bus, ...)`; matching agent bindings publish one `InboundMessage` each instead of a long-polling channel worker.
|
||||||
- Telegram photo/document updates use the largest photo size or document metadata, preserve `message.caption`, enforce the hosted Bot API's 20,000,000-byte download ceiling before and after download, and never expose the token-bearing Bot API file URL. Downloaded bytes cross the adapter/manager boundary only through `message_bus.INBOUND_FILE_CONTENT_KEY`; the manager consumes that transient field before persisting safe upload metadata.
|
- Telegram photo/document updates use the largest photo size or document metadata, preserve `message.caption`, enforce the hosted Bot API's 20,000,000-byte download ceiling before and after download, and never expose the token-bearing Bot API file URL. Downloaded bytes cross the adapter/manager boundary only through `message_bus.INBOUND_FILE_CONTENT_KEY`; the manager consumes that transient field before persisting safe upload metadata.
|
||||||
|
- Feishu/Lark inbound image/file downloads read at most 20,000,001 bytes and reject anything above 20,000,000 bytes before persistence or sandbox sync. Oversize, unsafe-path, and path-resolution failures rewrite only that attachment placeholder to `Failed to obtain the [type]` so later attachments in the same message can still load.
|
||||||
2. `ChannelManager._dispatch_loop()` consumes from queue
|
2. `ChannelManager._dispatch_loop()` consumes from queue
|
||||||
3. For user-owned channel connections, incoming messages carry `connection_id`, `owner_user_id`, and `workspace_id`; `owner_user_id` becomes the DeerFlow run `user_id`, while the raw platform user id remains `channel_user_id`. The Gateway accepts `channel_user_id` only from an internally authenticated channel caller's top-level `body.context`, clears it from both free-form `body.config` sections, and writes it into runtime context only (never `configurable`, which is checkpointed). `bash_tool` exposes it to sandbox commands as the fixed env var `DEERFLOW_CHANNEL_USER_ID` — via a shell-quoted command-string prefix, NOT the `execute_command(env=...)` channel, which is reserved for request-scoped secrets and would switch `AioSandbox` onto the `bash.exec` path (image >= 1.9.3, fresh session per call). Per-call injection keeps group-chat identity correct (one thread/sandbox, many senders) **without depending on the AIO shell's session semantics**: every IM-channel command carries an explicit `export VAR=<id>; ` (valid id) or `unset VAR; ` (empty / non-str / over the 256-char cap). The AIO no-env path reuses a persistent shell session (the reason for the class lock, #1433), so a bare command could otherwise resolve a stale id an earlier sender exported; the `unset` closes the window the length/type guard would open (a dropped id would inherit the previous sender's value). Non-IM runs (no `channel_user_id` in context) are left untouched. Not injected on the Windows local sandbox (its PowerShell/cmd.exe fallback has no `export`/`unset`). Propagates across `task` delegation: `task_tool` captures the dispatching turn's id and the subagent executor forwards it into the subagent's runtime context, same as the guardrail attribution fields. The runtime-context value is authorization-grade at the Gateway/guardrail boundary, but the exported shell variable remains informational because any bash command can overwrite its own environment; skills must not treat the shell variable itself as authenticated identity. Tests: `tests/test_gateway_services.py`, `tests/test_channel_user_id_env.py`
|
3. For user-owned channel connections, incoming messages carry `connection_id`, `owner_user_id`, and `workspace_id`; `owner_user_id` becomes the DeerFlow run `user_id`, while the raw platform user id remains `channel_user_id`. The Gateway accepts `channel_user_id` only from an internally authenticated channel caller's top-level `body.context`, clears it from both free-form `body.config` sections, and writes it into runtime context only (never `configurable`, which is checkpointed). `bash_tool` exposes it to sandbox commands as the fixed env var `DEERFLOW_CHANNEL_USER_ID` — via a shell-quoted command-string prefix, NOT the `execute_command(env=...)` channel, which is reserved for request-scoped secrets and would switch `AioSandbox` onto the `bash.exec` path (image >= 1.9.3, fresh session per call). Per-call injection keeps group-chat identity correct (one thread/sandbox, many senders) **without depending on the AIO shell's session semantics**: every IM-channel command carries an explicit `export VAR=<id>; ` (valid id) or `unset VAR; ` (empty / non-str / over the 256-char cap). The AIO no-env path reuses a persistent shell session (the reason for the class lock, #1433), so a bare command could otherwise resolve a stale id an earlier sender exported; the `unset` closes the window the length/type guard would open (a dropped id would inherit the previous sender's value). Non-IM runs (no `channel_user_id` in context) are left untouched. Not injected on the Windows local sandbox (its PowerShell/cmd.exe fallback has no `export`/`unset`). Propagates across `task` delegation: `task_tool` captures the dispatching turn's id and the subagent executor forwards it into the subagent's runtime context, same as the guardrail attribution fields. The runtime-context value is authorization-grade at the Gateway/guardrail boundary, but the exported shell variable remains informational because any bash command can overwrite its own environment; skills must not treat the shell variable itself as authenticated identity. Tests: `tests/test_gateway_services.py`, `tests/test_channel_user_id_env.py`
|
||||||
4. For chat: look up/create thread through Gateway's LangGraph-compatible API
|
4. For chat: look up/create thread through Gateway's LangGraph-compatible API
|
||||||
@ -37,6 +38,7 @@ Bridges external messaging platforms (Feishu, Slack, Telegram, Discord, DingTalk
|
|||||||
|
|
||||||
**Owner-scoped file storage**: inbound files, uploads, and output artifacts are staged under the DeerFlow owner's bucket so they land where the agent run reads/writes (`users/{user_id}/threads/{thread_id}/user-data/{uploads,outputs}`). `ChannelManager._handle_chat` resolves the storage owner once via `_channel_storage_user_id(msg)` (sanitized owner id, falling back to `safe(msg.user_id)` for unbound auth-enabled channels — mirroring `_resolve_run_params`'s run identity; `None` only when no identity is available) and threads it as the `user_id=` kwarg through the file pipeline:
|
**Owner-scoped file storage**: inbound files, uploads, and output artifacts are staged under the DeerFlow owner's bucket so they land where the agent run reads/writes (`users/{user_id}/threads/{thread_id}/user-data/{uploads,outputs}`). `ChannelManager._handle_chat` resolves the storage owner once via `_channel_storage_user_id(msg)` (sanitized owner id, falling back to `safe(msg.user_id)` for unbound auth-enabled channels — mirroring `_resolve_run_params`'s run identity; `None` only when no identity is available) and threads it as the `user_id=` kwarg through the file pipeline:
|
||||||
- `Channel.receive_file(msg, thread_id, user_id=...)` — owner-bound channels persist downloaded files under the owner's bucket instead of the default bucket
|
- `Channel.receive_file(msg, thread_id, user_id=...)` — owner-bound channels persist downloaded files under the owner's bucket instead of the default bucket
|
||||||
|
- `FeishuChannel._receive_single_file(...)` / `DingTalkChannel._receive_single_file(...)` — normalize provider filenames, claim a collision-free basename and write it through `write_upload_file_no_symlink` under the same channel lock; the returned basename drives both the agent-visible virtual path and non-local sandbox sync
|
||||||
- `_ingest_inbound_files(...)` and the underlying `ensure_uploads_dir` / `get_uploads_dir` — owner-scoped via the same kwarg
|
- `_ingest_inbound_files(...)` and the underlying `ensure_uploads_dir` / `get_uploads_dir` — owner-scoped via the same kwarg
|
||||||
- `_resolve_attachments` / `_prepare_artifact_delivery` — resolve output artifacts from the bound owner's bucket
|
- `_resolve_attachments` / `_prepare_artifact_delivery` — resolve output artifacts from the bound owner's bucket
|
||||||
The cached value is reused for both the blocking (`runs.wait`) and streaming (`_handle_streaming_chat`) paths, so uploads and artifact delivery always target the same bucket even if a channel returns a rewritten `InboundMessage` from `receive_file`. The bucket id matches the memory bucket resolved by `_resolve_memory_user_id` (both normalize through `make_safe_user_id`).
|
The cached value is reused for both the blocking (`runs.wait`) and streaming (`_handle_streaming_chat`) paths, so uploads and artifact delivery always target the same bucket even if a channel returns a rewritten `InboundMessage` from `receive_file`. The bucket id matches the memory bucket resolved by `_resolve_memory_user_id` (both normalize through `make_safe_user_id`).
|
||||||
|
|||||||
@ -659,7 +659,7 @@ class DingTalkChannel(Channel):
|
|||||||
paths.ensure_thread_dirs(thread_id, user_id=effective_user_id)
|
paths.ensure_thread_dirs(thread_id, user_id=effective_user_id)
|
||||||
uploads_dir = paths.sandbox_uploads_dir(thread_id, user_id=effective_user_id).resolve()
|
uploads_dir = paths.sandbox_uploads_dir(thread_id, user_id=effective_user_id).resolve()
|
||||||
with self._file_write_lock:
|
with self._file_write_lock:
|
||||||
seen = {entry.name for entry in uploads_dir.iterdir() if entry.is_file()}
|
seen = {entry.name for entry in uploads_dir.iterdir()}
|
||||||
unique_name = claim_unique_filename(safe_filename, seen)
|
unique_name = claim_unique_filename(safe_filename, seen)
|
||||||
# write_upload_file_no_symlink refuses a symlinked destination:
|
# write_upload_file_no_symlink refuses a symlinked destination:
|
||||||
# uploads dirs can be mounted into local sandboxes, so a sandbox
|
# uploads dirs can be mounted into local sandboxes, so a sandbox
|
||||||
|
|||||||
@ -26,10 +26,12 @@ from app.channels.message_bus import (
|
|||||||
from deerflow.config.paths import VIRTUAL_PATH_PREFIX, get_paths
|
from deerflow.config.paths import VIRTUAL_PATH_PREFIX, get_paths
|
||||||
from deerflow.runtime.user_context import get_effective_user_id
|
from deerflow.runtime.user_context import get_effective_user_id
|
||||||
from deerflow.sandbox.sandbox_provider import get_sandbox_provider
|
from deerflow.sandbox.sandbox_provider import get_sandbox_provider
|
||||||
|
from deerflow.uploads.manager import claim_unique_filename, normalize_filename, write_upload_file_no_symlink
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
PENDING_CLARIFICATION_TTL_SECONDS = 30 * 60
|
PENDING_CLARIFICATION_TTL_SECONDS = 30 * 60
|
||||||
FEISHU_INBOUND_BATCH_WINDOW_SECONDS = 0.75
|
FEISHU_INBOUND_BATCH_WINDOW_SECONDS = 0.75
|
||||||
|
FEISHU_MAX_INBOUND_FILE_BYTES = 20_000_000
|
||||||
SOURCE_PREVIEW_METADATA_KEY = "feishu_source_preview"
|
SOURCE_PREVIEW_METADATA_KEY = "feishu_source_preview"
|
||||||
|
|
||||||
|
|
||||||
@ -382,13 +384,23 @@ class FeishuChannel(Channel):
|
|||||||
logger.warning("[Feishu] received message with no files: %s", msg)
|
logger.warning("[Feishu] received message with no files: %s", msg)
|
||||||
return msg
|
return msg
|
||||||
text = msg.text
|
text = msg.text
|
||||||
|
search_from = 0
|
||||||
|
|
||||||
|
def replace_next_placeholder(placeholder: str, replacement: str) -> None:
|
||||||
|
nonlocal search_from, text
|
||||||
|
idx = text.find(placeholder, search_from)
|
||||||
|
if idx < 0:
|
||||||
|
return
|
||||||
|
text = f"{text[:idx]}{replacement}{text[idx + len(placeholder) :]}"
|
||||||
|
search_from = idx + len(replacement)
|
||||||
|
|
||||||
for file in files:
|
for file in files:
|
||||||
if file.get("image_key"):
|
if file.get("image_key"):
|
||||||
virtual_path = await self._receive_single_file(msg.thread_ts, file["image_key"], "image", thread_id, user_id=user_id)
|
virtual_path = await self._receive_single_file(msg.thread_ts, file["image_key"], "image", thread_id, user_id=user_id)
|
||||||
text = text.replace("[image]", virtual_path, 1)
|
replace_next_placeholder("[image]", virtual_path)
|
||||||
elif file.get("file_key"):
|
elif file.get("file_key"):
|
||||||
virtual_path = await self._receive_single_file(msg.thread_ts, file["file_key"], "file", thread_id, user_id=user_id)
|
virtual_path = await self._receive_single_file(msg.thread_ts, file["file_key"], "file", thread_id, user_id=user_id)
|
||||||
text = text.replace("[file]", virtual_path, 1)
|
replace_next_placeholder("[file]", virtual_path)
|
||||||
msg.text = text
|
msg.text = text
|
||||||
return msg
|
return msg
|
||||||
|
|
||||||
@ -429,42 +441,54 @@ class FeishuChannel(Channel):
|
|||||||
return f"Failed to obtain the [{type}]"
|
return f"Failed to obtain the [{type}]"
|
||||||
|
|
||||||
try:
|
try:
|
||||||
content: bytes = await asyncio.to_thread(image_stream.read)
|
content = await asyncio.to_thread(image_stream.read, FEISHU_MAX_INBOUND_FILE_BYTES + 1)
|
||||||
except Exception:
|
except Exception:
|
||||||
logger.exception("[Feishu] failed to read resource stream: resource_key=%s, type=%s", file_key, type)
|
logger.exception("[Feishu] failed to read resource stream: resource_key=%s, type=%s", file_key, type)
|
||||||
return f"Failed to obtain the [{type}]"
|
return f"Failed to obtain the [{type}]"
|
||||||
|
|
||||||
|
if isinstance(content, bytearray):
|
||||||
|
content = bytes(content)
|
||||||
|
elif isinstance(content, memoryview):
|
||||||
|
content = content.tobytes()
|
||||||
|
|
||||||
if not content:
|
if not content:
|
||||||
logger.warning("[Feishu] empty resource content: resource_key=%s, type=%s", file_key, type)
|
logger.warning("[Feishu] empty resource content: resource_key=%s, type=%s", file_key, type)
|
||||||
return f"Failed to obtain the [{type}]"
|
return f"Failed to obtain the [{type}]"
|
||||||
|
if not isinstance(content, bytes):
|
||||||
|
logger.warning("[Feishu] resource stream returned non-bytes content: resource_key=%s, type=%s", file_key, type)
|
||||||
|
return f"Failed to obtain the [{type}]"
|
||||||
|
if len(content) > FEISHU_MAX_INBOUND_FILE_BYTES:
|
||||||
|
logger.warning(
|
||||||
|
"[Feishu] inbound resource exceeds 20 MB download limit, skipping: resource_key=%s, type=%s",
|
||||||
|
file_key,
|
||||||
|
type,
|
||||||
|
)
|
||||||
|
return f"Failed to obtain the [{type}]"
|
||||||
|
|
||||||
effective_user_id = user_id or get_effective_user_id()
|
effective_user_id = user_id or get_effective_user_id()
|
||||||
paths = await asyncio.to_thread(get_paths)
|
paths = await asyncio.to_thread(get_paths)
|
||||||
|
|
||||||
ext = "png" if type == "image" else "bin"
|
default_ext = "png" if type == "image" else "bin"
|
||||||
raw_filename = getattr(response, "file_name", "") or f"feishu_{file_key[-12:]}.{ext}"
|
key_token = re.sub(r"[^A-Za-z0-9_-]", "", file_key)[-12:] or "attachment"
|
||||||
|
fallback_name = f"feishu_{key_token}.{default_ext}"
|
||||||
# Sanitize filename: preserve extension, replace path chars in name part
|
raw_filename = getattr(response, "file_name", "") or fallback_name
|
||||||
if "." in raw_filename:
|
try:
|
||||||
name_part, ext = raw_filename.rsplit(".", 1)
|
safe_filename = normalize_filename(raw_filename)
|
||||||
name_part = re.sub(r"[./\\]", "_", name_part)
|
except (TypeError, ValueError):
|
||||||
filename = f"{name_part}.{ext}"
|
safe_filename = fallback_name
|
||||||
else:
|
|
||||||
filename = re.sub(r"[./\\]", "_", raw_filename)
|
|
||||||
|
|
||||||
def _persist():
|
def _persist():
|
||||||
paths.ensure_thread_dirs(thread_id, user_id=effective_user_id)
|
paths.ensure_thread_dirs(thread_id, user_id=effective_user_id)
|
||||||
uploads_dir = paths.sandbox_uploads_dir(thread_id, user_id=effective_user_id).resolve()
|
uploads_dir = paths.sandbox_uploads_dir(thread_id, user_id=effective_user_id).resolve()
|
||||||
resolved_target = uploads_dir / filename
|
|
||||||
# Use thread_lock to avoid filename conflicts when writing.
|
|
||||||
with self._thread_lock:
|
with self._thread_lock:
|
||||||
resolved_target.write_bytes(content)
|
seen = {entry.name for entry in uploads_dir.iterdir()}
|
||||||
return resolved_target
|
unique_name = claim_unique_filename(safe_filename, seen)
|
||||||
|
return write_upload_file_no_symlink(uploads_dir, unique_name, content)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
resolved_target = await asyncio.to_thread(_persist)
|
resolved_target = await asyncio.to_thread(_persist)
|
||||||
except Exception:
|
except (OSError, ValueError, RuntimeError):
|
||||||
logger.exception("[Feishu] failed to persist downloaded resource: %s, type=%s", filename, type)
|
logger.exception("[Feishu] failed to persist downloaded resource: %s, type=%s", safe_filename, type)
|
||||||
return f"Failed to obtain the [{type}]"
|
return f"Failed to obtain the [{type}]"
|
||||||
|
|
||||||
virtual_path = f"{VIRTUAL_PATH_PREFIX}/uploads/{resolved_target.name}"
|
virtual_path = f"{VIRTUAL_PATH_PREFIX}/uploads/{resolved_target.name}"
|
||||||
|
|||||||
@ -230,6 +230,11 @@ the resulting virtual path (or a failure notice). Historical uploads are not
|
|||||||
automatically injected on later turns; the agent discovers them with
|
automatically injected on later turns; the agent discovers them with
|
||||||
`list_uploaded_files`.
|
`list_uploaded_files`.
|
||||||
|
|
||||||
|
Feishu/Lark inbound resource streams are read with a 20,000,000-byte cap before
|
||||||
|
they are persisted or synced into a non-local sandbox. Oversized resources and
|
||||||
|
per-file path failures are surfaced as a failure placeholder in the message text
|
||||||
|
without aborting later attachments in the same inbound message.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
sequenceDiagram
|
sequenceDiagram
|
||||||
autonumber
|
autonumber
|
||||||
|
|||||||
@ -2296,12 +2296,13 @@ class TestReceiveFile:
|
|||||||
|
|
||||||
_run(go())
|
_run(go())
|
||||||
|
|
||||||
def test_write_does_not_follow_planted_symlink(self, tmp_path, monkeypatch):
|
def test_write_reserves_planted_symlink_name(self, tmp_path, monkeypatch):
|
||||||
"""A symlink planted at the destination must not be written through.
|
"""A symlink planted at the requested name must force a unique suffix.
|
||||||
|
|
||||||
Upload dirs can be mounted into local sandboxes, so a sandbox process can
|
Upload dirs can be mounted into local sandboxes, so a sandbox process can
|
||||||
leave a symlink at a future upload name; following it would let a
|
leave a symlink at a future upload name; following it would let a
|
||||||
gateway-privileged write land outside the bucket.
|
gateway-privileged write land outside the bucket. The symlink name is
|
||||||
|
treated as occupied so the attachment still loads at the next suffix.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
async def go():
|
async def go():
|
||||||
@ -2323,7 +2324,9 @@ class TestReceiveFile:
|
|||||||
out = await channel.receive_file(msg, "t1", user_id="default")
|
out = await channel.receive_file(msg, "t1", user_id="default")
|
||||||
|
|
||||||
assert not outside.exists()
|
assert not outside.exists()
|
||||||
assert "[failed to load image: image.png]" in out.text
|
assert (uploads / "image.png").is_symlink()
|
||||||
|
assert (uploads / "image_1.png").read_bytes() == b"PWNED"
|
||||||
|
assert out.text == f"{VIRTUAL_PATH_PREFIX}/uploads/image_1.png"
|
||||||
|
|
||||||
_run(go())
|
_run(go())
|
||||||
|
|
||||||
|
|||||||
@ -7,6 +7,7 @@ from unittest.mock import AsyncMock, MagicMock
|
|||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
from app.channels import feishu as feishu_module
|
||||||
from app.channels.commands import KNOWN_CHANNEL_COMMANDS
|
from app.channels.commands import KNOWN_CHANNEL_COMMANDS
|
||||||
from app.channels.feishu import FeishuChannel
|
from app.channels.feishu import FeishuChannel
|
||||||
from app.channels.message_bus import (
|
from app.channels.message_bus import (
|
||||||
@ -17,6 +18,7 @@ from app.channels.message_bus import (
|
|||||||
OutboundMessage,
|
OutboundMessage,
|
||||||
)
|
)
|
||||||
from app.channels.store import ChannelStore
|
from app.channels.store import ChannelStore
|
||||||
|
from deerflow.uploads.manager import PathTraversalError
|
||||||
|
|
||||||
|
|
||||||
def _pending(
|
def _pending(
|
||||||
@ -44,6 +46,36 @@ def _run(coro):
|
|||||||
loop.close()
|
loop.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _feishu_file_response(filename: str, content: bytes):
|
||||||
|
response = MagicMock()
|
||||||
|
response.success.return_value = True
|
||||||
|
response.file = BytesIO(content)
|
||||||
|
response.file_name = filename
|
||||||
|
return response
|
||||||
|
|
||||||
|
|
||||||
|
def _feishu_stream_response(filename: str, stream):
|
||||||
|
response = MagicMock()
|
||||||
|
response.success.return_value = True
|
||||||
|
response.file = stream
|
||||||
|
response.file_name = filename
|
||||||
|
return response
|
||||||
|
|
||||||
|
|
||||||
|
def _feishu_file_channel(*responses):
|
||||||
|
channel = FeishuChannel(MessageBus(), {"app_id": "test", "app_secret": "test"})
|
||||||
|
channel._GetMessageResourceRequest = MagicMock()
|
||||||
|
builder = MagicMock()
|
||||||
|
builder.message_id.return_value = builder
|
||||||
|
builder.file_key.return_value = builder
|
||||||
|
builder.type.return_value = builder
|
||||||
|
builder.build.return_value = object()
|
||||||
|
channel._GetMessageResourceRequest.builder.return_value = builder
|
||||||
|
channel._api_client = MagicMock()
|
||||||
|
channel._api_client.im.v1.message_resource.get.side_effect = responses
|
||||||
|
return channel
|
||||||
|
|
||||||
|
|
||||||
def test_feishu_on_message_plain_text():
|
def test_feishu_on_message_plain_text():
|
||||||
bus = MessageBus()
|
bus = MessageBus()
|
||||||
config = {"app_id": "test", "app_secret": "test"}
|
config = {"app_id": "test", "app_secret": "test"}
|
||||||
@ -159,22 +191,7 @@ def test_feishu_receive_file_syncs_sandbox_with_explicit_user_id(tmp_path, monke
|
|||||||
async def go():
|
async def go():
|
||||||
from deerflow.config.paths import Paths
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
bus = MessageBus()
|
channel = _feishu_file_channel(_feishu_file_response("report.md", b"file-bytes"))
|
||||||
channel = FeishuChannel(bus, {"app_id": "test", "app_secret": "test"})
|
|
||||||
channel._GetMessageResourceRequest = MagicMock()
|
|
||||||
builder = MagicMock()
|
|
||||||
builder.message_id.return_value = builder
|
|
||||||
builder.file_key.return_value = builder
|
|
||||||
builder.type.return_value = builder
|
|
||||||
builder.build.return_value = object()
|
|
||||||
channel._GetMessageResourceRequest.builder.return_value = builder
|
|
||||||
|
|
||||||
response = MagicMock()
|
|
||||||
response.success.return_value = True
|
|
||||||
response.file = BytesIO(b"file-bytes")
|
|
||||||
response.file_name = "report.md"
|
|
||||||
channel._api_client = MagicMock()
|
|
||||||
channel._api_client.im.v1.message_resource.get.return_value = response
|
|
||||||
|
|
||||||
provider = MagicMock()
|
provider = MagicMock()
|
||||||
provider.uses_thread_data_mounts = False
|
provider.uses_thread_data_mounts = False
|
||||||
@ -197,6 +214,239 @@ def test_feishu_receive_file_syncs_sandbox_with_explicit_user_id(tmp_path, monke
|
|||||||
_run(go())
|
_run(go())
|
||||||
|
|
||||||
|
|
||||||
|
def test_feishu_receive_file_preserves_duplicate_filenames(tmp_path, monkeypatch):
|
||||||
|
async def go():
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
|
channel = _feishu_file_channel(
|
||||||
|
_feishu_file_response("report.txt", b"FIRST"),
|
||||||
|
_feishu_file_response("report.txt", b"SECOND"),
|
||||||
|
)
|
||||||
|
provider = MagicMock()
|
||||||
|
provider.uses_thread_data_mounts = True
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_paths", lambda: Paths(base_dir=tmp_path))
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_sandbox_provider", lambda: provider)
|
||||||
|
|
||||||
|
first = await channel._receive_single_file("message-1", "file-1", "file", "thread-1", user_id="ou-user")
|
||||||
|
second = await channel._receive_single_file("message-2", "file-2", "file", "thread-1", user_id="ou-user")
|
||||||
|
|
||||||
|
uploads = tmp_path / "users" / "ou-user" / "threads" / "thread-1" / "user-data" / "uploads"
|
||||||
|
assert first == "/mnt/user-data/uploads/report.txt"
|
||||||
|
assert second == "/mnt/user-data/uploads/report_1.txt"
|
||||||
|
assert (uploads / "report.txt").read_bytes() == b"FIRST"
|
||||||
|
assert (uploads / "report_1.txt").read_bytes() == b"SECOND"
|
||||||
|
|
||||||
|
_run(go())
|
||||||
|
|
||||||
|
|
||||||
|
def test_feishu_receive_file_does_not_follow_planted_symlink(tmp_path, monkeypatch):
|
||||||
|
async def go():
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
|
paths = Paths(base_dir=tmp_path)
|
||||||
|
paths.ensure_thread_dirs("thread-1", user_id="ou-user")
|
||||||
|
uploads = paths.sandbox_uploads_dir("thread-1", user_id="ou-user")
|
||||||
|
victim = tmp_path / "victim.txt"
|
||||||
|
victim.write_bytes(b"SAFE")
|
||||||
|
(uploads / "report.txt").symlink_to(victim)
|
||||||
|
|
||||||
|
channel = _feishu_file_channel(_feishu_file_response("report.txt", b"PAYLOAD"))
|
||||||
|
provider = MagicMock()
|
||||||
|
provider.uses_thread_data_mounts = True
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_paths", lambda: paths)
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_sandbox_provider", lambda: provider)
|
||||||
|
|
||||||
|
virtual_path = await channel._receive_single_file("message-1", "file-1", "file", "thread-1", user_id="ou-user")
|
||||||
|
|
||||||
|
assert virtual_path == "/mnt/user-data/uploads/report_1.txt"
|
||||||
|
assert victim.read_bytes() == b"SAFE"
|
||||||
|
assert (uploads / "report_1.txt").read_bytes() == b"PAYLOAD"
|
||||||
|
|
||||||
|
_run(go())
|
||||||
|
|
||||||
|
|
||||||
|
def test_feishu_receive_file_reserves_dangling_symlink_name(tmp_path, monkeypatch):
|
||||||
|
async def go():
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
|
paths = Paths(base_dir=tmp_path)
|
||||||
|
paths.ensure_thread_dirs("thread-1", user_id="ou-user")
|
||||||
|
uploads = paths.sandbox_uploads_dir("thread-1", user_id="ou-user")
|
||||||
|
missing_target = tmp_path / "missing.txt"
|
||||||
|
(uploads / "report.txt").symlink_to(missing_target)
|
||||||
|
|
||||||
|
channel = _feishu_file_channel(_feishu_file_response("report.txt", b"PAYLOAD"))
|
||||||
|
provider = MagicMock()
|
||||||
|
provider.uses_thread_data_mounts = True
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_paths", lambda: paths)
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_sandbox_provider", lambda: provider)
|
||||||
|
|
||||||
|
virtual_path = await channel._receive_single_file("message-1", "file-1", "file", "thread-1", user_id="ou-user")
|
||||||
|
|
||||||
|
assert virtual_path == "/mnt/user-data/uploads/report_1.txt"
|
||||||
|
assert not missing_target.exists()
|
||||||
|
assert (uploads / "report_1.txt").read_bytes() == b"PAYLOAD"
|
||||||
|
|
||||||
|
_run(go())
|
||||||
|
|
||||||
|
|
||||||
|
def test_feishu_receive_file_syncs_unique_path_to_remote_sandbox(tmp_path, monkeypatch):
|
||||||
|
async def go():
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
|
paths = Paths(base_dir=tmp_path)
|
||||||
|
paths.ensure_thread_dirs("thread-1", user_id="ou-user")
|
||||||
|
uploads = paths.sandbox_uploads_dir("thread-1", user_id="ou-user")
|
||||||
|
(uploads / "report.md").write_bytes(b"OLDER")
|
||||||
|
|
||||||
|
channel = _feishu_file_channel(_feishu_file_response("report.md", b"NEWER"))
|
||||||
|
provider = MagicMock()
|
||||||
|
provider.uses_thread_data_mounts = False
|
||||||
|
provider.acquire_async = AsyncMock(return_value="aio-1")
|
||||||
|
sandbox = MagicMock()
|
||||||
|
provider.get.return_value = sandbox
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_paths", lambda: paths)
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_sandbox_provider", lambda: provider)
|
||||||
|
|
||||||
|
virtual_path = await channel._receive_single_file("message-1", "file-1", "file", "thread-1", user_id="ou-user")
|
||||||
|
|
||||||
|
assert virtual_path == "/mnt/user-data/uploads/report_1.md"
|
||||||
|
assert (uploads / "report.md").read_bytes() == b"OLDER"
|
||||||
|
assert (uploads / "report_1.md").read_bytes() == b"NEWER"
|
||||||
|
provider.acquire_async.assert_awaited_once_with("thread-1", user_id="ou-user")
|
||||||
|
sandbox.update_file.assert_called_once_with("/mnt/user-data/uploads/report_1.md", b"NEWER")
|
||||||
|
|
||||||
|
_run(go())
|
||||||
|
|
||||||
|
|
||||||
|
def test_feishu_receive_file_path_traversal_failure_is_per_attachment(tmp_path, monkeypatch):
|
||||||
|
async def go():
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
|
paths = Paths(base_dir=tmp_path)
|
||||||
|
channel = _feishu_file_channel(
|
||||||
|
_feishu_file_response("bad.txt", b"BAD"),
|
||||||
|
_feishu_file_response("ok.txt", b"OK"),
|
||||||
|
)
|
||||||
|
provider = MagicMock()
|
||||||
|
provider.uses_thread_data_mounts = True
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_paths", lambda: paths)
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_sandbox_provider", lambda: provider)
|
||||||
|
|
||||||
|
real_write = feishu_module.write_upload_file_no_symlink
|
||||||
|
call_count = 0
|
||||||
|
|
||||||
|
def flaky_write(base_dir, filename, data):
|
||||||
|
nonlocal call_count
|
||||||
|
call_count += 1
|
||||||
|
if call_count == 1:
|
||||||
|
raise PathTraversalError("Path traversal detected")
|
||||||
|
return real_write(base_dir, filename, data)
|
||||||
|
|
||||||
|
monkeypatch.setattr(feishu_module, "write_upload_file_no_symlink", flaky_write)
|
||||||
|
msg = InboundMessage(
|
||||||
|
channel_name="feishu",
|
||||||
|
chat_id="chat_1",
|
||||||
|
user_id="user_1",
|
||||||
|
text="[file] then [file]",
|
||||||
|
thread_ts="message-1",
|
||||||
|
files=[{"file_key": "file-1"}, {"file_key": "file-2"}],
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await channel.receive_file(msg, "thread-1", user_id="ou-user")
|
||||||
|
|
||||||
|
uploads = tmp_path / "users" / "ou-user" / "threads" / "thread-1" / "user-data" / "uploads"
|
||||||
|
assert result.text == "Failed to obtain the [file] then /mnt/user-data/uploads/ok.txt"
|
||||||
|
assert (uploads / "ok.txt").read_bytes() == b"OK"
|
||||||
|
|
||||||
|
_run(go())
|
||||||
|
|
||||||
|
|
||||||
|
def test_feishu_receive_file_runtime_resolve_failure_is_per_attachment(tmp_path, monkeypatch):
|
||||||
|
async def go():
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
|
real_paths = Paths(base_dir=tmp_path)
|
||||||
|
|
||||||
|
class FlakyPaths:
|
||||||
|
def __init__(self):
|
||||||
|
self.resolve_calls = 0
|
||||||
|
|
||||||
|
def ensure_thread_dirs(self, thread_id, *, user_id=None):
|
||||||
|
return real_paths.ensure_thread_dirs(thread_id, user_id=user_id)
|
||||||
|
|
||||||
|
def sandbox_uploads_dir(self, thread_id, *, user_id=None):
|
||||||
|
uploads = real_paths.sandbox_uploads_dir(thread_id, user_id=user_id)
|
||||||
|
|
||||||
|
class UploadsDirProxy:
|
||||||
|
def resolve(inner_self):
|
||||||
|
self.resolve_calls += 1
|
||||||
|
if self.resolve_calls == 1:
|
||||||
|
raise RuntimeError("symlink loop")
|
||||||
|
return uploads.resolve()
|
||||||
|
|
||||||
|
return UploadsDirProxy()
|
||||||
|
|
||||||
|
paths = FlakyPaths()
|
||||||
|
channel = _feishu_file_channel(
|
||||||
|
_feishu_file_response("bad.txt", b"BAD"),
|
||||||
|
_feishu_file_response("ok.txt", b"OK"),
|
||||||
|
)
|
||||||
|
provider = MagicMock()
|
||||||
|
provider.uses_thread_data_mounts = True
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_paths", lambda: paths)
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_sandbox_provider", lambda: provider)
|
||||||
|
|
||||||
|
msg = InboundMessage(
|
||||||
|
channel_name="feishu",
|
||||||
|
chat_id="chat_1",
|
||||||
|
user_id="user_1",
|
||||||
|
text="[file] then [file]",
|
||||||
|
thread_ts="message-1",
|
||||||
|
files=[{"file_key": "file-1"}, {"file_key": "file-2"}],
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await channel.receive_file(msg, "thread-1", user_id="ou-user")
|
||||||
|
|
||||||
|
uploads = tmp_path / "users" / "ou-user" / "threads" / "thread-1" / "user-data" / "uploads"
|
||||||
|
assert result.text == "Failed to obtain the [file] then /mnt/user-data/uploads/ok.txt"
|
||||||
|
assert (uploads / "ok.txt").read_bytes() == b"OK"
|
||||||
|
|
||||||
|
_run(go())
|
||||||
|
|
||||||
|
|
||||||
|
def test_feishu_receive_file_rejects_oversized_resource(tmp_path, monkeypatch):
|
||||||
|
async def go():
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
|
class TrackingStream:
|
||||||
|
def __init__(self):
|
||||||
|
self.requested_size = None
|
||||||
|
|
||||||
|
def read(self, size=-1):
|
||||||
|
self.requested_size = size
|
||||||
|
return b"12345"
|
||||||
|
|
||||||
|
stream = TrackingStream()
|
||||||
|
paths = Paths(base_dir=tmp_path)
|
||||||
|
paths.ensure_thread_dirs("thread-1", user_id="ou-user")
|
||||||
|
uploads = paths.sandbox_uploads_dir("thread-1", user_id="ou-user")
|
||||||
|
channel = _feishu_file_channel(_feishu_stream_response("large.txt", stream))
|
||||||
|
provider = MagicMock()
|
||||||
|
provider.uses_thread_data_mounts = True
|
||||||
|
monkeypatch.setattr("app.channels.feishu.FEISHU_MAX_INBOUND_FILE_BYTES", 4)
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_paths", lambda: paths)
|
||||||
|
monkeypatch.setattr("app.channels.feishu.get_sandbox_provider", lambda: provider)
|
||||||
|
|
||||||
|
virtual_path = await channel._receive_single_file("message-1", "file-1", "file", "thread-1", user_id="ou-user")
|
||||||
|
|
||||||
|
assert virtual_path == "Failed to obtain the [file]"
|
||||||
|
assert stream.requested_size == 5
|
||||||
|
assert list(uploads.iterdir()) == []
|
||||||
|
|
||||||
|
_run(go())
|
||||||
|
|
||||||
|
|
||||||
def test_feishu_on_message_extracts_image_and_file_keys():
|
def test_feishu_on_message_extracts_image_and_file_keys():
|
||||||
bus = MessageBus()
|
bus = MessageBus()
|
||||||
channel = FeishuChannel(bus, {"app_id": "test", "app_secret": "test"})
|
channel = FeishuChannel(bus, {"app_id": "test", "app_secret": "test"})
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user