mirror of
https://github.com/OpenBMB/ChatDev.git
synced 2026-11-18 07:50:38 +00:00
Merge pull request #687 from NA-Wen/new_pr
fix: confine workflow writes and gate unsafe code execution
This commit is contained in:
commit
a2ea4a263c
@ -7,6 +7,11 @@
|
||||
BASE_URL=https://api.openai.com/v1
|
||||
API_KEY=sk-your-openai-api-key-here
|
||||
|
||||
# Disabled by default. Enable only for trusted workflows; this is not a sandbox.
|
||||
# Covers Python nodes, code_save_and_run, execute_code, uv tools, Manim, and local MCP.
|
||||
# Set in the server environment; workflow vars and tool env overrides cannot enable it.
|
||||
MAC_ALLOW_UNSAFE_CODE_EXECUTION=0
|
||||
|
||||
# Example BASE_URL values:
|
||||
# - OpenAI: https://api.openai.com/v1
|
||||
# - Gemini: https://generativelanguage.googleapis.com
|
||||
@ -21,4 +26,4 @@ API_KEY=sk-your-openai-api-key-here
|
||||
# Get from: https://serper.dev
|
||||
|
||||
# JINA_API_KEY=your-jina-api-key-here
|
||||
# Get from: https://jina.ai
|
||||
# Get from: https://jina.ai
|
||||
|
||||
@ -2,6 +2,16 @@
|
||||
|
||||
The Python node is used to execute Python scripts or inline code within workflows, implementing custom data processing, API calls, file operations, and other logic. Scripts are executed in the shared `code_workspace/` directory and can access workflow context data.
|
||||
|
||||
Code execution is disabled by default. For trusted workflows, set
|
||||
`MAC_ALLOW_UNSAFE_CODE_EXECUTION=1` in the server process environment (or the
|
||||
deployment `.env` file) and restart the service. Workflow `vars`, node `env`, and
|
||||
tool arguments cannot enable this permission. The same setting controls
|
||||
`code_save_and_run`, `execute_code`, uv tools (including package installation),
|
||||
Manim scripts, and local MCP server processes; remote MCP is unaffected.
|
||||
Enabling execution grants the backend user's permissions and is not a sandbox.
|
||||
The path restrictions on `code_save_and_run` remain active when execution is enabled:
|
||||
filenames must stay within the workspace and must not modify `attachments/`.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|
||||
@ -218,6 +218,24 @@ python -m tools.export_design_template \
|
||||
- **HTTP**: `POST /api/workflow/execute` with `session_name`, `graph_path` or `graph_content`, `task_prompt`, optional `attachments`, and `log_level` (defaults to `INFO`, supports `INFO` or `DEBUG`).
|
||||
- **CLI**: `python run.py --path yaml_instance/demo.yaml --name test_run`. Provide `TASK_PROMPT` via env var or respond to the CLI prompt.
|
||||
|
||||
For `POST /api/workflow/run` (JSON or SSE), upload attachments through
|
||||
`/api/uploads/{session_id}` first, then pass that exact ID as `attachment_session_id`
|
||||
and the returned attachment IDs in `attachments`. Existing paths are accepted only
|
||||
when they match registered uploads in that session. Arbitrary server file paths are
|
||||
rejected. `session_name` still names the output run; it is independent of the upload
|
||||
session. Local SDK and CLI file inputs are unchanged.
|
||||
|
||||
Upload, WebSocket execution, and download routes map each raw session ID to
|
||||
`WareHouse/session_<session_id>`. For example, `foo` and `session_foo` use
|
||||
`session_foo` and `session_session_foo`, respectively. Previously shared uploads
|
||||
for prefixed IDs must be uploaded again under the correct session.
|
||||
|
||||
Agent-generated attachment names and node IDs used for storage must be single
|
||||
path components. Local attachment sources must stay within the code workspace;
|
||||
generated destination paths cannot follow symbolic links. Session downloads skip
|
||||
symbolic links inside the session (including virtual environment interpreter links)
|
||||
and still reject a symbolic link used as the session root.
|
||||
|
||||
## 10. Debugging Tips
|
||||
- Use the Web UI context snapshots or `WareHouse/<session>/context.json` to inspect node I/O. Note that all node outputs are now standardized as `List[Message]`.
|
||||
- Leverage the Schema API breadcrumbs ([config_schema_contract.md](config_schema_contract.md)) or run `python run.py --inspect-schema` to view field specs quickly.
|
||||
|
||||
@ -2,6 +2,12 @@
|
||||
|
||||
Python 节点用于在工作流中执行 Python 脚本或内联代码,实现自定义数据处理、API 调用、文件操作等逻辑。脚本在共享的 `code_workspace/` 目录中执行,可访问工作流上下文数据。
|
||||
|
||||
代码执行默认关闭。仅在运行可信工作流时,在服务端进程环境(或部署的 `.env` 文件)中设置
|
||||
`MAC_ALLOW_UNSAFE_CODE_EXECUTION=1` 并重启服务。工作流 `vars`、节点 `env` 和工具参数无法开启此权限。
|
||||
该开关同时控制 `code_save_and_run`、`execute_code`、uv 工具(包括安装依赖)、Manim 脚本和本地 MCP
|
||||
服务进程;远程 MCP 不受影响。开启后,代码以后端用户的权限运行,不具备沙箱隔离。
|
||||
即使开启执行权限,`code_save_and_run` 的路径限制仍然生效:文件必须位于工作区内,且不得修改 `attachments/`。
|
||||
|
||||
## 配置项
|
||||
|
||||
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|
||||
|
||||
@ -241,6 +241,19 @@ python -m tools.export_design_template \
|
||||
- **HTTP**:`POST /api/workflow/execute`,payload 包含 `session_name`, `graph_path` 或 `graph_content`, `task_prompt`、可选的 `attachments`,以及 `log_level`(默认 `INFO`,支持 `INFO` 或 `DEBUG`)。
|
||||
- **CLI**:`python run.py --path yaml_instance/demo.yaml --name test_run`(执行前可设置 `TASK_PROMPT` 环境变量或在 CLI 提示中输入)。
|
||||
|
||||
调用 `POST /api/workflow/run`(JSON 或 SSE)时,先通过 `/api/uploads/{session_id}`
|
||||
上传附件,再把同一个原始会话 ID 填入 `attachment_session_id`,将上传返回的附件 ID
|
||||
放入 `attachments`。兼容该会话中已登记的上传文件路径,但拒绝任意服务器文件路径。
|
||||
`session_name` 仍用于指定运行输出名称,与上传会话独立。本地 SDK 和 CLI 的文件输入不变。
|
||||
|
||||
上传、WebSocket 运行和下载统一映射到 `WareHouse/session_<session_id>`:
|
||||
`foo` 对应 `session_foo`,`session_foo` 对应 `session_session_foo`。
|
||||
旧版本中因前缀合并而共用的附件,需要在正确的会话下重新上传。
|
||||
|
||||
agent 生成附件时,文件名和用于落盘的节点 ID 必须是单个路径分量;本地附件来源必须
|
||||
位于代码工作区内。生成附件的目标路径不跟随符号链接。会话下载跳过会话内的符号链接
|
||||
(包括虚拟环境的解释器链接),但仍拒绝以符号链接作为会话根目录。
|
||||
|
||||
## 10. 调试建议
|
||||
- 使用 Web UI 的上下文快照或 WareHouse 中的 `context.json` 检查节点输入输出。注意所有节点输出现已统一为 `List[Message]` 结构。
|
||||
- 结合 [config_schema_contract.md](config_schema_contract.md) 的 breadcrumbs 功能,用 CLI `python run.py --inspect-schema` 快速查看字段定义。
|
||||
|
||||
@ -5,8 +5,11 @@ import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
from pathlib import Path, PureWindowsPath
|
||||
from functions.function_calling.file import FileToolContext
|
||||
from utils.code_execution import require_unsafe_code_execution
|
||||
from utils.exceptions import SecurityError, ValidationError
|
||||
from utils.workspace_paths import validate_path_component
|
||||
|
||||
def uppercase_payload(data: str, _context: Dict[str, Any]) -> str:
|
||||
"""Return an uppercase copy of the payload text."""
|
||||
@ -23,6 +26,7 @@ def code_save_and_run(data: str, _context: Dict[str, Any]) -> str:
|
||||
```
|
||||
Repeated for multiple files.
|
||||
"""
|
||||
require_unsafe_code_execution("code_save_and_run")
|
||||
# Parse matches first
|
||||
pattern = re.compile(r"(?P<filename>[^\n]+)\n```(?:python)?\n(?P<code>.*?)\n```", re.DOTALL)
|
||||
matches = list(pattern.finditer(data))
|
||||
@ -32,6 +36,37 @@ def code_save_and_run(data: str, _context: Dict[str, Any]) -> str:
|
||||
|
||||
ctx = FileToolContext(_context)
|
||||
workspace_root = ctx.workspace_root
|
||||
|
||||
# Validate every target before deleting any existing workspace files.
|
||||
files_to_save = []
|
||||
attachments_root = workspace_root / "attachments"
|
||||
if attachments_root.is_symlink():
|
||||
raise SecurityError("The attachments directory must not be a symbolic link")
|
||||
attachments_root = attachments_root.resolve()
|
||||
for match in matches:
|
||||
filename = match.group("filename").strip()
|
||||
relative_path = Path(filename.replace("\\", "/"))
|
||||
if not filename or relative_path.is_absolute() or PureWindowsPath(filename).drive:
|
||||
raise SecurityError("Code filenames must be relative paths inside the workspace")
|
||||
try:
|
||||
for part in relative_path.parts:
|
||||
validate_path_component(part, field="filename")
|
||||
file_path = ctx.resolve_under_workspace(relative_path)
|
||||
# resolve() preserves case aliases on some filesystems; compare directory identities too.
|
||||
is_attachment_alias = attachments_root.exists() and any(
|
||||
ancestor.exists() and ancestor.samefile(attachments_root)
|
||||
for ancestor in (file_path, *file_path.parents)
|
||||
)
|
||||
except (ValueError, OSError, RuntimeError, ValidationError) as exc:
|
||||
raise SecurityError(f"Invalid code filename: {filename}") from exc
|
||||
if (
|
||||
file_path == workspace_root
|
||||
or relative_path.parts[0] == "attachments"
|
||||
or file_path.is_relative_to(attachments_root)
|
||||
or is_attachment_alias
|
||||
):
|
||||
raise SecurityError("Code files must not replace the workspace or modify attachments")
|
||||
files_to_save.append((filename, file_path, match.group("code")))
|
||||
|
||||
# 1. Clear workspace
|
||||
if workspace_root.exists():
|
||||
@ -39,18 +74,18 @@ def code_save_and_run(data: str, _context: Dict[str, Any]) -> str:
|
||||
if item.name == "attachments":
|
||||
continue
|
||||
|
||||
if item.is_dir():
|
||||
if item.is_symlink():
|
||||
item.unlink()
|
||||
elif attachments_root.exists() and item.samefile(attachments_root):
|
||||
continue
|
||||
elif item.is_dir():
|
||||
shutil.rmtree(item)
|
||||
else:
|
||||
item.unlink()
|
||||
|
||||
saved_code_blocks = []
|
||||
for match in matches:
|
||||
filename = match.group("filename").strip()
|
||||
code = match.group("code")
|
||||
|
||||
for filename, file_path, code in files_to_save:
|
||||
# Save to file
|
||||
file_path = workspace_root / filename
|
||||
# Ensure parent dirs exist if filename contains path separators
|
||||
file_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
file_path.write_text(code, encoding="utf-8")
|
||||
@ -67,28 +102,19 @@ def code_save_and_run(data: str, _context: Dict[str, Any]) -> str:
|
||||
env = os.environ.copy()
|
||||
env["PYTHONUNBUFFERED"] = "1"
|
||||
|
||||
if os.name == 'nt':
|
||||
command = f"cd {workspace_root} && dir && uv run main.py"
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
shell=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
|
||||
env=env
|
||||
)
|
||||
else:
|
||||
|
||||
command = "uv run main.py"
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
shell=True,
|
||||
cwd=str(workspace_root),
|
||||
preexec_fn=os.setsid,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
env=env
|
||||
)
|
||||
process_options = (
|
||||
{"creationflags": subprocess.CREATE_NEW_PROCESS_GROUP}
|
||||
if os.name == "nt"
|
||||
else {"start_new_session": True}
|
||||
)
|
||||
process = subprocess.Popen(
|
||||
["uv", "run", "main.py"],
|
||||
cwd=str(workspace_root),
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
env=env,
|
||||
**process_options,
|
||||
)
|
||||
|
||||
try:
|
||||
stdout, stderr = process.communicate(timeout=3)
|
||||
|
||||
@ -1,3 +1,6 @@
|
||||
from utils.code_execution import require_unsafe_code_execution
|
||||
|
||||
|
||||
def execute_code(code: str, time_out: int = 60) -> str:
|
||||
"""
|
||||
Execute code and return std outputs and std error.
|
||||
@ -9,6 +12,7 @@ def execute_code(code: str, time_out: int = 60) -> str:
|
||||
Returns:
|
||||
str: std output and std error
|
||||
"""
|
||||
require_unsafe_code_execution("execute_code")
|
||||
import os
|
||||
import sys
|
||||
import subprocess
|
||||
|
||||
@ -6,6 +6,8 @@ import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Mapping, Sequence
|
||||
|
||||
from utils.code_execution import require_unsafe_code_execution
|
||||
|
||||
_SAFE_PACKAGE_RE = re.compile(r"^[A-Za-z0-9_.\-+=<>!\[\],@:/]+$")
|
||||
_DEFAULT_TIMEOUT = float(os.getenv("LIB_INSTALL_TIMEOUT", "120"))
|
||||
_OUTPUT_SNIPPET_LIMIT = 240
|
||||
@ -156,6 +158,7 @@ def _run_uv_command(
|
||||
env: Dict[str, str] | None = None,
|
||||
timeout: float | None = None,
|
||||
) -> Dict[str, Any]:
|
||||
require_unsafe_code_execution("uv command")
|
||||
timeout_value = _DEFAULT_TIMEOUT if timeout is None else timeout
|
||||
env_vars = None if env is None else {**os.environ, **env}
|
||||
try:
|
||||
|
||||
@ -4,6 +4,7 @@ from pathlib import Path
|
||||
import ast
|
||||
import subprocess
|
||||
import tempfile
|
||||
from utils.code_execution import require_unsafe_code_execution
|
||||
|
||||
def _get_class_names(py_file: str) -> list[str]:
|
||||
file_path = Path(py_file)
|
||||
@ -16,6 +17,7 @@ def render_manim(
|
||||
quality: str = "h",
|
||||
preview: bool = True,
|
||||
) -> Path:
|
||||
require_unsafe_code_execution("render_manim")
|
||||
output_dir = Path.cwd() / "media"
|
||||
print("Clearing media folder:", output_dir)
|
||||
shutil.rmtree(output_dir, ignore_errors=True)
|
||||
|
||||
4
run.py
4
run.py
@ -11,6 +11,7 @@ from entity.messages import Message
|
||||
from utils.attachments import AttachmentStore
|
||||
from utils.schema_exporter import build_schema_response, SchemaResolutionError
|
||||
from utils.task_input import TaskInputBuilder
|
||||
from utils.workspace_paths import resolve_workspace_path
|
||||
from workflow.graph_context import GraphContext
|
||||
from workflow.graph import GraphExecutor
|
||||
|
||||
@ -28,8 +29,7 @@ def build_task_input_payload(
|
||||
if not attachment_paths:
|
||||
return prompt
|
||||
|
||||
code_workspace = graph_context.directory / "code_workspace"
|
||||
attachments_dir = code_workspace / "attachments"
|
||||
attachments_dir = resolve_workspace_path(graph_context.directory, "code_workspace", "attachments")
|
||||
attachments_dir.mkdir(parents=True, exist_ok=True)
|
||||
store = AttachmentStore(attachments_dir)
|
||||
builder = TaskInputBuilder(store)
|
||||
|
||||
@ -8,6 +8,7 @@ from entity.messages import Message, MessageRole
|
||||
from runtime.node.executor import ExecutionContext
|
||||
from utils.log_manager import LogManager
|
||||
from utils.structured_logger import get_server_logger
|
||||
from utils.exceptions import SecurityError
|
||||
from entity.configs.node.node import EdgeLink, Node
|
||||
from entity.configs.edge.edge_condition import EdgeConditionTypeConfig
|
||||
from utils.function_manager import FunctionManager
|
||||
@ -64,6 +65,8 @@ class EdgeConditionManager(Generic[TConfig], ABC):
|
||||
log_manager=log_manager,
|
||||
context=self.execution_context,
|
||||
)
|
||||
except SecurityError:
|
||||
raise
|
||||
except Exception as exc: # pragma: no cover
|
||||
error_msg = (
|
||||
f"Edge payload processor failed for {from_node.id}->{edge_link.target.id}: {exc}"
|
||||
|
||||
@ -5,6 +5,7 @@ from typing import Any, Callable, Mapping
|
||||
from entity.messages import Message
|
||||
from utils.log_manager import LogManager
|
||||
from utils.structured_logger import get_server_logger
|
||||
from utils.exceptions import SecurityError
|
||||
from .base import EdgePayloadProcessor, ProcessorFactoryContext
|
||||
from entity.configs.edge.edge_processor import FunctionEdgeProcessorConfig
|
||||
from runtime.node.executor import ExecutionContext
|
||||
@ -35,6 +36,8 @@ class FunctionEdgePayloadProcessor(EdgePayloadProcessor[FunctionEdgeProcessorCon
|
||||
return payload
|
||||
try:
|
||||
result = self._callable(payload.text_content(), context.global_state)
|
||||
except SecurityError:
|
||||
raise
|
||||
except Exception as exc: # pragma: no cover - defensive logging
|
||||
error_msg = f"Processor function '{self._name}' failed: {exc}"
|
||||
log_manager.error(error_msg)
|
||||
|
||||
@ -22,6 +22,7 @@ from entity.configs.node.tooling import FunctionToolConfig, McpLocalConfig, McpR
|
||||
from entity.messages import MessageBlock, MessageBlockType
|
||||
from entity.tool_spec import ToolSpec
|
||||
from utils.attachments import AttachmentStore
|
||||
from utils.code_execution import require_unsafe_code_execution
|
||||
from utils.function_manager import FUNCTION_CALLING_DIR, FunctionManager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@ -497,6 +498,7 @@ class ToolManager:
|
||||
return MessageBlockType.FILE
|
||||
|
||||
def _get_stdio_client(self, config: McpLocalConfig, launch_key: str) -> "_StdioClientWrapper":
|
||||
require_unsafe_code_execution("Local MCP server")
|
||||
client = self._mcp_stdio_clients.get(launch_key)
|
||||
if client is None:
|
||||
client = _StdioClientWrapper(config)
|
||||
|
||||
@ -11,6 +11,7 @@ import asyncio
|
||||
import base64
|
||||
import json
|
||||
import traceback
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Dict, List, Optional, Sequence
|
||||
|
||||
from entity.configs import Node
|
||||
@ -34,6 +35,8 @@ from runtime.node.agent.memory.memory_base import (
|
||||
from runtime.node.agent import ThinkingPayload
|
||||
from runtime.node.agent import ModelProvider, ProviderRegistry, ModelResponse
|
||||
from runtime.node.agent.skills import AgentSkillManager
|
||||
from utils.exceptions import SecurityError, ValidationError
|
||||
from utils.workspace_paths import resolve_workspace_path, validate_path_component
|
||||
from tenacity import Retrying, retry_if_exception, stop_after_attempt, wait_random_exponential
|
||||
|
||||
|
||||
@ -164,6 +167,8 @@ class AgentNodeExecutor(NodeExecutor):
|
||||
source=node.id,
|
||||
)]
|
||||
|
||||
except SecurityError:
|
||||
raise
|
||||
except Exception as e:
|
||||
traceback.print_exc()
|
||||
error_msg = f"[Node: {node.id}] Error calling model: {str(e)}"
|
||||
@ -1079,6 +1084,8 @@ class AgentNodeExecutor(NodeExecutor):
|
||||
continue
|
||||
try:
|
||||
self._persist_single_attachment(store, block, node_id)
|
||||
except SecurityError:
|
||||
raise
|
||||
except Exception as exc:
|
||||
raise RuntimeError(f"Failed to persist attachment '{attachment.name or attachment.attachment_id}': {exc}") from exc
|
||||
|
||||
@ -1086,6 +1093,14 @@ class AgentNodeExecutor(NodeExecutor):
|
||||
attachment = block.attachment
|
||||
if attachment is None:
|
||||
return
|
||||
attachment.mime_type = attachment.mime_type or self._guess_mime_from_data_uri(attachment.data_uri)
|
||||
try:
|
||||
validate_path_component(node_id, field="node_id")
|
||||
if attachment.attachment_id:
|
||||
validate_path_component(attachment.attachment_id, field="attachment_id")
|
||||
filename = validate_path_component(self._make_generated_filename(attachment), field="attachment name")
|
||||
except ValidationError as exc:
|
||||
raise SecurityError(f"Invalid generated attachment path: {exc}") from exc
|
||||
if attachment.remote_file_id and not attachment.data_uri and not attachment.local_path:
|
||||
record = store.register_remote_file(
|
||||
remote_file_id=attachment.remote_file_id,
|
||||
@ -1102,21 +1117,31 @@ class AgentNodeExecutor(NodeExecutor):
|
||||
if workspace_root is None or not node_id:
|
||||
raise RuntimeError("Workspace or node context missing for attachment persistence")
|
||||
|
||||
target_dir = workspace_root / "generated" / node_id
|
||||
target_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
inferred_mime = attachment.mime_type or self._guess_mime_from_data_uri(attachment.data_uri)
|
||||
attachment.mime_type = inferred_mime
|
||||
workspace_root = Path(workspace_root).resolve()
|
||||
try:
|
||||
target_path = resolve_workspace_path(workspace_root, "generated", node_id, filename)
|
||||
current = workspace_root
|
||||
for part in ("generated", node_id, filename):
|
||||
current = current / part
|
||||
if current.is_symlink():
|
||||
raise SecurityError("Generated attachment paths must not contain symbolic links")
|
||||
except (ValidationError, OSError, RuntimeError) as exc:
|
||||
raise SecurityError(f"Invalid generated attachment path: {exc}") from exc
|
||||
|
||||
data_bytes = self._decode_data_uri(attachment.data_uri) if attachment.data_uri else None
|
||||
target_path = None
|
||||
|
||||
if data_bytes is None and attachment.local_path:
|
||||
target_path = target_dir / (attachment.name or self._make_generated_filename(attachment))
|
||||
try:
|
||||
source = Path(attachment.local_path).resolve()
|
||||
if not source.is_relative_to(workspace_root) or not source.is_file():
|
||||
raise SecurityError("Generated attachment source must stay inside the workspace")
|
||||
except (OSError, RuntimeError, ValueError) as exc:
|
||||
raise SecurityError("Invalid generated attachment source") from exc
|
||||
target_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
import shutil
|
||||
shutil.copy2(attachment.local_path, target_path)
|
||||
if source != target_path:
|
||||
shutil.copy2(source, target_path)
|
||||
elif data_bytes is not None:
|
||||
target_path = target_dir / (attachment.name or self._make_generated_filename(attachment))
|
||||
target_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(target_path, "wb") as handle:
|
||||
handle.write(data_bytes)
|
||||
else:
|
||||
|
||||
@ -12,6 +12,7 @@ from entity.configs import Node
|
||||
from entity.configs.node.python_runner import PythonRunnerConfig
|
||||
from entity.messages import Message, MessageRole
|
||||
from runtime.node.executor.base import NodeExecutor
|
||||
from utils.code_execution import require_unsafe_code_execution
|
||||
|
||||
|
||||
_CODE_BLOCK_RE = re.compile(r"```(?P<lang>[a-zA-Z0-9_+-]*)?\s*\n(?P<code>.*?)```", re.DOTALL)
|
||||
@ -36,6 +37,7 @@ class PythonNodeExecutor(NodeExecutor):
|
||||
if node.node_type != "python":
|
||||
raise ValueError(f"Node {node.id} is not a python node")
|
||||
|
||||
require_unsafe_code_execution("Python node")
|
||||
workspace = self._ensure_workspace_root()
|
||||
last_message = inputs[-1] if inputs else None
|
||||
code_payload = self._extract_code(last_message)
|
||||
|
||||
@ -14,6 +14,7 @@ from utils.attachments import AttachmentStore
|
||||
from utils.exceptions import ValidationError
|
||||
from server.settings import YAML_DIR
|
||||
from utils.task_input import TaskInputBuilder
|
||||
from utils.workspace_paths import resolve_workspace_path, validate_path_component
|
||||
from workflow.graph import GraphExecutor
|
||||
from workflow.graph_context import GraphContext
|
||||
|
||||
@ -39,7 +40,7 @@ class WorkflowRunResult:
|
||||
|
||||
def _normalize_session_name(yaml_path: Path, session_name: Optional[str]) -> str:
|
||||
if session_name and session_name.strip():
|
||||
return session_name.strip()
|
||||
return validate_path_component(session_name.strip(), field="session_name")
|
||||
timestamp = datetime.now().strftime("%Y%m%d%H%M%S")
|
||||
return f"sdk_{yaml_path.stem}_{timestamp}"
|
||||
|
||||
@ -52,7 +53,7 @@ def _build_task_input(
|
||||
if not attachments:
|
||||
return prompt
|
||||
|
||||
attachments_dir = graph_context.directory / "code_workspace" / "attachments"
|
||||
attachments_dir = resolve_workspace_path(graph_context.directory, "code_workspace", "attachments")
|
||||
attachments_dir.mkdir(parents=True, exist_ok=True)
|
||||
store = AttachmentStore(attachments_dir)
|
||||
builder = TaskInputBuilder(store)
|
||||
@ -95,8 +96,8 @@ def run_workflow(
|
||||
details={"task_prompt_provided": bool(task_prompt)},
|
||||
)
|
||||
|
||||
design = load_config(yaml_path, fn_module=fn_module, vars_override=variables)
|
||||
normalized_session = _normalize_session_name(yaml_path, session_name)
|
||||
design = load_config(yaml_path, fn_module=fn_module, vars_override=variables)
|
||||
|
||||
graph_config = GraphConfig.from_definition(
|
||||
design.graph,
|
||||
|
||||
@ -17,6 +17,7 @@ class WorkflowRunRequest(BaseModel):
|
||||
yaml_file: str
|
||||
task_prompt: str
|
||||
attachments: Optional[List[str]] = None
|
||||
attachment_session_id: Optional[str] = None
|
||||
session_name: Optional[str] = None
|
||||
variables: Optional[Dict[str, Any]] = None
|
||||
log_level: Optional[Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]] = None
|
||||
|
||||
@ -19,11 +19,13 @@ from runtime.bootstrap.schema import ensure_schema_registry_populated
|
||||
from runtime.sdk import OUTPUT_ROOT, run_workflow
|
||||
from server.models import WorkflowRunRequest
|
||||
from server.settings import YAML_DIR
|
||||
from server.state import get_websocket_manager
|
||||
from utils.attachments import AttachmentStore
|
||||
from utils.exceptions import ValidationError, WorkflowExecutionError
|
||||
from utils.exceptions import ValidationError, WorkflowExecutionError, SecurityError
|
||||
from utils.logger import WorkflowLogger
|
||||
from utils.structured_logger import get_server_logger, LogType
|
||||
from utils.task_input import TaskInputBuilder
|
||||
from utils.workspace_paths import resolve_workspace_path, validate_path_component
|
||||
from workflow.graph import GraphExecutor
|
||||
from workflow.graph_context import GraphContext
|
||||
|
||||
@ -34,7 +36,7 @@ _SSE_CONTENT_TYPE = "text/event-stream"
|
||||
|
||||
def _normalize_session_name(yaml_path: Path, session_name: Optional[str]) -> str:
|
||||
if session_name and session_name.strip():
|
||||
return session_name.strip()
|
||||
return validate_path_component(session_name.strip(), field="session_name")
|
||||
timestamp = datetime.now().strftime("%Y%m%d%H%M%S")
|
||||
return f"sdk_{yaml_path.stem}_{timestamp}"
|
||||
|
||||
@ -58,7 +60,7 @@ def _build_task_input(
|
||||
if not attachments:
|
||||
return prompt
|
||||
|
||||
attachments_dir = graph_context.directory / "code_workspace" / "attachments"
|
||||
attachments_dir = resolve_workspace_path(graph_context.directory, "code_workspace", "attachments")
|
||||
attachments_dir.mkdir(parents=True, exist_ok=True)
|
||||
store = AttachmentStore(attachments_dir)
|
||||
builder = TaskInputBuilder(store)
|
||||
@ -89,8 +91,8 @@ def _run_workflow_with_logger(
|
||||
details={"task_prompt_provided": bool(task_prompt)},
|
||||
)
|
||||
|
||||
design = load_config(yaml_path, vars_override=variables)
|
||||
normalized_session = _normalize_session_name(yaml_path, session_name)
|
||||
design = load_config(yaml_path, vars_override=variables)
|
||||
|
||||
graph_config = GraphConfig.from_definition(
|
||||
design.graph,
|
||||
@ -152,9 +154,13 @@ def _sse_event(event_type: str, data: Any) -> str:
|
||||
@router.post("/api/workflow/run")
|
||||
async def run_workflow_sync(request: WorkflowRunRequest, http_request: Request):
|
||||
try:
|
||||
if request.session_name and request.session_name.strip():
|
||||
validate_path_component(request.session_name.strip(), field="session_name")
|
||||
resolved_log_level: Optional[LogLevel] = None
|
||||
if request.log_level:
|
||||
resolved_log_level = LogLevel(request.log_level)
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except ValueError:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@ -162,13 +168,26 @@ async def run_workflow_sync(request: WorkflowRunRequest, http_request: Request):
|
||||
)
|
||||
|
||||
accepts_stream = _SSE_CONTENT_TYPE in (http_request.headers.get("accept") or "")
|
||||
try:
|
||||
attachments = []
|
||||
if request.attachments:
|
||||
if not request.attachment_session_id:
|
||||
raise SecurityError("attachments require attachment_session_id from the upload session")
|
||||
attachments = get_websocket_manager().attachment_service.resolve_uploaded_paths(
|
||||
request.attachment_session_id, request.attachments,
|
||||
)
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except SecurityError as exc:
|
||||
raise HTTPException(status_code=403, detail=str(exc))
|
||||
|
||||
if not accepts_stream:
|
||||
try:
|
||||
result = await run_in_threadpool(
|
||||
run_workflow,
|
||||
request.yaml_file,
|
||||
task_prompt=request.task_prompt,
|
||||
attachments=request.attachments,
|
||||
attachments=attachments,
|
||||
session_name=request.session_name,
|
||||
variables=request.variables,
|
||||
log_level=resolved_log_level,
|
||||
@ -177,6 +196,8 @@ async def run_workflow_sync(request: WorkflowRunRequest, http_request: Request):
|
||||
raise HTTPException(status_code=404, detail=str(exc))
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except SecurityError as exc:
|
||||
raise HTTPException(status_code=403, detail=str(exc))
|
||||
except Exception as exc:
|
||||
logger = get_server_logger()
|
||||
logger.log_exception(exc, "Failed to run workflow via sync API")
|
||||
@ -215,7 +236,7 @@ async def run_workflow_sync(request: WorkflowRunRequest, http_request: Request):
|
||||
final_message, meta = _run_workflow_with_logger(
|
||||
yaml_file=request.yaml_file,
|
||||
task_prompt=request.task_prompt,
|
||||
attachments=request.attachments,
|
||||
attachments=attachments,
|
||||
session_name=request.session_name,
|
||||
variables=request.variables,
|
||||
log_level=resolved_log_level,
|
||||
@ -230,7 +251,7 @@ async def run_workflow_sync(request: WorkflowRunRequest, http_request: Request):
|
||||
"output_dir": str(meta["output_dir"].resolve()),
|
||||
},
|
||||
)
|
||||
except (FileNotFoundError, ValidationError) as exc:
|
||||
except (FileNotFoundError, ValidationError, SecurityError) as exc:
|
||||
enqueue("error", {"message": str(exc)})
|
||||
except Exception as exc:
|
||||
logger = get_server_logger()
|
||||
@ -250,4 +271,4 @@ async def run_workflow_sync(request: WorkflowRunRequest, http_request: Request):
|
||||
if done_event.is_set():
|
||||
break
|
||||
|
||||
return StreamingResponse(stream(), media_type=_SSE_CONTENT_TYPE)
|
||||
return StreamingResponse(stream(), media_type=_SSE_CONTENT_TYPE)
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, HTTPException
|
||||
@ -8,8 +8,9 @@ from fastapi.responses import FileResponse
|
||||
from starlette.background import BackgroundTask
|
||||
|
||||
from server.settings import WARE_HOUSE_DIR
|
||||
from utils.exceptions import ResourceNotFoundError, ValidationError
|
||||
from utils.exceptions import ResourceNotFoundError, SecurityError, ValidationError
|
||||
from utils.structured_logger import get_server_logger, LogType
|
||||
from utils.workspace_paths import resolve_workspace_path, session_directory_name
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@ -29,8 +30,8 @@ async def download_session(session_id: str):
|
||||
field="session_id",
|
||||
)
|
||||
|
||||
dir_name = f"session_{session_id}"
|
||||
session_path = WARE_HOUSE_DIR / dir_name
|
||||
dir_name = session_directory_name(session_id)
|
||||
session_path = resolve_workspace_path(WARE_HOUSE_DIR, dir_name)
|
||||
|
||||
if not session_path.exists() or not session_path.is_dir():
|
||||
raise ResourceNotFoundError(
|
||||
@ -39,12 +40,24 @@ async def download_session(session_id: str):
|
||||
resource_id=session_id,
|
||||
)
|
||||
|
||||
if (WARE_HOUSE_DIR / dir_name).is_symlink():
|
||||
raise SecurityError("Session downloads must not follow symbolic links")
|
||||
entries = [session_path]
|
||||
for path in session_path.rglob("*"):
|
||||
if path.is_symlink():
|
||||
continue
|
||||
if not path.resolve().is_relative_to(session_path):
|
||||
raise SecurityError("Session downloads must not follow symbolic links")
|
||||
if path.is_file() or path.is_dir():
|
||||
entries.append(path)
|
||||
|
||||
with tempfile.NamedTemporaryFile(delete=False, suffix=".zip") as tmp_file:
|
||||
zip_path = Path(tmp_file.name)
|
||||
|
||||
archive_base = zip_path.with_suffix("")
|
||||
try:
|
||||
shutil.make_archive(str(archive_base), "zip", root_dir=WARE_HOUSE_DIR, base_dir=dir_name)
|
||||
with zipfile.ZipFile(zip_path, "w", compression=zipfile.ZIP_DEFLATED) as archive:
|
||||
for path in entries:
|
||||
archive.write(path, arcname=str(Path(dir_name) / path.relative_to(session_path)))
|
||||
except Exception as exc:
|
||||
if zip_path.exists():
|
||||
zip_path.unlink()
|
||||
@ -73,6 +86,8 @@ async def download_session(session_id: str):
|
||||
)
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except SecurityError as exc:
|
||||
raise HTTPException(status_code=403, detail=str(exc))
|
||||
except ResourceNotFoundError:
|
||||
raise HTTPException(status_code=404, detail="Session directory not found")
|
||||
except HTTPException:
|
||||
|
||||
@ -14,8 +14,8 @@ async def upload_attachment(session_id: str, file: UploadFile = File(...)):
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
try:
|
||||
record = await manager.attachment_service.save_upload_file(session_id, file)
|
||||
except ValidationError:
|
||||
raise HTTPException(status_code=400, detail="Session not connected")
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except Exception as exc:
|
||||
logger = get_server_logger()
|
||||
logger.error(
|
||||
@ -39,7 +39,7 @@ async def upload_attachment(session_id: str, file: UploadFile = File(...)):
|
||||
async def list_attachments(session_id: str):
|
||||
try:
|
||||
manager = ensure_known_session(session_id, require_connection=False)
|
||||
manifest = manager.attachment_service.list_attachment_manifests(session_id)
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
manifest = manager.attachment_service.list_attachment_manifests(session_id)
|
||||
return {"attachments": manifest}
|
||||
|
||||
@ -12,6 +12,8 @@ from fastapi import UploadFile
|
||||
|
||||
from entity.messages import MessageBlock, MessageBlockType
|
||||
from utils.attachments import AttachmentStore, AttachmentRecord
|
||||
from utils.exceptions import SecurityError, ValidationError
|
||||
from utils.workspace_paths import resolve_workspace_path, session_directory_name
|
||||
|
||||
|
||||
class AttachmentService:
|
||||
@ -60,6 +62,7 @@ class AttachmentService:
|
||||
return candidate
|
||||
|
||||
async def save_upload_file(self, session_id: str, upload: UploadFile) -> AttachmentRecord:
|
||||
self._session_attachments_path(session_id, create=False)
|
||||
filename = self._safe_upload_filename(upload.filename)
|
||||
temp_dir = Path(tempfile.mkdtemp(prefix="mac_upload_"))
|
||||
temp_path = temp_dir / filename
|
||||
@ -122,9 +125,52 @@ class AttachmentService:
|
||||
store = self.get_attachment_store(session_id)
|
||||
return store.export_manifest()
|
||||
|
||||
def resolve_uploaded_paths(self, session_id: str, attachments: List[str]) -> List[str]:
|
||||
"""Accept only registered uploads belonging to the specified session."""
|
||||
root = self._session_attachments_path(session_id, create=False)
|
||||
if root is None:
|
||||
raise SecurityError("Attachment upload session does not exist")
|
||||
manifest = root / "attachments_manifest.json"
|
||||
if manifest.is_symlink():
|
||||
raise SecurityError("Attachment manifest must not be a symbolic link")
|
||||
store = AttachmentStore(root)
|
||||
uploaded = {}
|
||||
for attachment_id in store.export_manifest():
|
||||
record = store.get(attachment_id)
|
||||
if (
|
||||
record.extra.get("origin") == "web_upload"
|
||||
and record.extra.get("session_id") == session_id
|
||||
and record.ref.local_path
|
||||
):
|
||||
uploaded[attachment_id] = Path(record.ref.local_path)
|
||||
|
||||
paths = []
|
||||
for value in attachments:
|
||||
try:
|
||||
source = uploaded.get(value)
|
||||
if source is None:
|
||||
requested = Path(value).expanduser().resolve()
|
||||
source = next((path for path in uploaded.values() if path.resolve() == requested), None)
|
||||
if source is None:
|
||||
raise SecurityError("Attachment must reference a registered upload in attachment_session_id")
|
||||
resolved = source.resolve()
|
||||
if source.is_symlink() or not resolved.is_relative_to(root) or not resolved.is_file():
|
||||
raise SecurityError("Attachment source must stay inside its upload session")
|
||||
paths.append(str(resolved))
|
||||
except (OSError, RuntimeError, ValueError) as exc:
|
||||
raise SecurityError("Invalid uploaded attachment source") from exc
|
||||
return paths
|
||||
|
||||
def _session_attachments_path(self, session_id: str, *, create: bool = True) -> Optional[Path]:
|
||||
session_dir_name = session_id if session_id.startswith("session_") else f"session_{session_id}"
|
||||
path = self.attachments_root / session_dir_name / "code_workspace" / "attachments"
|
||||
session_dir_name = session_directory_name(session_id)
|
||||
current = self.attachments_root
|
||||
for part in (session_dir_name, "code_workspace", "attachments"):
|
||||
current = current / part
|
||||
if current.is_symlink():
|
||||
raise ValidationError("Upload session paths must not contain symbolic links", field="session_id")
|
||||
path = resolve_workspace_path(
|
||||
self.attachments_root, session_dir_name, "code_workspace", "attachments"
|
||||
)
|
||||
if create:
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
@ -15,6 +15,7 @@ from entity.enums import LogLevel
|
||||
from entity.graph_config import GraphConfig
|
||||
from utils.exceptions import ValidationError
|
||||
from utils.task_input import TaskInputBuilder
|
||||
from utils.workspace_paths import resolve_workspace_path, session_directory_name
|
||||
from workflow.graph import GraphExecutor
|
||||
from workflow.graph_context import GraphContext
|
||||
|
||||
@ -160,7 +161,7 @@ class BatchRunService:
|
||||
)
|
||||
|
||||
def _write_batch_outputs(self, session_id: str, result_rows: List[Dict[str, Any]]) -> None:
|
||||
output_root = WARE_HOUSE_DIR / f"session_{session_id}"
|
||||
output_root = resolve_workspace_path(WARE_HOUSE_DIR, session_directory_name(session_id))
|
||||
output_root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
csv_path = output_root / "batch_results.csv"
|
||||
@ -205,7 +206,7 @@ class BatchRunService:
|
||||
details={"yaml_file": yaml_file},
|
||||
)
|
||||
|
||||
output_root = WARE_HOUSE_DIR / f"session_{session_id}"
|
||||
output_root = resolve_workspace_path(WARE_HOUSE_DIR, session_directory_name(session_id))
|
||||
graph_config = GraphConfig.from_definition(
|
||||
design.graph,
|
||||
name=task_dir,
|
||||
|
||||
@ -8,13 +8,15 @@ import traceback
|
||||
import uuid
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from fastapi import WebSocket
|
||||
from fastapi import WebSocket, WebSocketException
|
||||
|
||||
from server.services.message_handler import MessageHandler
|
||||
from server.services.attachment_service import AttachmentService
|
||||
from server.services.session_execution import SessionExecutionController
|
||||
from server.services.session_store import WorkflowSessionStore, SessionStatus
|
||||
from server.services.workflow_run_service import WorkflowRunService
|
||||
from utils.exceptions import ValidationError
|
||||
from utils.workspace_paths import validate_path_component
|
||||
|
||||
|
||||
def _json_default(value):
|
||||
@ -68,6 +70,11 @@ class WebSocketManager:
|
||||
)
|
||||
|
||||
async def connect(self, websocket: WebSocket, session_id: Optional[str] = None) -> str:
|
||||
if session_id:
|
||||
try:
|
||||
validate_path_component(session_id, field="session_id")
|
||||
except ValidationError as exc:
|
||||
raise WebSocketException(code=1008, reason=str(exc)) from exc
|
||||
await websocket.accept()
|
||||
# Capture the event loop that owns the WebSocket connections so that
|
||||
# worker threads can safely schedule sends via run_coroutine_threadsafe.
|
||||
|
||||
@ -11,6 +11,7 @@ from entity.enums import LogLevel
|
||||
from utils.exceptions import ValidationError, WorkflowCancelledError
|
||||
from utils.structured_logger import get_server_logger, LogType
|
||||
from utils.task_input import TaskInputBuilder
|
||||
from utils.workspace_paths import session_directory_name
|
||||
from workflow.graph_context import GraphContext
|
||||
|
||||
from server.services.attachment_service import AttachmentService
|
||||
@ -148,7 +149,7 @@ class WorkflowRunService:
|
||||
design = load_config(yaml_path)
|
||||
graph_config = GraphConfig.from_definition(
|
||||
design.graph,
|
||||
name=f"session_{session_id}",
|
||||
name=session_directory_name(session_id),
|
||||
output_root=WARE_HOUSE_DIR,
|
||||
source_path=str(yaml_path),
|
||||
vars=design.vars,
|
||||
|
||||
310
tests/test_attachment_boundary_security.py
Normal file
310
tests/test_attachment_boundary_security.py
Normal file
@ -0,0 +1,310 @@
|
||||
"""Regression tests for HTTP attachment sources and generated attachment paths."""
|
||||
|
||||
import base64
|
||||
import io
|
||||
import sys
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from entity.messages import AttachmentRef, Message, MessageBlock, MessageBlockType, MessageRole
|
||||
from runtime import sdk
|
||||
from runtime.node.agent import ModelResponse, ProviderRegistry
|
||||
from runtime.node.executor.agent_executor import AgentNodeExecutor
|
||||
from runtime.node.executor.base import ExecutionContext
|
||||
from server import state
|
||||
from server.routes import execute_sync, sessions, uploads, websocket, workflows
|
||||
from server.services.websocket_manager import WebSocketManager
|
||||
from utils.attachments import AttachmentStore
|
||||
from utils.error_handler import add_exception_handlers
|
||||
from utils.exceptions import SecurityError
|
||||
|
||||
|
||||
DEMO = Path(__file__).resolve().parents[1] / "yaml_instance/demo_loop_counter.yaml"
|
||||
CONTENT = b"generated attachment content"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def api(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setenv("MAC_ALLOW_UNSAFE_CODE_EXECUTION", "0")
|
||||
manager = WebSocketManager()
|
||||
monkeypatch.setattr(state, "websocket_manager", manager)
|
||||
monkeypatch.setattr(sessions, "WARE_HOUSE_DIR", tmp_path / "WareHouse")
|
||||
yaml_dir = tmp_path / "yaml_instance"
|
||||
yaml_dir.mkdir()
|
||||
for module in (sdk, execute_sync, workflows):
|
||||
monkeypatch.setattr(module, "YAML_DIR", yaml_dir)
|
||||
app = FastAPI()
|
||||
add_exception_handlers(app)
|
||||
for module in (execute_sync, sessions, uploads, websocket, workflows):
|
||||
app.include_router(module.router)
|
||||
with TestClient(app) as client:
|
||||
yield client, manager
|
||||
|
||||
|
||||
def upload(client, sid, data=b"uploaded input"):
|
||||
with client.websocket_connect("/ws?session_id=" + sid) as ws:
|
||||
ws.receive_json()
|
||||
response = client.post("/api/uploads/" + sid, files={"file": ("input.txt", data)})
|
||||
assert response.status_code == 200, response.text
|
||||
manifest = client.get("/api/uploads/" + sid).json()["attachments"]
|
||||
attachment_id = response.json()["attachment_id"]
|
||||
return attachment_id, Path(manifest[attachment_id]["ref"]["local_path"])
|
||||
|
||||
|
||||
def request_run(client, accept, **overrides):
|
||||
payload = {"yaml_file": str(DEMO), "task_prompt": "test", "session_name": "session_result"}
|
||||
payload.update(overrides)
|
||||
return client.post("/api/workflow/run", headers={"Accept": accept}, json=payload)
|
||||
|
||||
|
||||
def archive_contents(client, sid):
|
||||
response = client.get(f"/api/sessions/{sid}/download")
|
||||
assert response.status_code == 200, response.text
|
||||
with zipfile.ZipFile(io.BytesIO(response.content)) as archive:
|
||||
return {name: archive.read(name) for name in archive.namelist()}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("accept", ["application/json", "text/event-stream"])
|
||||
@pytest.mark.parametrize("source_kind", ["absolute", "relative", "symlink", "unregistered", "other_session", "other_id"])
|
||||
def test_http_rejects_untrusted_sources_before_copying(api, tmp_path, accept, source_kind):
|
||||
client, _ = api
|
||||
own_id, own_path = upload(client, "owner")
|
||||
other_id, other_path = upload(client, "other", b"other session secret")
|
||||
secret = tmp_path / "secret.txt"
|
||||
secret.write_text("outside workspace secret")
|
||||
if source_kind == "symlink":
|
||||
own_path.unlink()
|
||||
own_path.symlink_to(secret)
|
||||
unregistered = own_path.parent / "unregistered.txt"
|
||||
unregistered.write_text("not uploaded")
|
||||
source = {"absolute": str(secret), "relative": "secret.txt", "symlink": own_id,
|
||||
"unregistered": str(unregistered), "other_session": str(other_path), "other_id": other_id}[source_kind]
|
||||
response = request_run(client, accept, attachments=[own_id, source], attachment_session_id="owner")
|
||||
assert response.status_code == 403, response.text
|
||||
assert not (tmp_path / "WareHouse/session_result").exists()
|
||||
assert client.get("/api/sessions/result/download").status_code == 404
|
||||
assert secret.read_text() == "outside workspace secret"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("accept", ["application/json", "text/event-stream"])
|
||||
def test_http_paths_without_upload_session_are_rejected(api, tmp_path, accept):
|
||||
client, _ = api
|
||||
secret = tmp_path / "secret.txt"
|
||||
secret.write_text("secret")
|
||||
response = request_run(client, accept, attachments=[str(secret)])
|
||||
assert response.status_code == 403
|
||||
assert "attachment_session_id" in response.text
|
||||
assert not (tmp_path / "WareHouse/session_result").exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("accept", ["application/json", "text/event-stream"])
|
||||
@pytest.mark.parametrize("use_id", [False, True])
|
||||
def test_uploaded_attachment_runs_and_downloads(api, accept, use_id):
|
||||
client, _ = api
|
||||
attachment_id, path = upload(client, "owner")
|
||||
response = request_run(client, accept, attachments=[attachment_id if use_id else str(path)],
|
||||
attachment_session_id="owner")
|
||||
assert response.status_code == 200, response.text
|
||||
assert "Final summary released" in response.text
|
||||
assert "event: error" not in response.text
|
||||
contents = archive_contents(client, "result")
|
||||
assert b"uploaded input" in contents.values()
|
||||
|
||||
|
||||
def test_local_sdk_still_accepts_explicit_file_paths(api, tmp_path):
|
||||
local = tmp_path / "local.txt"
|
||||
local.write_bytes(b"local CLI input")
|
||||
result = sdk.run_workflow(DEMO, task_prompt="test", attachments=[local], session_name="session_sdk")
|
||||
assert result.final_message.text_content() == "Final summary released"
|
||||
assert next(result.meta_info.output_dir.rglob("local.txt")).read_bytes() == b"local CLI input"
|
||||
|
||||
|
||||
def test_download_preserves_empty_directories(api, tmp_path):
|
||||
client, _ = api
|
||||
(tmp_path / "WareHouse/session_empty/nested").mkdir(parents=True)
|
||||
contents = archive_contents(client, "empty")
|
||||
assert "session_empty/" in contents
|
||||
assert "session_empty/nested/" in contents
|
||||
|
||||
|
||||
def test_prefixed_sessions_have_separate_uploads_execution_and_downloads(api, tmp_path):
|
||||
client, manager = api
|
||||
first_id, first_path = upload(client, "foo", b"first")
|
||||
second_id, second_path = upload(client, "session_foo", b"second")
|
||||
assert first_path.is_relative_to(tmp_path / "WareHouse/session_foo")
|
||||
assert second_path.is_relative_to(tmp_path / "WareHouse/session_session_foo")
|
||||
assert set(manager.attachment_service.list_attachment_manifests("foo")) == {first_id}
|
||||
assert set(manager.attachment_service.list_attachment_manifests("session_foo")) == {second_id}
|
||||
for sid, attachment_id, expected in (("foo", first_id, b"first"), ("session_foo", second_id, b"second")):
|
||||
contents = archive_contents(client, sid)
|
||||
assert expected in contents.values()
|
||||
assert (b"second" if expected == b"first" else b"first") not in contents.values()
|
||||
assert manager.attachment_service.build_attachment_blocks(sid, [attachment_id])
|
||||
assert not manager.attachment_service.build_attachment_blocks("foo", [second_id])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", ["file", "directory", "dangling", "loop"])
|
||||
def test_download_skips_links_and_preserves_regular_files(api, tmp_path, kind):
|
||||
client, _ = api
|
||||
secret_dir = tmp_path / "outside"
|
||||
secret_dir.mkdir()
|
||||
(secret_dir / "secret.txt").write_text("secret")
|
||||
session = tmp_path / "WareHouse/session_download"
|
||||
session.mkdir()
|
||||
(session / "run.log").write_bytes(b"workflow log")
|
||||
target = {"file": secret_dir / "secret.txt", "directory": secret_dir,
|
||||
"dangling": secret_dir / "missing", "loop": session / "link"}[kind]
|
||||
(session / "link").symlink_to(target, target_is_directory=kind == "directory")
|
||||
contents = archive_contents(client, "download")
|
||||
assert contents["session_download/run.log"] == b"workflow log"
|
||||
assert not any("link" in Path(name).parts for name in contents)
|
||||
assert b"secret" not in contents.values()
|
||||
|
||||
|
||||
def test_download_still_rejects_linked_session_root(api, tmp_path):
|
||||
client, _ = api
|
||||
secret_dir = tmp_path / "outside"
|
||||
secret_dir.mkdir()
|
||||
(secret_dir / "secret.txt").write_text("secret")
|
||||
(tmp_path / "WareHouse/session_download").symlink_to(secret_dir, target_is_directory=True)
|
||||
response = client.get("/api/sessions/download/download")
|
||||
assert response.status_code in (400, 403)
|
||||
assert "application/zip" not in response.headers.get("content-type", "")
|
||||
|
||||
|
||||
def test_download_with_venv_interpreter_links(api, tmp_path):
|
||||
client, _ = api
|
||||
session = tmp_path / "WareHouse/session_venv"
|
||||
venv = session / "code_workspace/.venv"
|
||||
(venv / "bin").mkdir(parents=True)
|
||||
(venv / "bin/python").symlink_to(sys.executable)
|
||||
(venv / "bin/python3").symlink_to("python")
|
||||
(venv / "pyvenv.cfg").write_bytes(b"include-system-site-packages = false\n")
|
||||
(session / "run.log").write_bytes(b"workflow completed")
|
||||
contents = archive_contents(client, "venv")
|
||||
assert contents["session_venv/run.log"] == b"workflow completed"
|
||||
assert contents["session_venv/code_workspace/.venv/pyvenv.cfg"] == (venv / "pyvenv.cfg").read_bytes()
|
||||
assert "session_venv/code_workspace/.venv/bin/python" not in contents
|
||||
assert "session_venv/code_workspace/.venv/bin/python3" not in contents
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def agent(tmp_path):
|
||||
workspace = tmp_path / "workspace"
|
||||
store = AttachmentStore(workspace / "attachments")
|
||||
context = ExecutionContext(tool_manager=Mock(), function_manager=Mock(), log_manager=Mock(),
|
||||
global_state={"python_workspace_root": workspace, "attachment_store": store})
|
||||
return AgentNodeExecutor(context), workspace, store
|
||||
|
||||
|
||||
def attachment_message(name="result.txt", attachment_id="generated", local_path=None):
|
||||
ref = AttachmentRef(attachment_id=attachment_id, name=name, mime_type="text/plain",
|
||||
local_path=str(local_path) if local_path else None,
|
||||
data_uri=None if local_path else "data:text/plain;base64," + base64.b64encode(CONTENT).decode())
|
||||
return Message(role=MessageRole.ASSISTANT, content=[MessageBlock(type=MessageBlockType.FILE, attachment=ref)])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["node_id", "name", "attachment_id"])
|
||||
@pytest.mark.parametrize("value", ["../outside", "/absolute", r"..\outside", "C:outside", ".", "..", "bad\x00name"])
|
||||
def test_generated_paths_reject_untrusted_components(agent, tmp_path, field, value):
|
||||
executor, workspace, store = agent
|
||||
node_id = value if field == "node_id" else "Generator"
|
||||
message = attachment_message(**({field: value} if field != "node_id" else {}))
|
||||
with pytest.raises(SecurityError):
|
||||
executor._persist_message_attachments(message, node_id)
|
||||
assert not (workspace / "generated").exists()
|
||||
assert not store.export_manifest()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("part", ["generated", "generated/Generator", "generated/Generator/result.txt"])
|
||||
def test_generated_paths_reject_symlinks(agent, tmp_path, part):
|
||||
executor, workspace, _ = agent
|
||||
outside = tmp_path / "outside"
|
||||
outside.mkdir()
|
||||
sentinel = outside / "result.txt"
|
||||
sentinel.write_text("original")
|
||||
link = workspace / part
|
||||
link.parent.mkdir(parents=True, exist_ok=True)
|
||||
link.symlink_to(sentinel if part.endswith(".txt") else outside, target_is_directory=not part.endswith(".txt"))
|
||||
with pytest.raises(SecurityError):
|
||||
executor._persist_message_attachments(attachment_message(), "Generator")
|
||||
assert sentinel.read_text() == "original"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("linked", [False, True])
|
||||
def test_generated_local_source_cannot_read_outside(agent, tmp_path, linked):
|
||||
executor, workspace, _ = agent
|
||||
secret = tmp_path / "secret.txt"
|
||||
secret.write_text("secret")
|
||||
source = secret
|
||||
if linked:
|
||||
source = workspace / "alias.txt"
|
||||
source.symlink_to(secret)
|
||||
with pytest.raises(SecurityError):
|
||||
executor._persist_message_attachments(attachment_message(local_path=source), "Generator")
|
||||
assert not (workspace / "generated").exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("local", [False, True])
|
||||
def test_generated_attachment_normal_round_trip(agent, local):
|
||||
executor, workspace, store = agent
|
||||
source = None
|
||||
if local:
|
||||
source = workspace / "input.txt"
|
||||
source.write_bytes(CONTENT)
|
||||
message = attachment_message(local_path=source)
|
||||
executor._persist_message_attachments(message, "Image Generator")
|
||||
target = workspace / "generated/Image Generator/result.txt"
|
||||
assert target.read_bytes() == CONTENT
|
||||
assert Path(message.blocks()[0].attachment.local_path) == target
|
||||
assert store.get("generated").ref.local_path == str(target)
|
||||
|
||||
|
||||
def test_generated_image_infers_extension_from_data_uri(agent):
|
||||
executor, workspace, _ = agent
|
||||
message = attachment_message(name=None)
|
||||
ref = message.blocks()[0].attachment
|
||||
ref.mime_type = None
|
||||
ref.data_uri = "data:image/png;base64," + base64.b64encode(CONTENT).decode()
|
||||
executor._persist_message_attachments(message, "Generator")
|
||||
assert (workspace / "generated/Generator/generated.png").read_bytes() == CONTENT
|
||||
assert message.blocks()[0].attachment.mime_type == "image/png"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("accept", ["application/json", "text/event-stream"])
|
||||
@pytest.mark.parametrize("malicious", ["node", "name", "id", "none"])
|
||||
def test_agent_attachment_security_reaches_http_api(api, tmp_path, monkeypatch, accept, malicious):
|
||||
client, _ = api
|
||||
outside = tmp_path / "outside"
|
||||
node_id = str(outside) if malicious == "node" else "Generator"
|
||||
message = attachment_message(name="../../../../escaped.txt" if malicious == "name" else "result.txt",
|
||||
attachment_id="../../escape" if malicious == "id" else "generated")
|
||||
provider = Mock()
|
||||
provider.call_model.return_value = ModelResponse(message=message)
|
||||
monkeypatch.setattr(ProviderRegistry, "get_provider", lambda _: Mock(return_value=provider))
|
||||
definition = {"version": "0.4.0", "graph": {"id": "attachment_test", "start": [node_id], "nodes": [
|
||||
{"id": node_id, "type": "agent", "config": {"provider": "openai", "name": "test-model",
|
||||
"api_key": "local-test", "role": "test"}},
|
||||
], "edges": []}}
|
||||
response = client.post("/api/workflows/upload/content", json={"filename": "agent.yaml", "content": yaml.safe_dump(definition)})
|
||||
assert response.status_code == 200, response.text
|
||||
response = request_run(client, accept, yaml_file="agent.yaml")
|
||||
if malicious == "none":
|
||||
assert response.status_code == 200, response.text
|
||||
assert "event: error" not in response.text
|
||||
assert CONTENT in archive_contents(client, "result").values()
|
||||
elif accept == "text/event-stream":
|
||||
assert response.status_code == 200
|
||||
assert "event: error" in response.text
|
||||
assert "event: completed" not in response.text
|
||||
else:
|
||||
assert response.status_code == 403, response.text
|
||||
assert not outside.exists()
|
||||
assert not (tmp_path / "escaped.txt").exists()
|
||||
440
tests/test_code_execution_security.py
Normal file
440
tests/test_code_execution_security.py
Normal file
@ -0,0 +1,440 @@
|
||||
"""Guard untrusted code entry points and processor file writes."""
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from runtime import sdk
|
||||
from entity.configs import Node
|
||||
from entity.configs.node.tooling import McpLocalConfig
|
||||
from entity.messages import Message, MessageRole
|
||||
from functions.edge_processor import transformers
|
||||
from functions.function_calling.code_executor import execute_code
|
||||
from functions.function_calling import uv_related, video
|
||||
from runtime.node.agent.tool import tool_manager
|
||||
from runtime.node.executor.base import ExecutionContext
|
||||
from runtime.node.executor.python_executor import PythonNodeExecutor
|
||||
from server.routes import execute_sync, workflows
|
||||
from utils.attachments import AttachmentStore
|
||||
from utils.code_execution import UNSAFE_CODE_EXECUTION_ENV, require_unsafe_code_execution
|
||||
from utils.error_handler import add_exception_handlers
|
||||
from utils.exceptions import SecurityError
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def disable_execution(monkeypatch):
|
||||
monkeypatch.delenv(UNSAFE_CODE_EXECUTION_ENV, raising=False)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace_context(tmp_path):
|
||||
workspace = tmp_path / "session" / "code_workspace"
|
||||
store = AttachmentStore(workspace / "attachments")
|
||||
(workspace / "keep.txt").write_text("original")
|
||||
(store.root / "input.txt").write_text("attachment")
|
||||
return {
|
||||
"python_workspace_root": workspace,
|
||||
"graph_directory": workspace.parent,
|
||||
"attachment_store": store,
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_launch(monkeypatch):
|
||||
process = Mock(returncode=0)
|
||||
process.communicate.return_value = (b"executed\n", b"")
|
||||
launch = Mock(return_value=process)
|
||||
monkeypatch.setattr(transformers.subprocess, "Popen", launch)
|
||||
return launch
|
||||
|
||||
|
||||
def _code_block(filename, code="print('ok')"):
|
||||
return f"{filename}\n```python\n{code}\n```"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [None, "", "0", "false", "no", "off", "typo"])
|
||||
def test_execution_requires_explicit_opt_in(monkeypatch, value):
|
||||
if value is not None:
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, value)
|
||||
with pytest.raises(SecurityError, match=UNSAFE_CODE_EXECUTION_ENV):
|
||||
require_unsafe_code_execution("test operation")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["1", "true", "YES", " on "])
|
||||
def test_explicit_opt_in_is_supported(monkeypatch, value):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, value)
|
||||
require_unsafe_code_execution("test operation")
|
||||
|
||||
|
||||
def test_disabled_processor_does_not_clear_or_write_files(workspace_context, fake_launch):
|
||||
workspace_context[UNSAFE_CODE_EXECUTION_ENV] = "1"
|
||||
workspace_context["vars"] = {UNSAFE_CODE_EXECUTION_ENV: "1"}
|
||||
with pytest.raises(SecurityError, match=UNSAFE_CODE_EXECUTION_ENV):
|
||||
transformers.code_save_and_run(_code_block("main.py"), workspace_context)
|
||||
workspace = workspace_context["python_workspace_root"]
|
||||
assert (workspace / "keep.txt").read_text() == "original"
|
||||
assert not (workspace / "main.py").exists()
|
||||
fake_launch.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("filename", [
|
||||
"../../escaped.py", "src/../../escaped.py", "src/../main.py",
|
||||
r"..\..\escaped.py", r"C:\escaped.py", "C:escaped.py",
|
||||
r"\\server\share\escaped.py", "attachments/changed.py",
|
||||
r"attachments\changed.py", "attachments", ".", "..", " ", "bad\x00name.py",
|
||||
])
|
||||
def test_processor_validates_all_files_before_clearing(
|
||||
workspace_context, fake_launch, monkeypatch, filename
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
payload = _code_block("main.py") + "\n" + _code_block(filename)
|
||||
with pytest.raises(SecurityError):
|
||||
transformers.code_save_and_run(payload, workspace_context)
|
||||
workspace = workspace_context["python_workspace_root"]
|
||||
assert (workspace / "keep.txt").read_text() == "original"
|
||||
assert (workspace / "attachments/input.txt").read_text() == "attachment"
|
||||
assert not (workspace / "main.py").exists()
|
||||
fake_launch.assert_not_called()
|
||||
|
||||
|
||||
def test_processor_rejects_absolute_path(workspace_context, tmp_path, fake_launch, monkeypatch):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
target = tmp_path / "outside.py"
|
||||
with pytest.raises(SecurityError):
|
||||
transformers.code_save_and_run(_code_block(str(target)), workspace_context)
|
||||
assert not target.exists()
|
||||
assert (workspace_context["python_workspace_root"] / "keep.txt").exists()
|
||||
fake_launch.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("target_kind", ["outside", "attachments"])
|
||||
def test_processor_rejects_symlink_targets(
|
||||
workspace_context, tmp_path, fake_launch, monkeypatch, target_kind
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
workspace = workspace_context["python_workspace_root"]
|
||||
target = tmp_path / "outside" if target_kind == "outside" else workspace / "attachments"
|
||||
target.mkdir(exist_ok=True)
|
||||
(workspace / "alias").symlink_to(target, target_is_directory=True)
|
||||
with pytest.raises(SecurityError):
|
||||
transformers.code_save_and_run(_code_block("alias/changed.py"), workspace_context)
|
||||
assert not (target / "changed.py").exists()
|
||||
assert (workspace / "keep.txt").exists()
|
||||
assert (workspace / "alias").is_symlink()
|
||||
fake_launch.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("filename", [
|
||||
"Attachments/input.txt", "ATTACHMENTS/nested/new.py", "aTtAcHmEnTs",
|
||||
])
|
||||
def test_processor_rejects_attachment_case_aliases(
|
||||
workspace_context, fake_launch, monkeypatch, filename
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
workspace = workspace_context["python_workspace_root"]
|
||||
attachments = workspace / "attachments"
|
||||
alias = workspace / "Attachments"
|
||||
if not alias.exists() or not alias.samefile(attachments):
|
||||
pytest.skip("Requires a case-insensitive filesystem")
|
||||
|
||||
payload = _code_block("main.py") + "\n" + _code_block(filename, "REVIEW_MARKER")
|
||||
with pytest.raises(SecurityError, match="attachments"):
|
||||
transformers.code_save_and_run(payload, workspace_context)
|
||||
|
||||
assert (attachments / "input.txt").read_text() == "attachment"
|
||||
assert not (attachments / "nested").exists()
|
||||
assert (workspace / "keep.txt").read_text() == "original"
|
||||
assert not (workspace / "main.py").exists()
|
||||
fake_launch.assert_not_called()
|
||||
|
||||
|
||||
def test_processor_cleanup_preserves_attachment_directory_with_different_case(
|
||||
workspace_context, fake_launch, monkeypatch
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
workspace = workspace_context["python_workspace_root"]
|
||||
attachments = workspace / "attachments"
|
||||
alias = workspace / "Attachments"
|
||||
if not alias.exists() or not alias.samefile(attachments):
|
||||
pytest.skip("Requires a case-insensitive filesystem")
|
||||
|
||||
temporary = workspace / "inputs_to_rename"
|
||||
attachments.rename(temporary)
|
||||
temporary.rename(alias)
|
||||
assert "Attachments" in {item.name for item in workspace.iterdir()}
|
||||
|
||||
transformers.code_save_and_run(_code_block("main.py"), workspace_context)
|
||||
|
||||
assert (attachments / "input.txt").read_text() == "attachment"
|
||||
assert (workspace / "main.py").exists()
|
||||
assert not (workspace / "keep.txt").exists()
|
||||
fake_launch.assert_called_once()
|
||||
|
||||
|
||||
def test_processor_allows_distinct_case_sensitive_directory(
|
||||
workspace_context, fake_launch, monkeypatch
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
workspace = workspace_context["python_workspace_root"]
|
||||
alias = workspace / "Attachments"
|
||||
if alias.exists():
|
||||
pytest.skip("Requires a case-sensitive filesystem")
|
||||
|
||||
transformers.code_save_and_run(_code_block("Attachments/new.py", "VALUE = 1"), workspace_context)
|
||||
|
||||
assert (alias / "new.py").read_text() == "VALUE = 1"
|
||||
assert (workspace / "attachments/input.txt").read_text() == "attachment"
|
||||
fake_launch.assert_called_once()
|
||||
|
||||
|
||||
def test_enabled_processor_preserves_nested_files_and_attachments(
|
||||
workspace_context, tmp_path, fake_launch, monkeypatch
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
workspace = workspace_context["python_workspace_root"]
|
||||
outside = tmp_path / "outside"
|
||||
outside.mkdir()
|
||||
(outside / "keep.txt").write_text("external")
|
||||
(workspace / "old_link").symlink_to(outside, target_is_directory=True)
|
||||
payload = _code_block("main.py") + "\n" + _code_block("src/helpers.py", "VALUE = 1")
|
||||
result = transformers.code_save_and_run(payload, workspace_context)
|
||||
assert (workspace / "main.py").read_text() == "print('ok')"
|
||||
assert (workspace / "src/helpers.py").read_text() == "VALUE = 1"
|
||||
assert not (workspace / "keep.txt").exists()
|
||||
assert not (workspace / "old_link").is_symlink()
|
||||
assert (outside / "keep.txt").read_text() == "external"
|
||||
assert (workspace / "attachments/input.txt").read_text() == "attachment"
|
||||
assert "executed" in result
|
||||
args, kwargs = fake_launch.call_args
|
||||
assert args == (["uv", "run", "main.py"],)
|
||||
assert kwargs["cwd"] == str(workspace)
|
||||
assert not kwargs.get("shell", False)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("target_kind", ["inside", "outside"])
|
||||
def test_processor_rejects_linked_attachments_before_cleanup(
|
||||
workspace_context, tmp_path, fake_launch, monkeypatch, target_kind
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
workspace = workspace_context["python_workspace_root"]
|
||||
target = (workspace if target_kind == "inside" else tmp_path) / "inputs"
|
||||
attachments = workspace / "attachments"
|
||||
attachments.rename(target)
|
||||
attachments.symlink_to(target, target_is_directory=True)
|
||||
with pytest.raises(SecurityError, match="symbolic link"):
|
||||
transformers.code_save_and_run(_code_block("main.py"), workspace_context)
|
||||
assert (target / "input.txt").read_text() == "attachment"
|
||||
assert (workspace / "keep.txt").read_text() == "original"
|
||||
assert not (workspace / "main.py").exists()
|
||||
fake_launch.assert_not_called()
|
||||
|
||||
|
||||
def _python_executor(workspace, env=None):
|
||||
context = ExecutionContext(
|
||||
tool_manager=Mock(), function_manager=Mock(), log_manager=Mock(),
|
||||
global_state={"python_workspace_root": workspace, "vars": {UNSAFE_CODE_EXECUTION_ENV: "1"}},
|
||||
)
|
||||
node = Node.from_dict(
|
||||
{"id": "Runner", "type": "python", "config": {"env": env or {}}}, path="node"
|
||||
)
|
||||
return PythonNodeExecutor(context), node
|
||||
|
||||
|
||||
def test_python_node_env_cannot_enable_execution(tmp_path, monkeypatch):
|
||||
workspace = tmp_path / "workspace"
|
||||
executor, node = _python_executor(workspace, {UNSAFE_CODE_EXECUTION_ENV: "1"})
|
||||
run = Mock(side_effect=AssertionError("Must not start a process"))
|
||||
monkeypatch.setattr(subprocess, "run", run)
|
||||
with pytest.raises(SecurityError, match=UNSAFE_CODE_EXECUTION_ENV):
|
||||
executor.execute(node, [Message(role=MessageRole.USER, content="print('ok')")])
|
||||
assert not workspace.exists()
|
||||
run.assert_not_called()
|
||||
|
||||
|
||||
def test_enabled_python_node_executes_harmless_code(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
executor, node = _python_executor(tmp_path / "workspace")
|
||||
messages = executor.execute(node, [Message(role=MessageRole.USER, content="print(6 * 7)")])
|
||||
assert messages[0].text_content().strip() == "42"
|
||||
|
||||
|
||||
def test_execute_code_is_blocked_before_saving_script(tmp_path, monkeypatch):
|
||||
temp_code_dir = tmp_path / "scripts"
|
||||
monkeypatch.setenv("TEMP_CODE_DIR", str(temp_code_dir))
|
||||
with pytest.raises(SecurityError, match=UNSAFE_CODE_EXECUTION_ENV):
|
||||
execute_code("print('ok')")
|
||||
assert not temp_code_dir.exists()
|
||||
|
||||
|
||||
def test_execute_code_works_when_enabled(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
monkeypatch.setenv("TEMP_CODE_DIR", str(tmp_path))
|
||||
assert execute_code("print(6 * 7)").strip() == "42"
|
||||
assert list(tmp_path.iterdir()) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("operation", ["run", "install", "init"])
|
||||
def test_uv_entry_points_cannot_override_server_policy(workspace_context, monkeypatch, operation):
|
||||
run = Mock(side_effect=AssertionError("Must not start a process"))
|
||||
monkeypatch.setattr(uv_related.subprocess, "run", run)
|
||||
with pytest.raises(SecurityError, match=UNSAFE_CODE_EXECUTION_ENV):
|
||||
if operation == "run":
|
||||
uv_related.uv_run(
|
||||
module="example", env={UNSAFE_CODE_EXECUTION_ENV: "1"}, _context=workspace_context
|
||||
)
|
||||
elif operation == "install":
|
||||
uv_related.install_python_packages(["example"], _context=workspace_context)
|
||||
else:
|
||||
uv_related.init_python_env(_context=workspace_context)
|
||||
run.assert_not_called()
|
||||
|
||||
|
||||
def test_uv_run_is_available_when_enabled(workspace_context, monkeypatch):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
run = Mock(return_value=subprocess.CompletedProcess([], 0, stdout="ok", stderr=""))
|
||||
monkeypatch.setattr(uv_related.subprocess, "run", run)
|
||||
result = uv_related.uv_run(module="example", _context=workspace_context)
|
||||
assert result["returncode"] == 0
|
||||
assert run.call_args.args[0] == ["uv", "run", "python", "-m", "example"]
|
||||
|
||||
|
||||
def test_manim_is_blocked_before_clearing_media(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
media = tmp_path / "media"
|
||||
media.mkdir()
|
||||
(media / "keep.txt").write_text("original")
|
||||
with pytest.raises(SecurityError, match=UNSAFE_CODE_EXECUTION_ENV):
|
||||
video.render_manim("unused.py")
|
||||
assert (media / "keep.txt").read_text() == "original"
|
||||
|
||||
|
||||
def test_local_mcp_env_cannot_enable_execution(monkeypatch):
|
||||
create_client = Mock(side_effect=AssertionError("Must not start MCP"))
|
||||
monkeypatch.setattr(tool_manager, "_StdioClientWrapper", create_client)
|
||||
manager = tool_manager.ToolManager()
|
||||
config = McpLocalConfig.from_dict(
|
||||
{"command": sys.executable, "env": {UNSAFE_CODE_EXECUTION_ENV: "1"}}, path="tooling"
|
||||
)
|
||||
with pytest.raises(SecurityError, match=UNSAFE_CODE_EXECUTION_ENV):
|
||||
manager._get_stdio_client(config, config.cache_key())
|
||||
create_client.assert_not_called()
|
||||
|
||||
|
||||
def test_local_mcp_opt_in_and_cached_client_policy(monkeypatch):
|
||||
manager = tool_manager.ToolManager()
|
||||
config = McpLocalConfig.from_dict({"command": sys.executable}, path="tooling")
|
||||
create_client = Mock()
|
||||
monkeypatch.setattr(tool_manager, "_StdioClientWrapper", create_client)
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
assert manager._get_stdio_client(config, config.cache_key()) is create_client.return_value
|
||||
monkeypatch.delenv(UNSAFE_CODE_EXECUTION_ENV)
|
||||
with pytest.raises(SecurityError):
|
||||
manager._get_stdio_client(config, config.cache_key())
|
||||
create_client.assert_called_once_with(config)
|
||||
|
||||
|
||||
def _workflow(kind, code):
|
||||
graph = {
|
||||
"id": "execution_policy_test", "start": ["Source"],
|
||||
"nodes": [
|
||||
{"id": "Source", "type": "literal", "config": {"content": code, "role": "assistant"}},
|
||||
{"id": "Runner", "type": "python" if kind == "python" else "passthrough", "config": {}},
|
||||
],
|
||||
"edges": [{"from": "Source", "to": "Runner", "trigger": True, "carry_data": True}],
|
||||
}
|
||||
if kind == "processor":
|
||||
graph["edges"][0]["process"] = {"type": "function", "config": {"name": "code_save_and_run"}}
|
||||
else:
|
||||
graph["nodes"][1]["config"]["env"] = {UNSAFE_CODE_EXECUTION_ENV: "1"}
|
||||
return {"version": "0.4.0", "vars": {UNSAFE_CODE_EXECUTION_ENV: "1"}, "graph": graph}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workflow_api(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
yaml_dir = tmp_path / "yaml_instance"
|
||||
yaml_dir.mkdir()
|
||||
for module in (sdk, execute_sync, workflows):
|
||||
monkeypatch.setattr(module, "YAML_DIR", yaml_dir)
|
||||
app = FastAPI()
|
||||
add_exception_handlers(app)
|
||||
app.include_router(workflows.router)
|
||||
app.include_router(execute_sync.router)
|
||||
with TestClient(app) as client:
|
||||
yield client
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", ["python", "processor"])
|
||||
@pytest.mark.parametrize("accept", ["application/json", "text/event-stream"])
|
||||
def test_uploaded_workflow_cannot_enable_execution(workflow_api, tmp_path, kind, accept):
|
||||
code = _code_block("main.py") if kind == "processor" else "print('ok')"
|
||||
response = workflow_api.post(
|
||||
"/api/workflows/upload/content",
|
||||
json={"filename": "policy.yaml", "content": yaml.safe_dump(_workflow(kind, code))},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
response = workflow_api.post(
|
||||
"/api/workflow/run", headers={"Accept": accept},
|
||||
json={"yaml_file": "policy.yaml", "task_prompt": "test", "session_name": "session_policy"},
|
||||
)
|
||||
if accept == "application/json":
|
||||
assert response.status_code == 403
|
||||
else:
|
||||
assert response.status_code == 200
|
||||
assert "event: error" in response.text
|
||||
assert "event: completed" not in response.text
|
||||
assert UNSAFE_CODE_EXECUTION_ENV in response.text
|
||||
assert not list((tmp_path / "WareHouse").rglob("*.py"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("accept", ["application/json", "text/event-stream"])
|
||||
def test_uploaded_processor_cannot_plant_module_even_when_enabled(
|
||||
workflow_api, tmp_path, monkeypatch, fake_launch, accept
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
code = _code_block("../../../functions/function_calling/planted.py", "VALUE = 1")
|
||||
response = workflow_api.post(
|
||||
"/api/workflows/upload/content",
|
||||
json={"filename": "traversal.yaml", "content": yaml.safe_dump(_workflow("processor", code))},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
response = workflow_api.post(
|
||||
"/api/workflow/run", headers={"Accept": accept},
|
||||
json={"yaml_file": "traversal.yaml", "task_prompt": "test", "session_name": "session_policy"},
|
||||
)
|
||||
if accept == "application/json":
|
||||
assert response.status_code == 403
|
||||
else:
|
||||
assert "event: error" in response.text
|
||||
assert "event: completed" not in response.text
|
||||
assert "Invalid code filename" in response.text
|
||||
assert not (tmp_path / "functions").exists()
|
||||
fake_launch.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", ["python", "processor"])
|
||||
def test_uploaded_workflow_runs_after_server_opt_in(
|
||||
workflow_api, monkeypatch, fake_launch, kind
|
||||
):
|
||||
monkeypatch.setenv(UNSAFE_CODE_EXECUTION_ENV, "1")
|
||||
code = _code_block("main.py") if kind == "processor" else "print('ok')"
|
||||
workflow_api.post(
|
||||
"/api/workflows/upload/content",
|
||||
json={"filename": "enabled.yaml", "content": yaml.safe_dump(_workflow(kind, code))},
|
||||
)
|
||||
# subprocess.run uses Popen as a context manager; keep this test deterministic.
|
||||
if kind == "python":
|
||||
monkeypatch.setattr(subprocess, "run", Mock(return_value=subprocess.CompletedProcess([], 0, b"ok\n", b"")))
|
||||
response = workflow_api.post(
|
||||
"/api/workflow/run",
|
||||
json={"yaml_file": "enabled.yaml", "task_prompt": "test", "session_name": "session_enabled"},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.json()["status"] == "completed"
|
||||
assert "ok" in response.json()["final_message"] or "executed" in response.json()["final_message"]
|
||||
272
tests/test_session_workspace_security.py
Normal file
272
tests/test_session_workspace_security.py
Normal file
@ -0,0 +1,272 @@
|
||||
"""Regression coverage for session traversal through uploads and workflow runs."""
|
||||
|
||||
import asyncio
|
||||
import io
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock
|
||||
from urllib.parse import urlencode
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI, UploadFile
|
||||
from fastapi.testclient import TestClient
|
||||
from starlette.websockets import WebSocketDisconnect
|
||||
|
||||
from runtime import sdk
|
||||
from entity.graph_config import GraphConfig
|
||||
from server import state
|
||||
from server.routes import execute_sync, uploads, websocket
|
||||
from server.services.attachment_service import AttachmentService
|
||||
from server.services.websocket_manager import WebSocketManager
|
||||
from utils.exceptions import ValidationError
|
||||
from workflow.graph_context import GraphContext
|
||||
from workflow.runtime.runtime_builder import RuntimeBuilder
|
||||
|
||||
|
||||
TRAVERSAL = "session_../../../functions/function_calling/pocplant"
|
||||
INVALID_NAMES = [
|
||||
TRAVERSAL,
|
||||
"../outside",
|
||||
"session_a/../../outside",
|
||||
"/tmp/session_absolute",
|
||||
r"session_..\..\outside",
|
||||
r"C:\outside",
|
||||
"C:outside",
|
||||
r"\\server\share",
|
||||
".",
|
||||
"..",
|
||||
"session_\x00invalid",
|
||||
"session_\ninvalid",
|
||||
]
|
||||
DEMO_YAML = Path(__file__).resolve().parents[1] / "yaml_instance/demo_loop_counter.yaml"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def graph_config(tmp_path):
|
||||
return GraphConfig.from_definition(
|
||||
sdk.load_config(DEMO_YAML).graph,
|
||||
name="session_safe",
|
||||
output_root=tmp_path / "WareHouse",
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def api(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
manager = WebSocketManager()
|
||||
monkeypatch.setattr(manager, "_start_gc", lambda: None)
|
||||
monkeypatch.setattr(state, "websocket_manager", manager)
|
||||
app = FastAPI()
|
||||
app.include_router(execute_sync.router)
|
||||
app.include_router(uploads.router)
|
||||
app.include_router(websocket.router)
|
||||
with TestClient(app) as client:
|
||||
yield client, manager
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", INVALID_NAMES)
|
||||
def test_graph_rejects_invalid_names_before_creating_directories(graph_config, name):
|
||||
graph_config.name = name
|
||||
with pytest.raises(ValidationError):
|
||||
GraphContext(graph_config)
|
||||
assert not graph_config.output_root.exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["session_ok-123", "demo.v1", "My project", "\u6d4b\u8bd5"])
|
||||
def test_graph_preserves_valid_names_and_timestamp_behavior(graph_config, name):
|
||||
graph_config.name = name
|
||||
graph = GraphContext(graph_config)
|
||||
assert graph.directory.parent == graph_config.output_root.resolve()
|
||||
assert graph.directory.is_dir()
|
||||
if "session_" in name:
|
||||
assert graph.directory.name == name
|
||||
else:
|
||||
assert graph.directory.name.startswith(name + "_")
|
||||
assert graph.directory.name[len(name) + 1:].isdigit()
|
||||
|
||||
|
||||
def test_fixed_output_directory_is_still_confined(graph_config):
|
||||
graph_config.name = "task_001"
|
||||
graph_config.metadata["fixed_output_dir"] = True
|
||||
assert GraphContext(graph_config).directory == graph_config.output_root / "task_001"
|
||||
graph_config.name = "../outside"
|
||||
with pytest.raises(ValidationError):
|
||||
GraphContext(graph_config)
|
||||
|
||||
|
||||
def test_graph_rejects_existing_session_symlink(graph_config, tmp_path):
|
||||
outside = tmp_path / "functions"
|
||||
outside.mkdir()
|
||||
graph_config.output_root.mkdir()
|
||||
(graph_config.output_root / graph_config.name).symlink_to(outside, target_is_directory=True)
|
||||
with pytest.raises(ValidationError, match="escapes"):
|
||||
GraphContext(graph_config)
|
||||
assert list(outside.iterdir()) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("builder", [sdk._build_task_input, execute_sync._build_task_input])
|
||||
@pytest.mark.parametrize("linked_part", ["code_workspace", "code_workspace/attachments"])
|
||||
def test_attachment_copy_rejects_escaping_symlinks(graph_config, tmp_path, builder, linked_part):
|
||||
graph = GraphContext(graph_config)
|
||||
outside = tmp_path / "functions"
|
||||
outside.mkdir()
|
||||
link = graph.directory / linked_part
|
||||
link.parent.mkdir(parents=True, exist_ok=True)
|
||||
link.symlink_to(outside, target_is_directory=True)
|
||||
source = tmp_path / "notes.txt"
|
||||
source.write_text("sample")
|
||||
with pytest.raises(ValidationError, match="escapes"):
|
||||
builder(graph, "test", [source])
|
||||
assert list(outside.iterdir()) == []
|
||||
|
||||
|
||||
def test_runtime_rejects_workspace_symlink_without_attachments(graph_config, tmp_path):
|
||||
graph = GraphContext(graph_config)
|
||||
outside = tmp_path / "functions"
|
||||
outside.mkdir()
|
||||
(graph.directory / "code_workspace").symlink_to(outside, target_is_directory=True)
|
||||
with pytest.raises(ValidationError, match="escapes"):
|
||||
RuntimeBuilder(graph).build(logger=Mock())
|
||||
assert list(outside.iterdir()) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("accept", ["application/json", "text/event-stream"])
|
||||
@pytest.mark.parametrize("name", INVALID_NAMES)
|
||||
def test_http_rejects_bad_session_before_starting_worker(api, monkeypatch, accept, name):
|
||||
client, _ = api
|
||||
worker = Mock(side_effect=AssertionError("Workflow must not start"))
|
||||
monkeypatch.setattr(execute_sync, "run_in_threadpool", worker)
|
||||
monkeypatch.setattr(execute_sync.threading, "Thread", worker)
|
||||
response = client.post(
|
||||
"/api/workflow/run",
|
||||
headers={"Accept": accept},
|
||||
json={"yaml_file": str(DEMO_YAML), "task_prompt": "test", "session_name": name},
|
||||
)
|
||||
assert response.status_code == 400
|
||||
assert "session_name" in response.json()["detail"]
|
||||
worker.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("streaming", [False, True])
|
||||
def test_direct_runner_rejects_traversal_before_loading_config(tmp_path, monkeypatch, streaming):
|
||||
module = execute_sync if streaming else sdk
|
||||
loader = Mock(side_effect=AssertionError("Config must not load"))
|
||||
monkeypatch.setattr(module, "load_config", loader)
|
||||
monkeypatch.setattr(module, "OUTPUT_ROOT", tmp_path / "WareHouse")
|
||||
kwargs = dict(
|
||||
yaml_file=DEMO_YAML,
|
||||
task_prompt="test",
|
||||
attachments=[],
|
||||
session_name=TRAVERSAL,
|
||||
)
|
||||
with pytest.raises(ValidationError):
|
||||
if streaming:
|
||||
execute_sync._run_workflow_with_logger(
|
||||
**kwargs, variables=None, log_level=None, log_callback=lambda *_: None
|
||||
)
|
||||
else:
|
||||
sdk.run_workflow(**kwargs)
|
||||
loader.assert_not_called()
|
||||
assert not (tmp_path / "WareHouse").exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", INVALID_NAMES)
|
||||
def test_websocket_rejects_bad_session_without_registering_connection(api, name):
|
||||
client, manager = api
|
||||
with pytest.raises(WebSocketDisconnect) as exc:
|
||||
with client.websocket_connect("/ws?" + urlencode({"session_id": name})):
|
||||
pytest.fail("Invalid session must not be accepted")
|
||||
assert exc.value.code == 1008
|
||||
assert not manager.active_connections
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", [TRAVERSAL, r"session_..\..\outside", ".."])
|
||||
def test_upload_service_rejects_bad_session_before_reading_file(tmp_path, name):
|
||||
service = AttachmentService(root=tmp_path / "WareHouse")
|
||||
upload = UploadFile(filename="sample.py", file=io.BytesIO(b"VALUE = 1\n"))
|
||||
with pytest.raises(ValidationError):
|
||||
asyncio.run(service.save_upload_file(name, upload))
|
||||
assert upload.file.tell() == 0
|
||||
assert list(service.attachments_root.iterdir()) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("session_name", [None, " "])
|
||||
def test_workflow_still_generates_default_session_name(api, tmp_path, session_name):
|
||||
client, _ = api
|
||||
payload = {"yaml_file": str(DEMO_YAML), "task_prompt": "test"}
|
||||
if session_name is not None:
|
||||
payload["session_name"] = session_name
|
||||
response = client.post("/api/workflow/run", json=payload)
|
||||
assert response.status_code == 200
|
||||
output_dir = Path(response.json()["output_dir"])
|
||||
assert output_dir.parent == tmp_path / "WareHouse"
|
||||
assert output_dir.name.startswith("sdk_demo_loop_counter_")
|
||||
|
||||
|
||||
def test_generated_websocket_session_can_reconnect(api):
|
||||
client, manager = api
|
||||
with client.websocket_connect("/ws") as ws:
|
||||
session_id = ws.receive_json()["data"]["session_id"]
|
||||
assert session_id in manager.active_connections
|
||||
session = manager.session_store.create_session(
|
||||
session_id=session_id, yaml_file=str(DEMO_YAML), task_prompt="test"
|
||||
)
|
||||
session.append_message({"type": "test", "data": "replayed"})
|
||||
with client.websocket_connect("/ws?" + urlencode({"session_id": session_id})) as ws:
|
||||
assert ws.receive_json()["data"]["session_id"] == session_id
|
||||
assert ws.receive_json() == {"type": "test", "data": "replayed"}
|
||||
assert ws.receive_json()["type"] == "session_resumed"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("accept", ["application/json", "text/event-stream"])
|
||||
def test_upload_then_workflow_round_trip_and_traversal_block(api, tmp_path, accept):
|
||||
client, _ = api
|
||||
with client.websocket_connect("/ws?session_id=upload_test") as ws:
|
||||
assert ws.receive_json()["data"]["session_id"] == "upload_test"
|
||||
response = client.post(
|
||||
"/api/uploads/upload_test",
|
||||
files={"file": ("sample.py", b"VALUE = 1\n", "text/x-python")},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert client.get("/api/uploads/upload_test").status_code == 200
|
||||
sources = list((tmp_path / "WareHouse/session_upload_test").rglob("sample.py"))
|
||||
assert len(sources) == 1
|
||||
payload = {
|
||||
"yaml_file": str(DEMO_YAML),
|
||||
"task_prompt": "test",
|
||||
"attachments": [str(sources[0])],
|
||||
"attachment_session_id": "upload_test",
|
||||
"session_name": TRAVERSAL,
|
||||
}
|
||||
response = client.post("/api/workflow/run", headers={"Accept": accept}, json=payload)
|
||||
assert response.status_code == 400
|
||||
assert not (tmp_path / "functions").exists()
|
||||
|
||||
payload["session_name"] = "session_safe_run"
|
||||
response = client.post("/api/workflow/run", headers={"Accept": accept}, json=payload)
|
||||
assert response.status_code == 200
|
||||
if accept == "text/event-stream":
|
||||
assert "event: completed" in response.text
|
||||
assert "event: error" not in response.text
|
||||
else:
|
||||
assert response.json()["status"] == "completed"
|
||||
stored = list((tmp_path / "WareHouse/session_safe_run").rglob("sample.py"))
|
||||
assert len(stored) == 1
|
||||
assert stored[0].read_bytes() == b"VALUE = 1\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("linked_part", ["", "code_workspace", "code_workspace/attachments"])
|
||||
def test_upload_endpoints_reject_escaping_symlinks(api, tmp_path, linked_part):
|
||||
client, _ = api
|
||||
outside = tmp_path / "functions"
|
||||
outside.mkdir()
|
||||
link = tmp_path / "WareHouse/session_upload_test" / linked_part
|
||||
link.parent.mkdir(parents=True, exist_ok=True)
|
||||
link.symlink_to(outside, target_is_directory=True)
|
||||
with client.websocket_connect("/ws?session_id=upload_test") as ws:
|
||||
ws.receive_json()
|
||||
assert client.get("/api/uploads/upload_test").status_code == 400
|
||||
response = client.post(
|
||||
"/api/uploads/upload_test", files={"file": ("sample.py", b"VALUE = 1\n")}
|
||||
)
|
||||
assert response.status_code == 400
|
||||
assert list(outside.iterdir()) == []
|
||||
@ -48,8 +48,11 @@ from server.services.websocket_manager import WebSocketManager # noqa: E402
|
||||
|
||||
def _make_manager() -> WebSocketManager:
|
||||
"""Create a WebSocketManager with minimal mocks."""
|
||||
session_store = MagicMock()
|
||||
session_store.has_session.return_value = False
|
||||
session_store.get_session.return_value = None
|
||||
return WebSocketManager(
|
||||
session_store=MagicMock(),
|
||||
session_store=session_store,
|
||||
session_controller=MagicMock(),
|
||||
attachment_service=MagicMock(),
|
||||
workflow_run_service=MagicMock(),
|
||||
|
||||
20
utils/code_execution.py
Normal file
20
utils/code_execution.py
Normal file
@ -0,0 +1,20 @@
|
||||
"""Server-controlled opt-in for running workflow-supplied code."""
|
||||
|
||||
import os
|
||||
|
||||
from utils.exceptions import SecurityError
|
||||
|
||||
|
||||
UNSAFE_CODE_EXECUTION_ENV = "MAC_ALLOW_UNSAFE_CODE_EXECUTION"
|
||||
|
||||
|
||||
def require_unsafe_code_execution(operation: str) -> None:
|
||||
# Never consult workflow variables, tool arguments, or child-process env overrides.
|
||||
enabled = os.environ.get(UNSAFE_CODE_EXECUTION_ENV, "").strip().lower()
|
||||
if enabled not in {"1", "true", "yes", "on"}:
|
||||
raise SecurityError(
|
||||
f"{operation} is disabled. Running workflow-supplied code requires "
|
||||
f"{UNSAFE_CODE_EXECUTION_ENV}=1 in the server environment. "
|
||||
"Enable this only for trusted workflows; execution is not sandboxed.",
|
||||
details={"operation": operation},
|
||||
)
|
||||
39
utils/workspace_paths.py
Normal file
39
utils/workspace_paths.py
Normal file
@ -0,0 +1,39 @@
|
||||
"""Validate names and confine generated files to their workspace."""
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from utils.exceptions import ValidationError
|
||||
|
||||
|
||||
def validate_path_component(name: str, *, field: str = "name") -> str:
|
||||
"""Accept a single directory name on both POSIX and Windows."""
|
||||
if (
|
||||
name.strip() in {"", ".", ".."}
|
||||
or re.search(r"[/\\:\x00-\x1f\x7f]", name)
|
||||
):
|
||||
raise ValidationError(
|
||||
f"Invalid {field}: expected a single directory name without path separators, "
|
||||
"drive prefixes, or control characters",
|
||||
field=field,
|
||||
)
|
||||
return name
|
||||
|
||||
|
||||
def resolve_workspace_path(root: Path | str, *parts: str) -> Path:
|
||||
"""Resolve a child path, rejecting traversal and existing escaping symlinks."""
|
||||
for part in parts:
|
||||
validate_path_component(part)
|
||||
try:
|
||||
resolved_root = Path(root).resolve()
|
||||
candidate = resolved_root.joinpath(*parts).resolve()
|
||||
except (OSError, RuntimeError) as exc:
|
||||
raise ValidationError("Unable to resolve workspace path", field="workspace") from exc
|
||||
if candidate == resolved_root or not candidate.is_relative_to(resolved_root):
|
||||
raise ValidationError("Workspace path escapes its root directory", field="workspace")
|
||||
return candidate
|
||||
|
||||
|
||||
def session_directory_name(session_id: str) -> str:
|
||||
"""Map a raw transport session ID to one distinct directory name."""
|
||||
return f"session_{validate_path_component(session_id, field='session_id')}"
|
||||
@ -12,7 +12,7 @@ from entity.messages import Message, MessageRole
|
||||
from runtime.node.executor.base import ExecutionContext
|
||||
from runtime.node.executor.factory import NodeExecutorFactory
|
||||
from utils.logger import WorkflowLogger
|
||||
from utils.exceptions import ValidationError, WorkflowExecutionError, WorkflowCancelledError
|
||||
from utils.exceptions import ValidationError, WorkflowExecutionError, WorkflowCancelledError, SecurityError
|
||||
from utils.structured_logger import get_server_logger
|
||||
from utils.human_prompt import (
|
||||
CliPromptChannel,
|
||||
@ -420,6 +420,8 @@ class GraphExecutor:
|
||||
from_node,
|
||||
self.log_manager,
|
||||
)
|
||||
except SecurityError:
|
||||
raise
|
||||
except Exception as exc: # pragma: no cover - defensive logging
|
||||
error_msg = (
|
||||
f"Edge manager failed for {from_node.id} -> {edge_link.target.id}: {exc}"
|
||||
|
||||
@ -10,6 +10,7 @@ import yaml
|
||||
|
||||
from entity.configs import Node
|
||||
from entity.graph_config import GraphConfig
|
||||
from utils.workspace_paths import resolve_workspace_path, validate_path_component
|
||||
|
||||
|
||||
class GraphContext:
|
||||
@ -60,12 +61,14 @@ class GraphContext:
|
||||
self.cycle_execution_order: List[Dict[str, Any]] = []
|
||||
|
||||
# Output directory
|
||||
validate_path_component(config.name, field="session_name")
|
||||
timestamp = datetime.now().strftime("%Y%m%d%H%M%S")
|
||||
fixed_output_dir = bool(config.metadata.get("fixed_output_dir"))
|
||||
if fixed_output_dir or "session_" in config.name:
|
||||
self.directory = config.output_root / config.name
|
||||
directory_name = config.name
|
||||
else:
|
||||
self.directory = config.output_root / f"{config.name}_{timestamp}"
|
||||
directory_name = f"{config.name}_{timestamp}"
|
||||
self.directory = resolve_workspace_path(config.output_root, directory_name)
|
||||
self.directory.mkdir(parents=True, exist_ok=True)
|
||||
# Voting mode flag
|
||||
self.is_majority_voting: bool = config.is_majority_voting
|
||||
|
||||
@ -9,6 +9,7 @@ from utils.function_manager import EDGE_FUNCTION_DIR, EDGE_PROCESSOR_FUNCTION_DI
|
||||
from utils.log_manager import LogManager
|
||||
from utils.logger import WorkflowLogger
|
||||
from utils.token_tracker import TokenTracker
|
||||
from utils.workspace_paths import resolve_workspace_path
|
||||
from workflow.graph_context import GraphContext
|
||||
|
||||
from .runtime_context import RuntimeContext
|
||||
@ -28,9 +29,9 @@ class RuntimeBuilder:
|
||||
log_manager = LogManager(logger)
|
||||
token_tracker = TokenTracker(workflow_id=self.graph.name)
|
||||
|
||||
code_workspace = (self.graph.directory / "code_workspace").resolve()
|
||||
code_workspace = resolve_workspace_path(self.graph.directory, "code_workspace")
|
||||
attachments_dir = resolve_workspace_path(self.graph.directory, "code_workspace", "attachments")
|
||||
code_workspace.mkdir(parents=True, exist_ok=True)
|
||||
attachments_dir = code_workspace / "attachments"
|
||||
attachments_dir.mkdir(parents=True, exist_ok=True)
|
||||
attachment_store = AttachmentStore(attachments_dir)
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user