fix(uploads): reject colons in normalize_filename like the skill installer (#8351)

* fix(uploads): reject colons in normalize_filename like the skill installer

normalize_filename() accepted a ':' in the basename, so the three upload
writer paths disagreed for the same name on Windows: the streaming write
(write_upload_file_no_symlink) happily stored the bytes in an invisible
NTFS Alternate Data Stream ("notes.txt:hidden" attaches a stream to
notes.txt; directory listings report the base file with 0 new bytes),
while copy/publish raised WinError 123 outright.

skills.installer.is_unsafe_zip_member already rejects any member path
containing ':' with exactly this rationale (drive letters are caught
earlier as absolute paths; a stray colon addresses an ADS that is
invisible to rglob/os.walk listings). Apply the same rule to upload
filenames so every writer path rejects the name identically.

* docs(uploads): clarify colon rejection and drive paths

---------

Co-authored-by: laoyyu <laoyyu@users.noreply.github.com>
Co-authored-by: Nan Gao <thunderggnn@gmail.com>
This commit is contained in:
laoyyu 2026-11-18 18:55:06 +08:00 committed by GitHub
parent ec3f3d4d22
commit a50663c2f1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 52 additions and 0 deletions

View File

@ -33,6 +33,25 @@ POST /api/threads/{thread_id}/uploads
检查也会拒绝 Windows 去掉末尾点号或空格、忽略大小写后会变成保留名称的别名,例如 `.upload-notes.part.`、`.upload-notes.part `、`.UPLOAD-NOTES.PART` 和 `.Upload-NoTeS.Part .`。这项检查在所有主机上生效,HTTP 上传会在整批文件写入前返回 `400`,提示改名。
规范化后的文件名只要仍含冒号也会被拒绝,避免 Windows 把 `notes.txt:hidden` 当成 `notes.txt` 的 NTFS Alternate Data Stream 而不是新文件。盘符绝对路径会先去掉目录部分,例如 `C:/reports/report.txt` 在 Linux、macOS 和 Windows 上都得到 `report.txt`;盘符相对写法 `C:report.txt` 在 Windows 上去掉盘符后得到 `report.txt`,在 Linux 和 macOS 上则因 basename 仍含冒号而被拒绝。这里拒绝的是会写入宿主或沙箱文件的名称;POSIX 文件系统上的普通文件即使允许冒号,也不应把该名称用于跨平台上传。
其他不安全文件名按文件跳过,不会让单文件上传整批返回 `400`。如果批次里只有不安全文件,HTTP 响应仍为 `200`,但 `success` 为 `false`、`files` 为空,原始名称会列入 `skipped_files`。混合批次只要至少有一个文件成功,`success` 就是 `true`;客户端仍必须检查 `skipped_files` 和 `files`,不能只看 `success`。例如同时上传 `readme.txt` 和 `10:30.txt` 时,响应可以是:
```json
{
"success": true,
"files": [
{
"filename": "readme.txt",
"size": 12,
"path": ".deer-flow/threads/{thread_id}/user-data/uploads/readme.txt"
}
],
"message": "Successfully uploaded 1 file(s); skipped 1 unsafe file(s)",
"skipped_files": ["10:30.txt"]
}
```
嵌入式 `DeerFlowClient.upload_files` 同样在复制前检查整批文件名,保留名称会抛出 `ValueError`。项目资料库上传和从聊天加入资料库时的重命名也遵循这项限制,包括沿用来源文件名和手动指定新名称两种情况;旧资料库中使用保留名称的文件仍可下载,但直接附加到聊天会返回 `400`。请先下载、改名,再上传到聊天。本修复不会迁移或恢复旧聊天目录中已经匹配该临时文件规则的文件。
**响应:**

View File

@ -119,6 +119,15 @@ def normalize_filename(filename: str) -> str:
# but they indicate a Windows-style path that should be stripped or rejected.
if "\\" in safe:
raise ValueError(f"Filename contains backslash: {filename!r}")
# Reject colons left in the basename. A drive-absolute spelling such as
# "C:/reports/report.txt" is stripped on every platform. Windows also strips
# drive-relative "C:report.txt", while POSIX keeps that colon in the basename
# and rejects it here. A colon that survives normalization addresses an NTFS
# Alternate Data Stream on Windows ("notes.txt:hidden" writes an invisible
# stream on notes.txt instead of a new file), matching
# skills.installer.is_unsafe_zip_member.
if ":" in safe:
raise ValueError(f"Filename contains colon: {filename!r}")
if len(safe.encode("utf-8")) > _MAX_FILENAME_BYTES:
raise ValueError(f"Filename too long: {len(safe)} chars")
if is_reserved_upload_filename(safe):

View File

@ -86,6 +86,30 @@ class TestNormalizeFilename:
with pytest.raises(ValueError, match="NUL"):
normalize_filename("report\x00.pdf")
@pytest.mark.parametrize(
"filename",
["notes.txt:hidden", "doc:hidden", "folder/doc.txt:hidden", "a:b:c.txt"],
)
def test_rejects_colon_ads_names(self, filename):
"""A colon in a basename addresses an NTFS Alternate Data Stream on
Windows (invisible to directory listings) instead of a new file,
matching the skill-archive installer's own rule."""
with pytest.raises(ValueError, match="colon"):
normalize_filename(filename)
def test_colon_name_rejected_before_writing_upload(self, tmp_path):
with pytest.raises(ValueError, match="colon"):
write_upload_file_no_symlink(tmp_path, "notes.txt:hidden", b"secret")
assert list(tmp_path.iterdir()) == []
def test_drive_reference_platform_behavior(self):
assert normalize_filename("C:/reports/report.txt") == "report.txt"
if os.name == "nt":
assert normalize_filename("C:report.txt") == "report.txt"
else:
with pytest.raises(ValueError, match="colon"):
normalize_filename("C:report.txt")
@pytest.mark.parametrize("filename", [".upload-notes.part", ".upload-.part", "folder/.upload-notes.part"])
def test_rejects_reserved_staging_names(self, filename):
with pytest.raises(ValueError, match="reserved upload staging"):