fix: invalidate converted uploads after same-size source edits (#6165)

* fix: invalidate converted uploads after same-size source edits

* fix: log converted upload ownership rejection reasons

* docs: clarify source timestamp limits and thread ownership

README.md

@@ -1857,6 +1857,16 @@ Outline titles are limited to 200 characters and fallback previews to 2,000
 characters per file, with truncation markers. Full uploaded files remain available
 for targeted reads.
 
+Converted upload outlines and previews require a matching source version, including
+modification timestamps. A detected source-version change invalidates its previous
+conversion, including equal-length edits. Older ownership records without source timestamps
+are also rejected; re-upload the source with `uploads.auto_convert_documents: true`
+to restore conversion-backed outlines. Files remain available, and unvalidated
+Markdown conversions appear as standalone files in the agent's historical listing.
+On Windows, `st_ctime_ns` may represent creation time; a same-size rewrite that
+restores the original `mtime` can evade validation. Timestamp checks are
+conservative metadata validation, not a guarantee of content equality.
+
 Image bytes loaded for a vision-model call are transient: DeerFlow removes the hidden base64 message after the model consumes it so later checkpoints do not keep duplicating that payload.
 
 After each run, DeerFlow records a workspace change summary for the run-owned `workspace` and `outputs` directories. The Web UI shows a compact "files changed" badge on the assistant turn; opening it reveals created, modified, and deleted files with text diffs when safe to display. Uploads are excluded because they are user inputs, not agent-generated changes, and stdio MCP temporary/debug files under the DeerFlow-owned `.mcp/` namespace are excluded because they are process-internal state (like `.git/` and `node_modules/`, any directory named `.mcp` is excluded at any depth). Large, binary, or sensitive-looking files are shown as metadata only.

README_zh.md

@@ -772,6 +772,8 @@ lead agent 只会在委派具有明确净收益时动态拉起 sub-agents，例
 
 ### Sandbox 与文件系统
 
+上传文档的转换大纲和预览会校验原文件版本，包括修改时间戳。检测到原文件版本变化后，即使大小不变，也不会再使用旧转换结果。缺少原文件时间戳的旧归属记录同样会被拒绝；可在启用 `uploads.auto_convert_documents: true` 后重新上传原文件以恢复转换大纲。文件都会保留，未通过校验的转换 Markdown 会在 Agent 的历史文件列表中作为独立文件显示。Windows 上的 `st_ctime_ns` 可能表示创建时间，因此等长度覆盖后若恢复原 `mtime`，校验可能无法识别变化。时间戳校验是保守的元数据校验，不能保证文件内容完全一致。
+
 DeerFlow 不只是“会说它能做”，它是真的有一台自己的“电脑”。
 
 每个任务都运行在隔离的 Docker 容器里，里面有完整的文件系统，包括 skills、workspace、uploads、outputs。agent 可以读写和编辑文件，可以执行 bash 命令和代码，也可以查看图片。整个过程都在 sandbox 内完成，可审计、会隔离，不会在不同 session 之间互相污染。

backend/AGENTS.md

@@ -338,7 +338,7 @@ Outlines use ATX syntax (1–6 hashes, space/tab separator, ≤3 leading spaces)
 - Rejects directories before copying to keep uploads all-or-nothing
 - One conversion worker per request when called from an active event loop
 - Files stored in thread-isolated directories under the resolving user's bucket (`users/{user_id}/threads/{thread_id}/user-data/uploads`). For IM channels the owner is threaded explicitly via the `user_id=` kwarg (see IM Channels → Owner-scoped file storage); HTTP/embedded callers resolve it from `get_effective_user_id()`
-- Thread ownership: `upload-companions/`.
+- Per-thread `upload-companions/`: source mtime/ctime.
 - Duplicate filenames within one request get `_N` suffixes to prevent overwrites.
 - Gateway HTTP uploads stage `.upload-*.part` files, hidden from upload listings, agent context, and sandbox listings/searches. After size validation, publication is atomic; staged-name cleanup logs errors and leaves leftovers for startup sweep.
 - Gateway HTTP upload/list/delete handlers offload filesystem work through `deerflow.utils.file_io.run_file_io`, a dedicated ContextVar-preserving file IO executor. Non-mounted sandbox uploads acquire sandboxes with `SandboxProvider.acquire_async()` and offload `read_bytes()` plus `sandbox.update_file()` together.

backend/docs/FILE_UPLOAD.md

@@ -118,7 +118,9 @@ DELETE /api/threads/{thread_id}/uploads/{filename}
 
 转换后的 Markdown 文件会保存在同一目录下，通常使用原文件的主干名加 `.md`；若该名称已被占用，则追加 `_N` 后缀。上传响应中的 `markdown_file` 给出实际名称。
 
-新转换会在沙箱挂载目录之外保存原文件与转换文件的归属记录。Agent 的历史文件列表只隐藏归属已验证的转换文件；文档大纲也只读取这份记录指定的 Markdown。旧版本生成的转换文件没有归属记录，升级后会作为独立 Markdown 显示，原文件也不会再从同名文件推断大纲。因为旧文件无法与用户自行上传的同名 Markdown 安全地区分，系统不会自动补建记录；需要大纲时，可在启用自动转换后重新上传原文件。
+新转换会在沙箱挂载目录之外保存原文件与转换文件的归属记录。Agent 的历史文件列表只隐藏归属已验证的转换文件；文档大纲也只读取这份记录指定的 Markdown。原文件校验包含修改时间戳，即使修改前后大小相同，也会拒绝使用旧转换结果。旧版本没有归属记录或记录缺少原文件时间戳的转换文件，升级后会作为独立 Markdown 显示，原文件也不会再从这些文件读取大纲。系统不会自动补建记录或填入当前时间戳，以免将过期转换结果认定为有效；需要大纲时，可在启用 `uploads.auto_convert_documents: true` 后重新上传原文件。原文件和转换文件都会保留。
+
+时间戳校验是保守的元数据校验，不能保证文件内容完全一致。Windows 上的 `st_ctime_ns` 可能表示创建时间，因此等长度覆盖后若恢复原 `mtime`，校验可能无法识别变化。
 
 默认情况下，自动转换是关闭的，以避免在网关主机上对不受信任的 Office/PDF 上传执行解析。只有在受信任部署中明确接受此风险时，才应将 `uploads.auto_convert_documents` 设置为 `true`。
 

backend/packages/harness/deerflow/uploads/companions.py

@@ -8,10 +8,13 @@ from __future__ import annotations
 
 import hashlib
 import json
+import logging
 import os
 import tempfile
 from pathlib import Path
 
+logger = logging.getLogger(__name__)
+
 
 def _records_dir(uploads_dir: Path) -> Path:
     # Production layout is <thread>/user-data/uploads. Tests and embedded
@@ -41,7 +44,7 @@ def register_companion(original: Path, markdown: Path) -> None:
                 {
                     "original": original.name,
                     "markdown": markdown.name,
-                    "source_identity": [source_stat.st_dev, source_stat.st_ino, source_stat.st_size],
+                    "source_identity": [source_stat.st_dev, source_stat.st_ino, source_stat.st_size, source_stat.st_mtime_ns, source_stat.st_ctime_ns],
                     "markdown_identity": [markdown_stat.st_dev, markdown_stat.st_ino, markdown_stat.st_size, markdown_stat.st_ctime_ns],
                 },
                 output,
@@ -87,13 +90,21 @@ def resolve_companion(original: Path) -> Path | None:
         if data["original"] != original.name or not isinstance(name, str) or Path(name).name != name or not name.endswith(".md"):
             return None
         source_stat = original.stat(follow_symlinks=False)
-        if data.get("source_identity") != [source_stat.st_dev, source_stat.st_ino, source_stat.st_size]:
+        # Legacy three-field records cannot establish whether an equal-size
+        # source edit happened. Reject them rather than backfilling timestamps.
+        if data.get("source_identity") != [source_stat.st_dev, source_stat.st_ino, source_stat.st_size, source_stat.st_mtime_ns, source_stat.st_ctime_ns]:
+            source_identity = data.get("source_identity")
+            if isinstance(source_identity, list) and len(source_identity) == 3:
+                logger.debug("Rejected upload companion for %r: legacy source identity lacks version timestamps", original.name)
+            else:
+                logger.debug("Rejected upload companion for %r: source identity or version timestamps do not match", original.name)
             return None
         markdown = original.parent / name
         if not markdown.is_file() or markdown.is_symlink():
             return None
         markdown_stat = markdown.stat(follow_symlinks=False)
         if data.get("markdown_identity") != [markdown_stat.st_dev, markdown_stat.st_ino, markdown_stat.st_size, markdown_stat.st_ctime_ns]:
+            logger.debug("Rejected upload companion for %r: Markdown identity or version timestamp does not match", original.name)
             return None
         return markdown
     except (OSError, ValueError, KeyError, TypeError):

backend/tests/test_list_uploaded_files_tool.py

@@ -286,6 +286,24 @@ class TestListUploadedFiles:
         assert "outline" not in files["report.pdf"]
         assert files["report.md"]["outline"] == [{"title": "User replacement with different size", "line": 1}]
 
+    def test_same_size_source_edit_exposes_companion_and_drops_stale_outline(self, tmp_path):
+        uploads_dir = _uploads_dir(tmp_path)
+        original = uploads_dir / "report.pdf"
+        companion = uploads_dir / "report.md"
+        original.write_bytes(b"%PDF ORIGINAL A")
+        companion.write_text("# Original A\n", encoding="utf-8")
+        register_companion(original, companion)
+        before = original.stat()
+        original.write_bytes(b"%PDF REPLACED B")
+        os.utime(original, ns=(before.st_atime_ns, before.st_mtime_ns + 2_000_000_000))
+
+        result = _list_uploaded_files_impl(include_outline=True, runtime=_runtime(), _paths=_paths(tmp_path))
+
+        files = {f["filename"]: f for f in result["files"]}
+        assert set(files) == {"report.pdf", "report.md"}
+        assert "outline" not in files["report.pdf"]
+        assert files["report.md"]["outline"] == [{"title": "Original A", "line": 1}]
+
     def test_cross_turn_state_clear_does_not_exclude_historical_file(self, tmp_path):
         """Two-turn regression: file uploaded in turn 1 must appear in turn 2.
 

backend/tests/test_upload_companions.py

@@ -0,0 +1,100 @@
+"""Source-version validation for converted upload ownership records."""
+
+import json
+import os
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+
+from deerflow.sandbox.local.local_sandbox import LocalSandbox, PathMapping
+from deerflow.uploads.companions import companion_names, register_companion, resolve_companion
+from deerflow.utils.file_outline import extract_outline_for_file
+
+
+@pytest.fixture
+def converted_upload(tmp_path):
+    uploads = tmp_path / "thread" / "user-data" / "uploads"
+    uploads.mkdir(parents=True)
+    original = uploads / "report.pdf"
+    markdown = uploads / "report.md"
+    original.write_bytes(b"%PDF ORIGINAL A")
+    markdown.write_text("# Original A\n", encoding="utf-8")
+    register_companion(original, markdown)
+    return original, markdown
+
+
+def test_unchanged_source_keeps_companion_after_read(converted_upload):
+    original, markdown = converted_upload
+    assert original.read_bytes() == b"%PDF ORIGINAL A"
+    assert resolve_companion(original) == markdown
+    assert companion_names(original.parent) == {markdown.name}
+    assert extract_outline_for_file(original)[0] == [{"title": "Original A", "line": 1}]
+
+
+def test_same_size_sandbox_rewrite_rejects_stale_outline(converted_upload):
+    original, markdown = converted_upload
+    before = original.stat()
+    sandbox = LocalSandbox("repro", [PathMapping("/mnt/user-data", str(original.parent.parent))])
+    sandbox.write_file("/mnt/user-data/uploads/report.pdf", "%PDF REPLACED B")
+    # Force a distinct modification time even on coarse-resolution filesystems.
+    os.utime(original, ns=(before.st_atime_ns, before.st_mtime_ns + 2_000_000_000))
+    after = original.stat()
+    assert (before.st_dev, before.st_ino, before.st_size) == (after.st_dev, after.st_ino, after.st_size)
+
+    assert resolve_companion(original) is None
+    assert companion_names(original.parent) == set()
+    assert extract_outline_for_file(original) == ([], [])
+    assert markdown.read_text(encoding="utf-8") == "# Original A\n"
+
+
+@pytest.mark.parametrize("timestamp", ["st_mtime_ns", "st_ctime_ns"])
+def test_each_source_timestamp_invalidates_companion(converted_upload, monkeypatch, timestamp, caplog):
+    original, _ = converted_upload
+    real_stat = Path.stat
+    before = original.stat()
+
+    def changed_stat(path, *args, **kwargs):
+        result = real_stat(path, *args, **kwargs)
+        if path == original:
+            fields = {name: getattr(before, name) for name in ("st_mode", "st_dev", "st_ino", "st_size", "st_mtime_ns", "st_ctime_ns")}
+            fields[timestamp] += 2_000_000_000
+            return SimpleNamespace(**fields)
+        return result
+
+    # Model mtime-only updates (e.g. Windows creation-time ctime) and ctime-only
+    # updates (e.g. POSIX writes followed by restoring the old mtime).
+    monkeypatch.setattr(Path, "stat", changed_stat)
+    with caplog.at_level("DEBUG", logger="deerflow.uploads.companions"):
+        assert resolve_companion(original) is None
+    assert "source identity or version timestamps do not match" in caplog.text
+
+
+def test_legacy_source_record_requires_reregistration(converted_upload, caplog):
+    original, markdown = converted_upload
+    record = next((original.parent.parent.parent / "upload-companions").glob("*.json"))
+    data = json.loads(record.read_text(encoding="utf-8"))
+    data["source_identity"] = data["source_identity"][:3]
+    record.write_text(json.dumps(data), encoding="utf-8")
+    legacy = record.read_bytes()
+
+    with caplog.at_level("DEBUG", logger="deerflow.uploads.companions"):
+        assert resolve_companion(original) is None
+    assert "legacy source identity lacks version timestamps" in caplog.text
+    assert extract_outline_for_file(original) == ([], [])
+    assert record.read_bytes() == legacy  # Do not bless stale content on read.
+
+    register_companion(original, markdown)
+    assert resolve_companion(original) == markdown
+
+
+def test_new_conversion_restores_outline_after_source_edit(converted_upload):
+    original, markdown = converted_upload
+    before = original.stat()
+    original.write_bytes(b"%PDF REPLACED B")
+    os.utime(original, ns=(before.st_atime_ns, before.st_mtime_ns + 2_000_000_000))
+    markdown.write_text("# Replaced B\n", encoding="utf-8")
+    register_companion(original, markdown)
+
+    assert resolve_companion(original) == markdown
+    assert extract_outline_for_file(original)[0] == [{"title": "Replaced B", "line": 1}]