sjr666666 6ce385b963
fix(uploads): stop Windows replacement uploads from being skipped by the multi-link guard (#6821)
* fix(uploads): stop Windows replacement uploads from being skipped by the multi-link guard

NTFS never decrements st_nlink when one of a file's names is removed, so
every upload published through os.link(staging, dest) + unlink keeps
reading st_nlink == 2 on Windows. The non-exclusive replacement guards
then reject that destination forever, and DeerFlowClient.upload_files
turns the UnsafeUploadPathError into a log line - a replacement upload
is silently skipped (Closes #6820).

- Add _replacement_destination_is_hardlinked: POSIX-authoritative
  st_nlink > 1, skipped on Windows, used only on the not exclusive
  replacement path. The exclusive (first-claim) path keeps the strict
  count check, as does the POSIX O_NOFOLLOW branch.
- Decide "published alias" in cleanup_stale_upload_staging_files by
  inode identity against siblings (st_dev/st_ino via os.lstat) instead
  of the link count; DirEntry.stat reports zeros for both on Windows,
  matching the existing _staging_entry_stat note.
- Guard the POSIX-only st_nlink assertions in the affected tests and
  mark the three inbound-attachment mode-bit tests POSIX-only, mirroring
  the existing skipif in the same class.

* fix(uploads): restore the exclusive post-open link guard and harden the staging sweep

Address the three review findings on 5353715:

- Restore the strict post-open st_nlink != 1 check for exclusive=True
  on the no-O_NOFOLLOW (Windows fallback) branch, which the previous
  commit dropped there against what the PR description claimed. Only
  the non-exclusive replacement path defers to the Windows-aware guard.
- Never match a degenerate st_ino == 0 in the sibling-inode sweep: on
  FAT/exFAT or some SMB volumes every entry would look like a published
  alias and in-flight uploads would be deleted at any age. Such volumes
  fall back to the min_age guard only.
- Build one {name: stat} map per uploads directory before the entry
  loop so the sweep scans each directory once instead of rescanning it
  for every staging file (O(N) instead of O(N^2) lstat calls).

Tests: 407 passed / 27 skipped across test_uploads_manager.py,
test_uploads_router.py, test_upload_copy_publication.py,
test_upload_windows_staging_aliases.py, test_channel_file_attachments.py
(Windows 11, Python 3.13 venv). The three new regression tests fail on
5353715 and pass with this change.

* style(uploads): apply Ruff formatting

* fix(uploads): report staging directory scan failures

---------

Co-authored-by: sjr666666 <sjr666666@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-14 08:43:26 +08:00

2.0 KiB

Uploads

manager.py::list_files_in_dir returns a best-effort snapshot under concurrent cleanup. Entry stat races (ENOENT, ENOTDIR, ELOOP) are skipped with debug logging. Directory removal or replacement by a non-directory during scanning returns entries already collected and logs the directory path and retained entry count at debug level. Permission and other operational errors propagate.

Keep both entry and directory race coverage in backend/tests/test_uploads_manager.py.

Staging cleanup is best-effort: warn with the directory and exception on scan errors, keep its files and continue elsewhere. A vanished directory stays quiet.

Embedded first-time uploads use publish_upload_copy for suffix retries from the normalized original name, for originals and companions. With snapshot_path, capture the opened source once before retries and retain its original fstat metadata; every attempt reads the held private capture. Replacement copies share this capture helper. Never copy snapshot permission bits onto the saved upload. Close the caller's descriptor after capture and finish conversion workers before removing captures. Never reopen the caller/sandbox upload path for conversion. Names already present in the call's initial regular-file snapshot retain replacement behavior; these replacements are not serialized. Apply source timestamps after new-file publication so cleanup cannot sweep an in-flight copy of an old source. New final names must expose complete copies.

No-replace publication binds pre-link staging and post-link destination checks to the open staging descriptor. Use a no-follow hard link where supported; fallback identity checks narrow but do not eliminate the link-time race. Reject substituted regular files and symlinks, and remove a mismatched publication before reporting failure. Cleanup errors are logged without masking success or the original error. Keep publication/fallback coverage in backend/tests/test_upload_copy_publication.py and capture-through-collision cases in backend/tests/test_client_upload_snapshot.py.