* fix(uploads): stop Windows replacement uploads from being skipped by the multi-link guard NTFS never decrements st_nlink when one of a file's names is removed, so every upload published through os.link(staging, dest) + unlink keeps reading st_nlink == 2 on Windows. The non-exclusive replacement guards then reject that destination forever, and DeerFlowClient.upload_files turns the UnsafeUploadPathError into a log line - a replacement upload is silently skipped (Closes #6820). - Add _replacement_destination_is_hardlinked: POSIX-authoritative st_nlink > 1, skipped on Windows, used only on the not exclusive replacement path. The exclusive (first-claim) path keeps the strict count check, as does the POSIX O_NOFOLLOW branch. - Decide "published alias" in cleanup_stale_upload_staging_files by inode identity against siblings (st_dev/st_ino via os.lstat) instead of the link count; DirEntry.stat reports zeros for both on Windows, matching the existing _staging_entry_stat note. - Guard the POSIX-only st_nlink assertions in the affected tests and mark the three inbound-attachment mode-bit tests POSIX-only, mirroring the existing skipif in the same class. * fix(uploads): restore the exclusive post-open link guard and harden the staging sweep Address the three review findings on 5353715: - Restore the strict post-open st_nlink != 1 check for exclusive=True on the no-O_NOFOLLOW (Windows fallback) branch, which the previous commit dropped there against what the PR description claimed. Only the non-exclusive replacement path defers to the Windows-aware guard. - Never match a degenerate st_ino == 0 in the sibling-inode sweep: on FAT/exFAT or some SMB volumes every entry would look like a published alias and in-flight uploads would be deleted at any age. Such volumes fall back to the min_age guard only. - Build one {name: stat} map per uploads directory before the entry loop so the sweep scans each directory once instead of rescanning it for every staging file (O(N) instead of O(N^2) lstat calls). Tests: 407 passed / 27 skipped across test_uploads_manager.py, test_uploads_router.py, test_upload_copy_publication.py, test_upload_windows_staging_aliases.py, test_channel_file_attachments.py (Windows 11, Python 3.13 venv). The three new regression tests fail on 5353715 and pass with this change. * style(uploads): apply Ruff formatting * fix(uploads): report staging directory scan failures --------- Co-authored-by: sjr666666 <sjr666666@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2.0 KiB
Uploads
manager.py::list_files_in_dir returns a best-effort snapshot under concurrent
cleanup. Entry stat races (ENOENT, ENOTDIR, ELOOP) are skipped with debug
logging. Directory removal or replacement by a non-directory during scanning
returns entries already collected and logs the directory path and retained
entry count at debug level. Permission and other operational errors propagate.
Keep both entry and directory race coverage in
backend/tests/test_uploads_manager.py.
Staging cleanup is best-effort: warn with the directory and exception on scan errors, keep its files and continue elsewhere. A vanished directory stays quiet.
Embedded first-time uploads use publish_upload_copy for suffix retries from the
normalized original name, for originals and companions. With snapshot_path,
capture the opened source once before retries and retain its original fstat
metadata; every attempt reads the held private capture. Replacement copies share
this capture helper. Never copy snapshot permission bits onto the saved upload.
Close the caller's descriptor after capture and finish conversion workers before
removing captures. Never reopen the caller/sandbox upload path for conversion.
Names already present in the call's initial regular-file snapshot retain
replacement behavior; these replacements are not serialized. Apply source
timestamps after new-file publication so cleanup cannot sweep an in-flight copy
of an old source. New final names must expose complete copies.
No-replace publication binds pre-link staging and post-link destination checks to
the open staging descriptor. Use a no-follow hard link where supported; fallback
identity checks narrow but do not eliminate the link-time race. Reject substituted
regular files and symlinks, and remove a mismatched publication before reporting
failure. Cleanup errors are logged without masking success or the original error.
Keep publication/fallback coverage in backend/tests/test_upload_copy_publication.py
and capture-through-collision cases in backend/tests/test_client_upload_snapshot.py.