mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-25 14:06:18 +00:00
29d285731b
* fix(uploads): convert the bytes we wrote, not the name they landed under Document conversion re-opened the upload by name after it was already visible in the thread's uploads directory: the Gateway converted the committed file_path, and DeerFlowClient converted the copy it had just placed there. That directory is writable from local and AIO sandboxes, so a process watching it can replace the name with a symlink in the window between the upload landing and the converter opening it. The converter then reads whatever host file the link points at and writes that content back into the thread as the .md companion, which the sandbox can read. Reproduced end to end on both paths with a real xlsx: the companion came back holding the host file's rows. The Gateway now duplicates the descriptor of the staged file before the link-commit, copies those bytes into a private directory outside the uploads tree, and converts there. A descriptor cannot be redirected by replacing a name, so the conversion input is the content this request wrote. The client converts the caller's own source file instead of the copy in uploads; the source is the file the caller handed in, which the sandbox cannot reach. Both already wrote the companion without following a symlink, so only the read side changes. The uploads directory still receives exactly the same files. * docs(changelog): note upload conversion source fix (#5611) * fix(uploads): close the conversion descriptor when staging its copy fails Review follow-up. The private directory for the conversion copy was created before the try that owns the duplicated descriptor, so a failure there — a full or unwritable temporary filesystem — propagated without closing it. The upload's own cleanup only unlinks the committed name and releases the sandbox lease, so the descriptor stayed open for the life of the process and kept the unlinked staged bytes allocated with it; repeated failures accumulated both. Directory creation now happens inside that try, and the finally removes the directory only once it exists. * fix(uploads): keep the conversion descriptor owned across cancellation Review follow-up. run_file_io cannot interrupt its worker, so cancelling the await around os.dup only abandoned the result: the duplicate was created moments later with nothing left to close it, and it pinned the staged bytes of an upload whose name the cleanup had already unlinked. Cancellation after the duplication was just as leaky, because the commit-path handler caught Exception and CancelledError is not one. The duplication now runs as its own task, shielded from the caller's cancellation, and closes its own result when the caller is gone by the time the worker finishes. The commit path catches BaseException, closing the descriptor it already owns before re-raising. Both windows are pinned: one test stalls the duplication worker after it allocates and cancels ingestion, the other stalls the commit so the cancellation lands while the descriptor is owned. * fix(uploads): drain the conversion copy so its descriptor always closes Review follow-up. The copy worker owns the duplicated descriptor and closes it in its own finally, but a bare await let a cancellation cancel the executor job while it was still queued: the worker never ran, so that finally never ran either, and the enclosing scope had already handed ownership away and saw None. Draining also keeps a late worker from writing into a private directory this scope has since removed. The copy now goes through await_drained, the shield-and-drain helper the Gateway already uses for offloads that must not be abandoned mid-flight. Pinned by a test that holds the copy job queued, cancels ingestion, then releases it and requires the descriptor to come back closed.