mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 21:26:39 +00:00
52046aa82e
* fix(projects): convert shelf documents outside the SQLite write lock Lazy conversion ran pymupdf/markitdown inside the BEGIN IMMEDIATE transaction that serializes the derived/converted.md publish against trash and purge. On SQLite that lock is database-wide, so every unrelated writer (run status, thread metadata, the scheduler) waited for the whole conversion and failed with "database is locked" past busy_timeout (30s). Conversion now writes into .staging/ outside any transaction. The document row lock is taken only to revalidate the live row and atomically rename the output into place (publish_under_live_lock, renamed from convert_under_live_lock); a row trashed or purged meanwhile discards the staged output, so nothing is published after purge. A read-only probe skips converting rows that are already gone, a failed conversion is still revalidated so a purged row reports content_missing, and the conversion is drained on cancellation so its staging file is always removed. * docs(changelog): reference #6305 in the shelf conversion lock entry * fix(projects): share one conversion across concurrent first reads A first-read conversion holds a bounded file-IO pool thread for its whole duration. With the conversion moved out of the row lock, concurrent first reads of one document each converted on their own thread, so a burst of reads could starve unrelated offloaded file work. Route the conversion through a single-flight keyed by (user, document, project): the first reader leads and the rest wait on its result without holding a pool thread. The map holds concurrent.futures.Future values under a threading.Lock because the read tool also runs on the isolated subagent loop. Followers wait through asyncio.shield, so a cancelled follower cannot cancel the shared future, and a leader that is cancelled or raises hands the work to a follower instead of failing it.