deer-flow

mirror of https://github.com/bytedance/deer-flow.git synced 2026-05-30 20:38:09 +00:00

History

fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#3084 )

* fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#2816)

- JsonlRunEventStore: offload all file I/O to asyncio.to_thread() so the
  event loop is never blocked; add per-thread asyncio.Lock to serialise
  concurrent puts and prevent interleaved JSONL lines
- Split _ensure_seq_loaded into a sync _compute_max_seq (runs in thread)
  and an async wrapper; seq counter is recovered from disk on fresh store init
- DbRunEventStore.put_batch: raise ValueError when events span multiple
  thread_ids (previously silently assumed same thread)
- Add test_jsonl_event_store_async_io.py: 12 tests covering lock reuse,
  concurrent seq monotonicity, disk recovery, and mixed-thread batch rejection

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address Copilot review comments

- delete_by_thread: pop _write_locks after releasing the lock to prevent
  unbounded growth when threads are repeatedly created and deleted
- tests: add regression guard asserting asyncio.to_thread is called for
  _write_record in put(); assert _write_locks entry removed on delete

* fix(lint): move patch import to local scope to fix ruff I001

* fix(lint): apply ruff check+format fixes to test file

* fix(runtime): address review feedback for JSONL async I/O hardening (#2816)

Use setdefault for atomic lock init in _get_write_lock; pop _write_locks
inside the held lock scope in delete_by_thread; update test docstring
and assert lock entry also cleared on delete.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: rayhpeng <rayhpeng@gmail.com>

2026-05-29 09:27:53 +08:00

checkpointer

feat(tests): add Blockbuster runtime gate for event-loop blocking IO (#3229 )

2026-05-26 23:03:49 +08:00

events

fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#3084 )

2026-05-29 09:27:53 +08:00

runs

fix(runtime): make run creation persistence atomic (#3152 )

2026-05-23 22:43:34 +08:00

store

fix(packaging): add postgres extra for store/checkpointer supportFix postgres extra install guidance (#2584 )