mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-27 16:37:55 +00:00
8145d66a33
* feat(memory): signals-based update pipeline + always-on watermark/trivial filter
Refactor the DeerMem memory update pipeline (message_processing -> queue ->
updater) around a signals frozenset seam, replacing the
(filtered, correction_detected, reinforcement_detected) 3-tuple with
(filtered, signals: frozenset[str]) end to end.
message_processing:
- Externalize signal-detection patterns to YAML (message_patterns/*.yaml).
- Extend signals from correction/reinforcement to a 6-class set
(correction/reinforcement/preference/identity/goal/decision); detect_signals
returns a frozenset aligned with the fact category enum.
- Pure-acknowledgment turns ("ok"/"好的"/...) are always filtered out before
enqueue (whole-message fullmatch), saving an extraction LLM call.
queue (core/queue.py):
- In-memory list + debounce timer, with flush_sync (graceful-shutdown drain
that joins an in-flight worker under a hard timeout) and queue_max_depth
backpressure (signal-bearing updates always admitted; QueueFull otherwise).
- Same-key updates coalesce with a signal union; per-batch success/fail summary.
updater (core/updater.py):
- head500+tail500 message truncation (replaces the 1000-char head chop).
- Always-on per-thread watermark: feed only messages added since the last
extraction. The watermark is in-memory and is not advanced on failure, so a
failed/lost update is re-fed on the next conversation turn.
- [MANUAL] prompt marker for user-authored facts (source.type="manual").
- Post-invoke extraction_callback (host-injected) emitting facts_extracted /
facts_accepted / rejected_low_confidence; the host default logs metrics and
flags >60% rejection.
Confidence filtering remains in _apply_updates (the existing
fact_confidence_threshold check); there is no separate write gate.
Consolidation stays opt-in (lossy). The ABC add/add_nowait signature is
unchanged, so the summarization flush hook and host are unaffected.
Tests: add test_message_processing_signals, test_updater_truncation,
test_updater_watermark; update queue/updater/consolidation/staleness/pluggable
tests for the signals seam.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(memory): harden update pipeline per PR review
- Catch QueueFull in DeerMem.add/add_nowait so backpressure degrades to
'update skipped' instead of propagating into after_agent /
summarization_hook and breaking the agent run (peer middlewares
self-guard; MemoryMiddleware was the lone exception). Emergency
(add_nowait) always admits under backpressure -- its data cannot be
re-fed next turn.
- Rewrite the watermark from index-based to content/identity-based
(_message_identity + _feed_after_watermark) so it stays correct when
summarization removes the conversation front -- an index watermark
pointed at the wrong message and silently skipped un-extracted tail
turns. The emergency flush bypasses the watermark (bypass_watermark on
ConversationContext, threaded through update_memory) and coexists with
(does not replace) a pending normal update, so a flush cannot drop a
pending update's un-extracted tail.
- Populate facts_accepted / rejected_low_confidence inside _apply_updates
at the real confidence-filter site (passed_threshold) instead of
re-deriving the threshold in _finalize_update -- eliminates metric drift.
- Emit extraction metrics in a finally with an 'attempted' flag so
exception failures (parse error, apply_changes raise after retry) are
observable, not only the happy path.
- Re-detect signals on the post-watermark feed for the extraction hint so
it no longer references turns the LLM cannot see; admission-time signals
still drive backpressure.
- Move the post-batch reschedule inside the queue lock to close a
non-atomic self._timer race with a concurrent add.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(memory): address follow-up review nits (LRU, metric name, docstring)
- Bound the in-memory watermark cache with a configurable LRU
(watermark_max_keys, default 4096, 0=unbounded). A dropped key re-extracts
one batch on that thread's next turn (the documented restart behavior), so
eviction is safe and preserves the content-identity watermark's
front-removal guarantee. Adds _watermark_get/_watermark_set helpers and a
bounded-LRU regression test.
- Rename the extraction metric facts_accepted -> facts_passed_confidence so
the name matches what the >60% rejection-rate warning assumes (a
confidence-gate signal, not a persisted-fact count); drop the stale
"historical semantics" justification. Brand-new callback, one consumer.
- Fix the stale test_message_processing_signals module docstring: the signals
seam is already swapped to frozenset, and a stale stage-numbering prefix is
removed.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
131 lines
4.3 KiB
Python
131 lines
4.3 KiB
Python
"""Tests for message_processing: externalized signal patterns.
|
|
|
|
Covers:
|
|
- load_patterns (bundled defaults, caching, custom dir override, missing file)
|
|
- detect_correction / detect_reinforcement (backward-compatible signature,
|
|
patterns override, last-6 window)
|
|
- DeerMem._prepare_update (3-tuple, returns None when missing a role)
|
|
- DeerMemConfig.patterns_dir default
|
|
"""
|
|
|
|
import re
|
|
|
|
from langchain_core.messages import AIMessage, HumanMessage
|
|
|
|
from deerflow.agents.memory.backends.deermem.deer_mem import DeerMem
|
|
from deerflow.agents.memory.backends.deermem.deermem.config import DeerMemConfig
|
|
from deerflow.agents.memory.backends.deermem.deermem.core.message_processing import (
|
|
detect_correction,
|
|
detect_reinforcement,
|
|
filter_messages_for_memory,
|
|
load_patterns,
|
|
)
|
|
|
|
|
|
def _human(text: str) -> HumanMessage:
|
|
return HumanMessage(content=text)
|
|
|
|
|
|
def _ai(text: str) -> AIMessage:
|
|
return AIMessage(content=text)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# load_patterns
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_load_patterns_bundled_nonempty():
|
|
assert len(load_patterns("correction")) > 0
|
|
assert len(load_patterns("reinforcement")) > 0
|
|
|
|
|
|
def test_load_patterns_cached():
|
|
assert load_patterns("correction") is load_patterns("correction")
|
|
|
|
|
|
def test_load_patterns_custom_dir_overrides(tmp_path):
|
|
(tmp_path / "correction.yaml").write_text("- 'foobarbaz'\n", encoding="utf-8")
|
|
pats = load_patterns("correction", patterns_dir=str(tmp_path))
|
|
assert len(pats) == 1
|
|
assert pats[0].search("hello foobarbaz world")
|
|
# bundled defaults remain intact (different cache key)
|
|
assert len(load_patterns("correction")) > 1
|
|
|
|
|
|
def test_load_patterns_missing_file_explicit_dir_raises(tmp_path):
|
|
import pytest
|
|
|
|
with pytest.raises(FileNotFoundError, match="nope"):
|
|
load_patterns("nope", patterns_dir=str(tmp_path))
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# detect_correction / detect_reinforcement
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_detect_correction_default_bundled():
|
|
msgs = [_human("That's wrong, use uv"), _ai("ok")]
|
|
assert detect_correction(msgs) is True
|
|
assert detect_reinforcement(msgs) is False
|
|
|
|
|
|
def test_detect_reinforcement_default_bundled():
|
|
msgs = [_human("perfect, exactly right"), _ai("great")]
|
|
assert detect_reinforcement(msgs) is True
|
|
|
|
|
|
def test_detect_correction_patterns_override():
|
|
custom = [re.compile(r"zzz")]
|
|
assert detect_correction([_human("zzz here")], patterns=custom) is True
|
|
assert detect_correction([_human("That's wrong")], patterns=custom) is False
|
|
|
|
|
|
def test_detect_window_is_last_six():
|
|
# 7 human turns; a correction in the oldest (outside [-6:]) is not detected.
|
|
msgs = [_human(f"msg {i}") for i in range(7)]
|
|
msgs[0] = _human("That's wrong, old")
|
|
assert detect_correction(msgs) is False
|
|
msgs[-1] = _human("That's wrong, recent")
|
|
assert detect_correction(msgs) is True
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# DeerMem._prepare_update (3-tuple, signal detection with externalized patterns)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _make_deermem(tmp_path) -> DeerMem:
|
|
return DeerMem(backend_config={"storage_path": str(tmp_path)})
|
|
|
|
|
|
def test_prepare_update_missing_role_returns_none(tmp_path):
|
|
m = _make_deermem(tmp_path)
|
|
assert m._prepare_update([_human("only human")]) is None
|
|
assert m._prepare_update([_ai("only ai")]) is None
|
|
assert m._prepare_update([]) is None
|
|
|
|
|
|
def test_prepare_update_returns_signals_with_correction(tmp_path):
|
|
m = _make_deermem(tmp_path)
|
|
r = m._prepare_update([_human("That's wrong, use uv"), _ai("ok")])
|
|
assert r is not None and len(r) == 2
|
|
filtered, signals = r
|
|
assert "correction" in signals
|
|
assert len(filtered) == 2
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Config defaults
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_config_patterns_dir_default():
|
|
assert DeerMemConfig().patterns_dir is None
|
|
|
|
|
|
def test_filter_messages_backward_compat():
|
|
filtered = filter_messages_for_memory([_human("hello"), _ai("hi there")])
|
|
assert len(filtered) == 2
|