mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-26 07:57:57 +00:00
01a89f2379
* refactor(memory): pluggable MemoryManager interface for backend onboarding
Optimize the MemoryManager interface layer so new backends (mem0/openviking)
onboard with less code and the contract stays stable as capabilities are
added. A minimal backend now implements only from_config + add + get_context
(verified by test_memory_manager_interface.py::_MinimalBackend onboarding via
the factory); the factory no longer knows a backend's private hooks.
- MemoryManager: ABC -> pydantic BaseModel; three-tier methods (tier-1
add/get_context abstract; tier-2 management defaults; tier-3 optional hooks
warm/reload/fact + on_pre_compress/on_turn_start). Dropped 3 self-serving
hooks. 6 hasattr probe sites -> direct call + try/except NotImplementedError.
- from_config classmethod: factory thins to resolve + inject storage_path +
collect host hooks + call from_config; DeerMem-specific hook consumption
moved from factory to DeerMem.from_config.
- Invariants: @model_validator (mode='tool' requires search via supports_search
ClassVar); DeerMemConfig storage_path-is-file check moved here from factory.
- Async: aadd/aget_context/asearch default to the sync path (speculative).
- Callbacks: MemoryCallbacks + LangfuseMemoryCallbacks; on_memory_llm_call
subsumes tracing_callback (same signature/timing/mutation); deleted the
tracing_callback field. DeerMem decoupled from langfuse (portability).
- noop keeps read-op empty overrides (avoids router 500s on the
disable-memory-via-noop path); only delete/export inherit the base raise.
Behavior preserved: 661 passed / 13 skipped. Docs: backends/README.md rewritten
(three-tier + from_config + callbacks); samples README updated; removed stale
private doc paths.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(memory): 501 on unsupported read/manage endpoints + accurate warm log
Review follow-up on the three-tier MemoryManager refactor.
- Read/manage endpoints (GET /memory, /memory/export, /memory/status,
DELETE /memory, POST /memory/import) and the /memory/reload fallback now
catch NotImplementedError -> 501, matching the fact-CRUD endpoints. The
hasattr->try/except migration had skipped these: they were @abstractmethod
before (every backend implemented them, so they never raised), so once they
became tier-2 default-raise a minimal backend (only add + get_context) hit a
raw 500 -- there is no global NotImplementedError handler. get_memory is
shared via _get_memory_or_501 (covers /memory, export, status, reload
fallback). noop is unchanged: its read-op empty overrides never raise.
- warm() base default returns None (tri-state: True=warmed, False=failed,
None=nothing to warm) so the Gateway lifespan logs "skipping" for a
non-DeerMem backend (e.g. noop) instead of the inaccurate "warmed
successfully" it never earned. DeerMem.warm keeps True/False.
- Tests: 6 router 501 tests (read/manage + reload fallback) + 2 lifespan
warm-log tests (None->skipping, False->warning); conformance/pluggable
assert warm() is None.
705 passed / 13 skipped; lint clean.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(memory): review follow-ups - search-flag consistency, client reload, backend_config purity
Address review feedback on the three-tier MemoryManager refactor:
- [Medium] supports_search/search drift: the invariant now requires the
supports_search ClassVar flag to MATCH whether search() is actually
overridden (type(self).search is not MemoryManager.search), so the flag
can't drift from the impl. Catches both directions at instantiation: a
backend that overrides search() but forgets supports_search=True (was a
misleading tool-mode rejection), and one that sets the flag without
overriding (was a runtime NotImplementedError on the first memory_search).
noop sets supports_search=True to match its search() override. Conformance
adds drift + consistent-backend tests.
- [Low] client.reload_memory fallback: wrap the get_memory fallback so a
minimal backend (only add + get_context) surfaces a clean NotImplementedError
("implements neither reload_memory nor get_memory") instead of an uncaught
propagation -- mirrors the router's 501. Test added.
- [Low] backend_config purity: DeerMem.from_config restores backend_config to
the pure data the host passed after model_post_init parses the injected hooks
into DeerMemConfig (self._config, PrivateAttr); the field stays serializable
(no callables/LLM) and matches the README ("host hooks NOT in backend_config").
Test asserts purity + hooks wired.
- [Low] CHANGELOG: breaking-change note that mode='tool' + non-search backend
now fails fast at startup (was silently empty) so operators recognize it on
upgrade.
- [Nit] .gitignore: drop the env-specific .tmp-pytest/ entry (--basetemp is
local-only, not make test/CI).
709 passed / 13 skipped; lint clean.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(changelog): correct memory tool-mode fail-fast note
The CHANGELOG entry said mode='tool' + a non-search backend "(e.g. noop)"
fails fast at startup, but noop overrides search() (returns []) and sets
supports_search=True (required by the consistency invariant), so noop IS
search-capable and noop+tool does NOT fail fast. The fail-fast only affects a
custom backend that onboards without overriding search(). Reworded to drop the
misleading noop example and state both shipping backends implement search().
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
58 lines
2.4 KiB
Markdown
58 lines
2.4 KiB
Markdown
# DeerMem portability demo (other-agent integration)
|
|
|
|
`backends/deermem/` is a **self-contained, portable** memory backend. It has
|
|
exactly **one** `from deerflow` line -- the ABC contract
|
|
(`from deerflow.agents.memory.manager import MemoryManager` in `deer_mem.py`).
|
|
Everything else is relative imports within the folder. So another agent can
|
|
adopt DeerMem in three steps, with **zero deer-flow code**.
|
|
|
|
## Three steps
|
|
|
|
1. **Vendor the host contract** -- copy `agents/memory/manager.py` (the `MemoryManager`
|
|
ABC + `get_memory_manager()` factory + `_scan_backends()`) into your agent's
|
|
tree. It is small and host-neutral (9 abstract methods + a drop-in factory).
|
|
(A minimal ABC is enough if you instantiate `DeerMem` directly.)
|
|
|
|
2. **Drop the backend** -- copy the `backends/deermem/` folder into your agent's
|
|
`backends/`. Change exactly **one line** in `deer_mem.py`:
|
|
```python
|
|
# from
|
|
from deerflow.agents.memory.manager import MemoryManager
|
|
# to (your agent's vendored contract)
|
|
from <your_agent>.memory.manager import MemoryManager
|
|
```
|
|
Nothing else in the folder needs editing (all other imports are relative).
|
|
|
|
3. **Configure** -- drop a `deermem_manager.yaml` (see below) and call
|
|
`get_memory_manager()` (or construct `DeerMem(backend_config=...)` directly).
|
|
|
|
DeerMem runs with **zero `backend_config`** (defaults: storage at
|
|
`$DEERMEM_DATA_DIR` / `~/.deermem/`, no LLM so non-LLM ops work; set `model` to
|
|
enable memory extraction). See `deermem_manager.yaml`.
|
|
|
|
## Proof
|
|
|
|
`tests/test_deermem_self_contained.py::test_portability_vendor_to_other_agent`
|
|
copies `backends/deermem/` into a temp package, repoints the one ABC import to
|
|
a minimal vendored `manager.py`, imports it, and runs an `import_memory` ->
|
|
`get_context` round-trip -- with **zero deer-flow dependency at runtime**.
|
|
|
|
## Sample config
|
|
|
|
`deermem_manager.yaml`:
|
|
```yaml
|
|
manager_class: deermem
|
|
backend_config:
|
|
storage_path: ~/.myagent/memory # empty = $DEERMEM_DATA_DIR / ~/.deermem/
|
|
model: # memory-update LLM (empty = no LLM)
|
|
provider: openai # any langchain init_chat_model provider
|
|
model: gpt-4o-mini
|
|
api_key: ${OPENAI_API_KEY}
|
|
base_url: https://api.openai.com/v1
|
|
debounce_seconds: 30
|
|
max_facts: 100
|
|
# callbacks / should_keep_hidden_message: programmatic (cannot come from
|
|
# YAML); the host factory injects them via from_config. Set programmatically
|
|
# only if overriding the host defaults.
|
|
```
|