From 6cbf20fd39b34514f53f50fa097220a1deded65a Mon Sep 17 00:00:00 2001 From: ajayr Date: Wed, 12 Aug 2026 02:02:08 +0100 Subject: [PATCH] feat(memory): add Honcho backend (user-model memory provider) (#4730) * feat(memory): honcho backend config parsing Co-Authored-By: Claude Fable 5 * feat(memory): honcho v3 http client Co-Authored-By: Claude Fable 5 * feat(memory): honcho memory manager (workspace-per-user, fail-closed identity, async offload) - HonchoMemoryManager implements the MemoryManager contract (add/get_context/ search/get_memory/shutdown_flush + aadd/aget_context/asearch offloaded via asyncio.to_thread), signatures verified against manager.py's tier-1/tier-2/ async abstracts. - Workspace resolution: workspace_overrides[user_id] else workspace_prefix + sanitize_id(user_id); missing/empty user_id fails closed (no-op write, empty read) rather than falling back to a shared workspace. User peer: user_peer_overrides[user_id] else sanitize_id(user_id). - get_context self-truncates to max_injection_chars and raises MemoryManagerError only under failure_policy.read=fail_closed; default is log-and-return "". - Restore backends/honcho/__init__.py to the noop direct-import convention (MANAGER_CLASS = HonchoMemoryManager) now that honcho_manager.py exists, replacing Task 10's temporary lazy __getattr__ scaffold. - Fix Task 10 deferred docstring minor: sanitize_id docstring now states the grammar allows up to 100 chars while this helper caps at 64. - 19 new tests appended to test_honcho_memory_backend.py (write/read/async/ lifecycle/factory-discovery); 27/27 pass. Verified end-to-end that manager.py's drop-in backend scanner resolves "honcho" with no core edits. Co-Authored-By: Claude Fable 5 * fix(memory): collision-resistant identity derivation, exception containment, passive-writes flag Task review findings (2 Critical + 1 Important), all fixed in the same worktree: - CRITICAL (cross-user bleed): sanitize_id is lossy -- "user.name@example.com" and "user-name@example.com" both sanitized to the same string, merging two users' memory into one workspace/peer. Add _stable_id() (sanitize_id output + 8-hex-char SHA-256 suffix of the raw id) and use it on the default (non-override) path in _workspace/_user_peer; workspace_overrides / user_peer_overrides still match on the raw key, unchanged. The hash suffix also guarantees a non-empty result for a raw id that sanitizes to "" (e.g. "!!!"), so _user_peer can no longer return "". Documented in the manager's isolation docstring. - CRITICAL (exception containment): client.py's _post() called response.json() outside the try block, so a 200 with a non-JSON body raised a bare JSONDecodeError that would escape add() with no upstream handler. Wrap the parse and raise HonchoRequestError (mirrors Mem0Client._request). Broadened the manager's four boundary excepts from `except HonchoRequestError` to `except Exception` (mirrors openviking_manager.py's broad-guard precedent), with `except MemoryManagerError: raise` first so a contract error is never swallowed or double-wrapped. - IMPORTANT: added requires_passive_writes_in_tool_mode: ClassVar[bool] = True -- Honcho's only write path is passive add() (no fact CRUD hooks), so tool mode must keep MemoryMiddleware writes flowing to the deriver. Mirrors mem0_manager.py's identical flag/rationale. Minors addressed: get_memory(user_id=None) empty-shape-with-no-calls test; empty-string user_id tests for add()/get_context(); dedicated collision test proving two colliding raw ids resolve to different workspaces/peers. 10 new tests (37/37 total pass); RED verified by stashing only the implementation files (tests import the not-yet-existing _stable_id, so the whole module fails to collect) before restoring the fix. Co-Authored-By: Claude Fable 5 * test(memory): blocking-io anchor for honcho backend; docs + config example - Adds test_honcho_memory_backend.py in tests/blocking_io/ with fake-client blocking IO - Mirrors openviking anchor structure and conftest conventions - Updates backends/README.md with honcho row and config keys section - Updates config.example.yaml with honcho commented block - Updates backend/AGENTS.md with honcho memory backend bullet - Documents workspace resolution (prefix + collision-resistant sanitized id) - Documents tool mode passive write retention via MemoryMiddleware - Documents async entrypoint offloading via asyncio.to_thread - Documents fail_closed vs fail_open recall failure policy Co-Authored-By: Claude Fable 5 * fix(memory): wire close() to shutdown hook; correct honcho README defaults and tool-mode note - HonchoMemoryManager.close() releases the HTTP client, mirroring mem0_manager.py's pattern and the base MemoryManager.close() shutdown hook. - README: fix workspace_prefix (deerflow-u-), message_char_limit (8000), max_injection_chars (6000), and base_url (default http://localhost:8000, not required) against backends/honcho/config.py; add missing timeout_seconds/connect_timeout_seconds rows; replace the "middleware mode only" claim with wording matching reality (tool mode supported, search implemented, passive writes retained via MemoryMiddleware). Co-Authored-By: Claude Fable 5 * fix(memory): honor failure_policy.read on all honcho recall paths; review nits Addresses PR #4730 review feedback: - search() and get_memory() now route through a _read_or_fallback policy gate (mem0's pattern), so failure_policy.read: fail_closed raises MemoryManagerError on every recall path as documented; get_context() uses the same helper, preventing future drift. - Session ids use the collision-resistant _stable_id derivation; bare sanitize_id would merge threads like "t.1"/"t-1" into one session. - HonchoClient accepts a transport kwarg (Mem0Client precedent) so tests inject httpx.MockTransport through the constructor. - Config: empty/null workspace/peer override values fail fast at parse time instead of silently falling through to the default derivation. - _UTC_NOW_FIELDS 1-tuple replaced by a plain _UTC_NOW_FORMAT constant. - README: user_peer_overrides row described the wrong target (it overrides the user's own peer, not assistant_peer); document the non-empty constraint on override values. Co-Authored-By: Claude Fable 5 * docs(memory): qualify honcho isolation claim for shared workspace_overrides The module docstring claimed users cannot see each other's memory by construction, unconditionally. That holds for the default one-workspace-per-user derivation, but a workspace_overrides entry mapping several users to one workspace shares that workspace's search index: search() uses Honcho's workspace-scoped /search (no peer filter), while get_context()/get_memory() stay peer-scoped via working_representation. State the asymmetry in the docstring, the README Workspace Resolution section, and the workspace_overrides table row. Docs-only; no behavior change (review follow-up on #4730). Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- backend/AGENTS.md | 1 + .../deerflow/agents/memory/backends/README.md | 26 + .../agents/memory/backends/honcho/__init__.py | 5 + .../agents/memory/backends/honcho/client.py | 71 +++ .../agents/memory/backends/honcho/config.py | 75 +++ .../memory/backends/honcho/honcho_manager.py | 298 +++++++++++ .../blocking_io/test_honcho_memory_backend.py | 75 +++ backend/tests/test_honcho_memory_backend.py | 479 ++++++++++++++++++ config.example.yaml | 12 + 9 files changed, 1042 insertions(+) create mode 100644 backend/packages/harness/deerflow/agents/memory/backends/honcho/__init__.py create mode 100644 backend/packages/harness/deerflow/agents/memory/backends/honcho/client.py create mode 100644 backend/packages/harness/deerflow/agents/memory/backends/honcho/config.py create mode 100644 backend/packages/harness/deerflow/agents/memory/backends/honcho/honcho_manager.py create mode 100644 backend/tests/blocking_io/test_honcho_memory_backend.py create mode 100644 backend/tests/test_honcho_memory_backend.py diff --git a/backend/AGENTS.md b/backend/AGENTS.md index 01b33dbc1..4a58e2ffd 100644 --- a/backend/AGENTS.md +++ b/backend/AGENTS.md @@ -1094,6 +1094,7 @@ The cached value is reused for both the blocking (`runs.wait`) and streaming (`_ imports of the OpenViking embedded runtime. Multi-user provisioning, query-aware refresh policy and new lifecycle scheduling are separate changes, not part of this backend. +- The optional `honcho` backend under `packages/harness/deerflow/agents/memory/backends/honcho/` is a remote-only HTTP adapter for user-model memory (RFC #1898's user-dimension option). Select with `memory.manager_class: honcho`, keep `memory.mode: middleware` (tool mode also supported — it implements `search`). It writes filtered turns as Honcho messages (no local LLM calls; Honcho's deriver builds representations server-side), resolves one workspace per `user_id` (`workspace_overrides` else `workspace_prefix + collision-resistant sanitized id`; missing user fails closed to no memory), offloads sync HTTP in its `a*` overrides via `asyncio.to_thread`, and tool mode retains passive writes via MemoryMiddleware, mirroring mem0. `failure_policy.read: fail_closed` rethrows recall failures; default is log-and-empty. - `memory.mode: tool` skips `MemoryMiddleware` and registers `memory_search`, `memory_add`, `memory_update`, and `memory_delete` on the agent. The model decides when to search, add, update, or delete facts; this is opt-in/experimental and should not be described as better than middleware mode without eval evidence. - Both modes share `FileMemoryStorage`, per-user/per-agent isolation, manual CRUD primitives, and the updater backend. Injection is mode-aware: middleware mode injects global `user`/`history` summaries plus the selected agent's facts, while tool mode injects only the global summaries and leaves every agent fact behind `memory_search` to avoid duplicating automatically injected and retrieval-returned context. `memory.injection_enabled: false` suppresses the complete block in either mode. - Middleware extraction classifies proposed facts with extraction-only `scope`/`durability`/`authority` labels. `_apply_updates` accepts only `user` + `durable` + `descriptive` new/consolidated facts, accepts only wholly user-scoped summary prose with `authority=descriptive`, and rejects missing labels per item without aborting unrelated updates. Contradiction removals use object entries with `id`, `scope`, `reason`, and optional zero-based `replacementFactIndex`; task/project removals fail closed, and a paired removal runs only when the referenced replacement survives the scope/confidence gates, deduplication, and max-fact trim under another fact ID. The labels are not persisted, so no storage migration is required. Staleness removals retain their independent candidate/cap guardrails, while tool-mode CRUD remains outside this extraction gate. Custom `memory.backend_config.prompts_dir` templates (including per-agent overrides) must carry the same classification fields; an un-migrated template makes the fail-closed gate reject every extraction-driven write, observable only through `rejected_by_scope_gate` and the >60% fact-rejection warning. diff --git a/backend/packages/harness/deerflow/agents/memory/backends/README.md b/backend/packages/harness/deerflow/agents/memory/backends/README.md index e3a9e7c54..e1aaa72fc 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/README.md +++ b/backend/packages/harness/deerflow/agents/memory/backends/README.md @@ -6,6 +6,7 @@ Each subfolder under `agents/memory/backends/` is a pluggable memory backend. Sw - `noop/` - an empty backend and the **template** to copy when adding a new one. - `openviking/` - optional remote backend using the official `langchain-openviking` package (single-user middleware mode). +- `honcho/` - optional remote Honcho backend over HTTP: user-model memory (representations built by Honcho's server-side deriver; no local LLM calls). Workspace-per-user isolation with per-user overrides. This guide tells you **which files to touch** when you change, swap, or add a memory system. Paths are relative to `backend/` unless noted. @@ -127,6 +128,31 @@ Lessons from integrating external backends: 5. **Restart deer-flow after changes.** The manager is a process-level singleton; a running process does not hot-reload config or backend code. 6. **Cap `get_context` length yourself.** The host applies no token budget; the backend must truncate (DeerMem has `max_injection_tokens`; noop does not). +## Honcho Backend + +The optional `honcho/` backend is a remote-only HTTP adapter for user-model memory over Honcho (self-hosted or via api.honcho.dev). Middleware mode is the default; tool mode is also supported (search is implemented) and retains passive writes via `MemoryMiddleware` (`requires_passive_writes_in_tool_mode = True`) so Honcho's server-side deriver keeps building representations from every turn (no local LLM calls). + +**Configuration** (under `memory.backend_config`): + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `base_url` | str | `http://localhost:8000` | Honcho instance URL (e.g., `http://localhost:8000` or `https://api.honcho.dev`) | +| `api_key` | str | optional | API key for hosted Honcho; required if `base_url` is `api.honcho.dev`. Can use `$HONCHO_API_KEY` env var syntax. Requires `allow_insecure_http: true` when using plain HTTP | +| `allow_insecure_http` | bool | false | Allow HTTP (non-HTTPS) connections; needed for localhost development with api_key | +| `timeout_seconds` | float | `10.0` | HTTP client timeout (seconds) for calls to Honcho — read/write/pool; see `connect_timeout_seconds` for the connect phase | +| `connect_timeout_seconds` | float | `3.0` | HTTP connect timeout (seconds) for establishing the connection to Honcho | +| `workspace_prefix` | str | `deerflow-u-` | Prefix for isolated workspaces; each user gets one workspace named `{prefix}{sanitized_id}` | +| `workspace_overrides` | dict | `{}` | Map specific user ids to custom workspace names; overrides the prefix-based derivation. Values must be non-empty (parse error otherwise). Mapping several users to one workspace shares its search index across them (see Workspace Resolution) | +| `user_peer_overrides` | dict | `{}` | Map specific user ids to custom names for the user's own peer; overrides the stable-id derivation. Values must be non-empty (parse error otherwise) | +| `assistant_peer` | str | `deerflow` | Default peer name for the assistant when storing messages | +| `message_char_limit` | int | `8000` | Character limit per message; longer messages are truncated | +| `max_injection_chars` | int | `6000` | Character limit for injected memory into the system prompt | +| `failure_policy.read` | str | `fail_open` | Recall failure handling: `fail_open` (log and return empty) or `fail_closed` (rethrow) | + +**Workspace Resolution**: Each DeerFlow user maps to one Honcho workspace. The workspace name is derived as: `workspace_overrides[user_id]` (if present) else `workspace_prefix + sanitized_id`, where `sanitized_id` is a collision-resistant hash suffix (sanitize[:48]-sha256[:8]). Missing user fails closed to no memory. The default derivation is isolated per user; a `workspace_overrides` entry that maps several users to one workspace deliberately shares that workspace's **search index** across them (`search` uses Honcho's workspace-scoped `/search`, which has no peer filter), while `get_context` / `get_memory` remain peer-scoped. + +**Tool Mode**: While tool-mode memory tools are not fully supported, the backend implements `requires_passive_writes_in_tool_mode = True` to retain passive writes via MemoryMiddleware while also enabling memory search through the `memory_search` tool. + ## Reference - **Template**: `noop/` - minimal implementation with full docstrings; copy and go. diff --git a/backend/packages/harness/deerflow/agents/memory/backends/honcho/__init__.py b/backend/packages/harness/deerflow/agents/memory/backends/honcho/__init__.py new file mode 100644 index 000000000..00f447521 --- /dev/null +++ b/backend/packages/harness/deerflow/agents/memory/backends/honcho/__init__.py @@ -0,0 +1,5 @@ +"""Honcho memory backend package. See ``honcho_manager.py``.""" + +from .honcho_manager import HonchoMemoryManager + +MANAGER_CLASS = HonchoMemoryManager diff --git a/backend/packages/harness/deerflow/agents/memory/backends/honcho/client.py b/backend/packages/harness/deerflow/agents/memory/backends/honcho/client.py new file mode 100644 index 000000000..408564981 --- /dev/null +++ b/backend/packages/harness/deerflow/agents/memory/backends/honcho/client.py @@ -0,0 +1,71 @@ +"""Minimal synchronous Honcho v3 HTTP client for the memory backend. + +Deliberately dependency-light: plain httpx against the v3 REST API (peers and +sessions are get-or-create server-side, so every call here is idempotent). +Moving to the official ``honcho-ai`` SDK is a possible follow-up, mirroring the +OpenViking custom-HTTP -> official-adapter arc. +""" + +from __future__ import annotations + +from typing import Any + +import httpx + +from .config import HonchoConfig + + +class HonchoRequestError(RuntimeError): + """A Honcho API call failed (transport error or non-2xx response).""" + + +class HonchoClient: + def __init__(self, config: HonchoConfig, *, transport: httpx.BaseTransport | None = None) -> None: + # `transport` exists so tests can inject httpx.MockTransport (Mem0Client precedent). + headers = {"Content-Type": "application/json"} + if config.api_key: + headers["Authorization"] = f"Bearer {config.api_key}" + self._http = httpx.Client( + base_url=config.base_url, + headers=headers, + timeout=httpx.Timeout(config.timeout_seconds, connect=config.connect_timeout_seconds), + transport=transport, + ) + + def close(self) -> None: + self._http.close() + + def _post(self, path: str, payload: Any) -> Any: + try: + response = self._http.post(path, json=payload) + response.raise_for_status() + except httpx.HTTPError as exc: + raise HonchoRequestError(f"Honcho request failed: POST {path}: {exc}") from exc + if response.content: + try: + return response.json() + except ValueError as exc: + raise HonchoRequestError(f"Honcho returned non-JSON response: POST {path}: {exc}") from exc + return None + + def get_or_create_peer(self, workspace: str, peer_id: str) -> None: + self._post(f"/v3/workspaces/{workspace}/peers", {"id": peer_id}) + + def get_or_create_session(self, workspace: str, session_id: str) -> None: + self._post(f"/v3/workspaces/{workspace}/sessions", {"id": session_id}) + + def set_session_peers(self, workspace: str, session_id: str, peer_ids: list[str]) -> None: + self._post(f"/v3/workspaces/{workspace}/sessions/{session_id}/peers", {peer_id: {} for peer_id in peer_ids}) + + def add_messages(self, workspace: str, session_id: str, messages: list[dict[str, str]]) -> None: + self._post(f"/v3/workspaces/{workspace}/sessions/{session_id}/messages", {"messages": messages}) + + def working_representation(self, workspace: str, peer_id: str, *, max_conclusions: int = 25) -> str: + data = self._post(f"/v3/workspaces/{workspace}/peers/{peer_id}/representation", {"max_conclusions": max_conclusions}) + if isinstance(data, dict): + return str(data.get("representation") or "") + return "" + + def search(self, workspace: str, query: str, *, limit: int = 5) -> list[dict[str, Any]]: + data = self._post(f"/v3/workspaces/{workspace}/search", {"query": query, "limit": limit}) + return list(data) if isinstance(data, list) else [] diff --git a/backend/packages/harness/deerflow/agents/memory/backends/honcho/config.py b/backend/packages/harness/deerflow/agents/memory/backends/honcho/config.py new file mode 100644 index 000000000..5bbc726de --- /dev/null +++ b/backend/packages/harness/deerflow/agents/memory/backends/honcho/config.py @@ -0,0 +1,75 @@ +"""Honcho backend config — parses ``backend_config`` (see noop/config.py for the golden rule). + +The backend receives everything through the ABC method args and this dict; it +imports nothing from deer-flow. Self-hosted Honcho commonly runs auth-less over +plain HTTP; a configured ``api_key`` over plain HTTP requires the explicit +``allow_insecure_http: true`` opt-in (same posture as the mem0 backend). +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass, field +from typing import Any + +_ID_RE = re.compile(r"[^a-zA-Z0-9_-]+") + + +def sanitize_id(raw: str) -> str: + """Map an arbitrary string onto Honcho's id grammar (``^[a-zA-Z0-9_-]+$``; grammar allows up to 100, capped here at 64).""" + return _ID_RE.sub("-", str(raw)).strip("-")[:64] + + +def _parse_override_map(cfg: dict[str, Any], key: str) -> dict[str, str]: + """Overrides map raw user ids to explicit workspace/peer ids; an empty or + null VALUE is always a config mistake (empty string is falsy and would + silently fall through to the default derivation; YAML null would stringify + into an id literally named "None"), so fail fast at parse time.""" + out: dict[str, str] = {} + for k, v in (cfg.get(key) or {}).items(): + if v is None or not str(v).strip(): + raise ValueError(f"Honcho backend: {key}[{k!r}] has an empty value; remove the entry or set a non-empty id.") + out[str(k)] = str(v) + return out + + +@dataclass +class HonchoConfig: + base_url: str = "http://localhost:8000" + api_key: str | None = None + workspace_prefix: str = "deerflow-u-" + workspace_overrides: dict[str, str] = field(default_factory=dict) + user_peer_overrides: dict[str, str] = field(default_factory=dict) + assistant_peer: str = "deerflow" + timeout_seconds: float = 10.0 + connect_timeout_seconds: float = 3.0 + message_char_limit: int = 8000 + max_injection_chars: int = 6000 + allow_insecure_http: bool = False + read_fail_closed: bool = False + storage_path: str = "" + + @classmethod + def from_backend_config(cls, backend_config: dict[str, Any] | None) -> HonchoConfig: + cfg = dict(backend_config or {}) + failure_policy = cfg.get("failure_policy") or {} + base_url = str(cfg.get("base_url", "http://localhost:8000")).rstrip("/") + api_key = cfg.get("api_key") or None + allow_insecure = bool(cfg.get("allow_insecure_http", False)) + if api_key and base_url.startswith("http://") and not allow_insecure: + raise ValueError("Honcho backend: api_key over plain http requires backend_config.allow_insecure_http: true (the key would be sent unencrypted). Use https, or set the opt-in for local development.") + return cls( + base_url=base_url, + api_key=api_key, + workspace_prefix=str(cfg.get("workspace_prefix", "deerflow-u-")), + workspace_overrides=_parse_override_map(cfg, "workspace_overrides"), + user_peer_overrides=_parse_override_map(cfg, "user_peer_overrides"), + assistant_peer=str(cfg.get("assistant_peer", "deerflow")), + timeout_seconds=float(cfg.get("timeout_seconds", 10.0)), + connect_timeout_seconds=float(cfg.get("connect_timeout_seconds", 3.0)), + message_char_limit=int(cfg.get("message_char_limit", 8000)), + max_injection_chars=int(cfg.get("max_injection_chars", 6000)), + allow_insecure_http=allow_insecure, + read_fail_closed=str(failure_policy.get("read", "")).lower() == "fail_closed", + storage_path=str(cfg.get("storage_path") or ""), + ) diff --git a/backend/packages/harness/deerflow/agents/memory/backends/honcho/honcho_manager.py b/backend/packages/harness/deerflow/agents/memory/backends/honcho/honcho_manager.py new file mode 100644 index 000000000..fc763a6b8 --- /dev/null +++ b/backend/packages/harness/deerflow/agents/memory/backends/honcho/honcho_manager.py @@ -0,0 +1,298 @@ +"""Honcho memory backend — user-model memory via a Honcho (v3) instance. + +Positioning (upstream RFC #1898): Honcho covers the user-dimension of memory — +long-term user modeling, preferences, cross-session working representation — +complementing project/task-oriented backends. Ingestion is cheap (plain message +writes); Honcho's own server-side deriver performs fact extraction and +representation building asynchronously, so this backend makes **no LLM calls**. + +Multi-user isolation: every operation resolves a workspace from ``user_id`` +(``workspace_overrides`` exact match, else ``workspace_prefix + _stable_id``). +``_stable_id`` appends an 8-hex-char SHA-256 suffix to ``sanitize_id``'s output +because ``sanitize_id`` alone is lossy -- it collapses every run of non +``[a-zA-Z0-9_-]`` characters to a single ``-``, so distinct raw ids can collide +(``"user.name@x"`` and ``"user-name@x"`` both sanitize to ``"user-name-x"``). +Reusing the lossy form bare for the default (non-override) workspace/peer +derivation would silently merge two different people's memory into one +workspace; the hash suffix makes the default path collision-resistant while +staying readable. ``workspace_overrides`` / ``user_peer_overrides`` match on +the raw, un-sanitized key and are unaffected. Honcho scopes all queries to one +workspace, so under the default one-workspace-per-user derivation users cannot +see each other's memory by construction. A ``workspace_overrides`` entry +shared across users deliberately shares that workspace — ``get_context`` / +``get_memory`` stay peer-scoped there, but ``search`` uses Honcho's +workspace-scoped ``/search`` (no peer filter), so those users share one +search index. A missing ``user_id`` fails closed: the call becomes a no-op / +empty read, never a shared fallback workspace. Session ids reuse the same derivation +(``df-`` + ``_stable_id(thread_id)``) — bare ``sanitize_id`` would merge +threads like ``"t.1"`` and ``"t-1"`` into one Honcho session. + +Portability golden rule: the only ``from deerflow`` import is the contract line +below. Everything else arrives via ``backend_config``. +""" + +from __future__ import annotations + +import asyncio +import hashlib +import logging +from typing import Any, ClassVar, Literal + +from pydantic import PrivateAttr + +# ABC contract -- the ONE allowed `from deerflow` import in this backend folder. +from deerflow.agents.memory.manager import MemoryManager, MemoryManagerError + +from .client import HonchoClient +from .config import HonchoConfig, sanitize_id + +logger = logging.getLogger(__name__) + +_UTC_NOW_FORMAT = "%Y-%m-%dT%H:%M:%SZ" + + +def _now_iso() -> str: + import datetime + + return datetime.datetime.now(datetime.UTC).strftime(_UTC_NOW_FORMAT) + + +def _content_to_text(content: Any) -> str: + """Normalize LangChain message content (str or content-block list) to text.""" + if isinstance(content, str): + return content + if isinstance(content, list): + parts: list[str] = [] + for block in content: + if isinstance(block, str): + parts.append(block) + elif isinstance(block, dict) and isinstance(block.get("text"), str): + parts.append(block["text"]) + return "\n".join(parts) + return str(content or "") + + +def _stable_id(raw: str) -> str: + """Readable-but-collision-resistant id for the default (non-override) path. + + ``sanitize_id`` alone is lossy (every run of disallowed characters + collapses to one ``-``), so two distinct raw ids can sanitize to the same + string. Appending an 8-hex-char SHA-256 suffix of the *original* raw id + keeps the result readable while making distinct inputs resolve to + distinct outputs. The digest is always 8 hex characters, so the result is + never empty even when ``sanitize_id`` strips a degenerate raw id (e.g. + ``"!!!"``) down to ``""``. + """ + digest = hashlib.sha256(raw.encode("utf-8")).hexdigest()[:8] + readable = sanitize_id(raw)[:48].rstrip("-") + return f"{readable}-{digest}" if readable else digest + + +class HonchoMemoryManager(MemoryManager): + """MemoryManager backed by a Honcho v3 instance (self-hosted or hosted).""" + + _config: HonchoConfig = PrivateAttr(default=None) + _client: Any = PrivateAttr(default=None) + + supports_search: ClassVar[bool] = True + # Honcho's server-side deriver extracts facts/representation from add() + # writes asynchronously; this backend implements no fact CRUD hooks + # (create_fact/delete_fact/update_fact are unsupported), so tool mode + # must retain passive writes (MemoryMiddleware -> add()) to keep feeding + # the deriver, while search() supplies the query-aware retrieval tool + # mode expects. Mirrors mem0_manager.py's identical rationale. + requires_passive_writes_in_tool_mode: ClassVar[bool] = True + + def model_post_init(self, __context: Any) -> None: + self._config = HonchoConfig.from_backend_config(self.backend_config) + self._client = HonchoClient(self._config) + + @classmethod + def from_config( + cls, + backend_config: dict[str, Any] | None = None, + *, + mode: Literal["middleware", "tool"] = "middleware", + **host_hooks: Any, + ) -> HonchoMemoryManager: + """Config errors (bad URL/insecure key) raise here — fail fast at startup. + + Connectivity is deliberately NOT probed: a temporarily unreachable Honcho + must not block Gateway startup; reads degrade per ``failure_policy.read``. + """ + return cls(backend_config=backend_config, mode=mode) + + # ── identity resolution (fail closed) ──────────────────────────────── + def _workspace(self, user_id: str | None) -> str | None: + if not user_id: + return None + override = self._config.workspace_overrides.get(user_id) + if override: + return override + return f"{self._config.workspace_prefix}{_stable_id(user_id)}" + + def _user_peer(self, user_id: str) -> str: + return self._config.user_peer_overrides.get(user_id) or _stable_id(user_id) + + # ── recall policy gate (get_context / search / get_memory) ─────────── + def _read_or_fallback(self, fallback: Any, fn: Any) -> Any: + """Single ``failure_policy.read`` gate for every recall path, mirroring + mem0's helper of the same name: fail-open (default) logs and returns + ``fallback``; ``fail_closed`` wraps into ``MemoryManagerError``. The + broad ``except Exception`` is the containment boundary — no client + exception may escape into ``MemoryMiddleware.after_agent``.""" + try: + return fn() + except MemoryManagerError: + raise + except Exception as exc: + if self._config.read_fail_closed: + raise MemoryManagerError(f"honcho memory recall failed: {exc}") from exc + logger.warning("honcho memory: recall failed (fail-open): %s", exc) + return fallback + + # ── Tier 1: write ──────────────────────────────────────────────────── + def add( + self, + thread_id: str, + messages: list[Any], + *, + agent_name: str | None = None, + user_id: str | None = None, + trace_id: str | None = None, + ) -> None: + workspace = self._workspace(user_id) + if workspace is None or not user_id: + logger.debug("honcho memory: no resolvable user for thread %s; skipping write", thread_id) + return + user_peer = self._user_peer(user_id) + assistant_peer = self._config.assistant_peer + outgoing: list[dict[str, str]] = [] + for message in messages or []: + msg_type = getattr(message, "type", None) + text = _content_to_text(getattr(message, "content", "")).strip() + if not text: + continue + if msg_type == "human": + outgoing.append({"peer_id": user_peer, "content": text[: self._config.message_char_limit]}) + elif msg_type in ("ai", "AIMessageChunk"): + outgoing.append({"peer_id": assistant_peer, "content": text[: self._config.message_char_limit]}) + if not outgoing: + return + session_id = f"df-{_stable_id(thread_id)}" + try: + self._client.get_or_create_peer(workspace, user_peer) + self._client.get_or_create_peer(workspace, assistant_peer) + self._client.get_or_create_session(workspace, session_id) + self._client.set_session_peers(workspace, session_id, [user_peer, assistant_peer]) + self._client.add_messages(workspace, session_id, outgoing) + except MemoryManagerError: + raise + except Exception as exc: + logger.warning("honcho memory: write failed for thread %s: %s", thread_id, exc) + + # ── Tier 1: read ───────────────────────────────────────────────────── + def get_context( + self, + user_id: str | None, + *, + agent_name: str | None = None, + thread_id: str | None = None, + ) -> str: + workspace = self._workspace(user_id) + if workspace is None or not user_id: + return "" + representation = self._read_or_fallback("", lambda: self._client.working_representation(workspace, self._user_peer(user_id), max_conclusions=25)) + return representation.strip()[: self._config.max_injection_chars] + + # ── Tier 2 ─────────────────────────────────────────────────────────── + def search( + self, + query: str, + top_k: int = 5, + *, + user_id: str | None = None, + agent_name: str | None = None, + category: str | None = None, + ) -> list[dict[str, Any]]: + workspace = self._workspace(user_id) + if workspace is None: + return [] + results = self._read_or_fallback([], lambda: self._client.search(workspace, query, limit=top_k)) + return [ + { + "content": item.get("content", ""), + "category": category or "memory", + "session_id": item.get("session_id"), + "peer_id": item.get("peer_id"), + "created_at": item.get("created_at"), + } + for item in results + if isinstance(item, dict) + ][:top_k] + + def get_memory( + self, + *, + user_id: str | None = None, + agent_name: str | None = None, + ) -> dict[str, Any]: + """Minimal DeerMem-shape view: representation as the work-context summary. + + Honcho has no DeerMem-style fact CRUD; the gateway fills missing fields + with defaults (same contract as the noop backend's ``{"facts": []}``). + """ + empty = {"facts": [], "lastUpdated": _now_iso(), "user": {}, "history": {}} + workspace = self._workspace(user_id) + if workspace is None or not user_id: + return empty + representation = self._read_or_fallback(None, lambda: self._client.working_representation(workspace, self._user_peer(user_id), max_conclusions=25)) + if representation is None: # fail-open fallback: keep the noop-shaped doc + return empty + now = _now_iso() + return { + "facts": [], + "lastUpdated": now, + "user": {"workContext": {"summary": representation.strip()[: self._config.max_injection_chars], "updatedAt": now}}, + "history": {}, + } + + def shutdown_flush(self, timeout: float) -> bool: + """Writes are synchronous per-call; nothing is buffered locally.""" + return True + + def close(self) -> None: + """Release the HTTP client (gateway shutdown hook).""" + self._client.close() + + # ── async offload (blocking-io gate: never run httpx on the event loop) ── + async def aadd( + self, + thread_id: str, + messages: list[Any], + *, + agent_name: str | None = None, + user_id: str | None = None, + trace_id: str | None = None, + ) -> None: + await asyncio.to_thread(self.add, thread_id, messages, agent_name=agent_name, user_id=user_id, trace_id=trace_id) + + async def aget_context( + self, + user_id: str | None, + *, + agent_name: str | None = None, + thread_id: str | None = None, + ) -> str: + return await asyncio.to_thread(self.get_context, user_id, agent_name=agent_name, thread_id=thread_id) + + async def asearch( + self, + query: str, + top_k: int = 5, + *, + user_id: str | None = None, + agent_name: str | None = None, + category: str | None = None, + ) -> list[dict[str, Any]]: + return await asyncio.to_thread(self.search, query, top_k, user_id=user_id, agent_name=agent_name, category=category) diff --git a/backend/tests/blocking_io/test_honcho_memory_backend.py b/backend/tests/blocking_io/test_honcho_memory_backend.py new file mode 100644 index 000000000..97028f024 --- /dev/null +++ b/backend/tests/blocking_io/test_honcho_memory_backend.py @@ -0,0 +1,75 @@ +"""Regression anchors: Honcho async methods must not block the loop.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +import pytest +from langchain_core.messages import AIMessage, HumanMessage + +from deerflow.agents.memory.backends.honcho.honcho_manager import HonchoMemoryManager + + +class _BlockingFakeClient: + """Every method does real blocking file IO — trips Blockbuster if run on the loop.""" + + def __init__(self, probe_dir: Path): + self._probe = probe_dir / "probe.txt" + + def _block(self) -> None: + self._probe.write_text("blocked io") + + def get_or_create_peer(self, ws: str, pid: str) -> None: + self._block() + + def get_or_create_session(self, ws: str, sid: str) -> None: + self._block() + + def set_session_peers(self, ws: str, sid: str, pids: list[str]) -> None: + self._block() + + def add_messages(self, ws: str, sid: str, msgs: list[Any]) -> None: + self._block() + + def working_representation(self, ws: str, pid: str, *, max_conclusions: int = 25) -> str: + self._block() + return "rep" + + def search(self, ws: str, query: str, *, limit: int = 5) -> list[Any]: + self._block() + return [] + + def close(self) -> None: + pass + + +def _manager(tmp_path: Path) -> HonchoMemoryManager: + mgr = HonchoMemoryManager.from_config({"base_url": "http://honcho.test"}) + mgr._client = _BlockingFakeClient(tmp_path) + return mgr + + +@pytest.mark.asyncio +async def test_async_honcho_operations_do_not_block_event_loop( + tmp_path: Path, +) -> None: + manager = _manager(tmp_path) + messages: list[Any] = [ + HumanMessage("hello", id="h1"), + AIMessage("hi", id="a1"), + ] + + await manager.aadd( + "thread-1", + messages, + user_id="alice", + ) + assert await manager.aget_context("alice") is not None + assert ( + await manager.asearch( + "query", + user_id="alice", + ) + == [] + ) diff --git a/backend/tests/test_honcho_memory_backend.py b/backend/tests/test_honcho_memory_backend.py new file mode 100644 index 000000000..0d21b4089 --- /dev/null +++ b/backend/tests/test_honcho_memory_backend.py @@ -0,0 +1,479 @@ +"""Tests for the Honcho memory backend (config + manager).""" + +from __future__ import annotations + +import asyncio +from types import SimpleNamespace + +import httpx +import pytest + +from deerflow.agents.memory.backends.honcho.client import HonchoClient, HonchoRequestError +from deerflow.agents.memory.backends.honcho.config import HonchoConfig, sanitize_id +from deerflow.agents.memory.backends.honcho.honcho_manager import HonchoMemoryManager, _stable_id +from deerflow.agents.memory.manager import MemoryManagerError + + +class TestHonchoConfig: + def test_defaults(self): + cfg = HonchoConfig.from_backend_config(None) + assert cfg.base_url == "http://localhost:8000" + assert cfg.api_key is None + assert cfg.workspace_prefix == "deerflow-u-" + assert cfg.workspace_overrides == {} + assert cfg.user_peer_overrides == {} + assert cfg.assistant_peer == "deerflow" + assert cfg.message_char_limit == 8000 + assert cfg.max_injection_chars == 6000 + assert cfg.read_fail_closed is False + + def test_parses_knobs_and_ignores_unknown_keys(self): + cfg = HonchoConfig.from_backend_config( + { + "base_url": "https://api.honcho.dev/", + "api_key": "sk-test", + "workspace_prefix": "df-", + "workspace_overrides": {"user-1": "shared"}, + "user_peer_overrides": {"user-1": "alice"}, + "assistant_peer": "deer", + "failure_policy": {"read": "fail_closed"}, + "storage_path": "/tmp/x", + "unknown_key": True, + } + ) + assert cfg.base_url == "https://api.honcho.dev" # trailing slash stripped + assert cfg.workspace_overrides == {"user-1": "shared"} + assert cfg.user_peer_overrides == {"user-1": "alice"} + assert cfg.read_fail_closed is True + assert cfg.storage_path == "/tmp/x" + + def test_http_with_api_key_requires_opt_in(self): + with pytest.raises(ValueError, match="allow_insecure_http"): + HonchoConfig.from_backend_config({"base_url": "http://internal:8000", "api_key": "sk-x"}) + cfg = HonchoConfig.from_backend_config({"base_url": "http://internal:8000", "api_key": "sk-x", "allow_insecure_http": True}) + assert cfg.api_key == "sk-x" + + def test_http_without_api_key_is_fine(self): + cfg = HonchoConfig.from_backend_config({"base_url": "http://host.docker.internal:8000"}) + assert cfg.api_key is None + + def test_empty_override_values_rejected(self): + """An override entry with an empty/null value is a config mistake: silently + falling through to the default derivation (empty string is falsy) or + stringifying YAML null into a workspace literally named "None" would both + mask the operator's intent. Fail fast at parse time instead.""" + with pytest.raises(ValueError, match="workspace_overrides"): + HonchoConfig.from_backend_config({"workspace_overrides": {"alice": ""}}) + with pytest.raises(ValueError, match="workspace_overrides"): + HonchoConfig.from_backend_config({"workspace_overrides": {"alice": None}}) + with pytest.raises(ValueError, match="user_peer_overrides"): + HonchoConfig.from_backend_config({"user_peer_overrides": {"bob": " "}}) + + +class TestSanitizeId: + def test_passthrough_and_cleanup(self): + assert sanitize_id("user_1-ok") == "user_1-ok" + assert sanitize_id("weird id@example.com") == "weird-id-example-com" + assert len(sanitize_id("x" * 200)) == 64 + assert sanitize_id("") == "" + + +def _client_with_handler(handler, **cfg_over): + cfg = HonchoConfig.from_backend_config({"base_url": "http://honcho.test", **cfg_over}) + return HonchoClient(cfg, transport=httpx.MockTransport(handler)) + + +class TestHonchoClient: + def test_paths_and_payloads(self): + seen: list[tuple[str, str, bytes]] = [] + + def handler(request: httpx.Request) -> httpx.Response: + seen.append((request.method, request.url.path, request.content)) + if request.url.path.endswith("/representation"): + return httpx.Response(200, json={"representation": "knows things"}) + if request.url.path.endswith("/search"): + return httpx.Response(200, json=[{"content": "hit", "peer_id": "p", "session_id": "s", "created_at": "t"}]) + return httpx.Response(200, json={"id": "x"}) + + c = _client_with_handler(handler) + c.get_or_create_peer("ws1", "alice") + c.get_or_create_session("ws1", "df-t1") + c.set_session_peers("ws1", "df-t1", ["alice", "deerflow"]) + c.add_messages("ws1", "df-t1", [{"peer_id": "alice", "content": "hi"}]) + assert c.working_representation("ws1", "alice", max_conclusions=10) == "knows things" + assert c.search("ws1", "q", limit=5)[0]["content"] == "hit" + + paths = [p for _, p, _ in seen] + assert paths == [ + "/v3/workspaces/ws1/peers", + "/v3/workspaces/ws1/sessions", + "/v3/workspaces/ws1/sessions/df-t1/peers", + "/v3/workspaces/ws1/sessions/df-t1/messages", + "/v3/workspaces/ws1/peers/alice/representation", + "/v3/workspaces/ws1/search", + ] + assert b'"alice"' in seen[2][2] and b'"deerflow"' in seen[2][2] + assert b'"messages"' in seen[3][2] + + def test_errors_wrap_as_honcho_request_error(self): + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(500, text="boom") + + c = _client_with_handler(handler) + with pytest.raises(HonchoRequestError): + c.get_or_create_peer("ws1", "alice") + + def test_non_json_200_response_wraps_as_honcho_request_error(self): + """A 200 with a non-JSON body (e.g. a maintenance page from a proxy in + front of Honcho) must not let a bare JSONDecodeError escape -- it has + to surface through the same HonchoRequestError contract as any other + transport failure so callers have exactly one exception type to + handle.""" + + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(200, text="maintenance") + + c = _client_with_handler(handler) + with pytest.raises(HonchoRequestError): + c.get_or_create_peer("ws1", "alice") + + def test_api_key_header(self): + def handler(request: httpx.Request) -> httpx.Response: + assert request.headers.get("Authorization") == "Bearer sk-h" + return httpx.Response(200, json={"id": "x"}) + + c = _client_with_handler(handler, api_key="sk-h", allow_insecure_http=True) + c.get_or_create_peer("ws1", "alice") + + +class _FakeClient: + """Records calls; canned returns. Replaces the real HonchoClient in tests.""" + + def __init__(self): + self.calls: list[tuple[str, tuple]] = [] + self.representation_text = "ajayr likes concise answers" + self.raise_on: str | None = None + # Which exception type _maybe_raise raises; default matches the real + # HonchoClient's contract. Tests override this to a non-HonchoRequestError + # (e.g. RuntimeError) to prove the manager's boundary excepts are broad, + # not narrowly typed to the client's own exception class. + self.raise_exc_cls: type[BaseException] = HonchoRequestError + self.closed = False + + def _maybe_raise(self, name): + if self.raise_on == name: + raise self.raise_exc_cls(name) + + def get_or_create_peer(self, ws, pid): + self._maybe_raise("peer") + self.calls.append(("peer", (ws, pid))) + + def get_or_create_session(self, ws, sid): + self._maybe_raise("session") + self.calls.append(("session", (ws, sid))) + + def set_session_peers(self, ws, sid, pids): + self._maybe_raise("peers") + self.calls.append(("set_peers", (ws, sid, tuple(pids)))) + + def add_messages(self, ws, sid, msgs): + self._maybe_raise("messages") + self.calls.append(("messages", (ws, sid, tuple((m["peer_id"], m["content"]) for m in msgs)))) + + def working_representation(self, ws, pid, *, max_conclusions=25): + self._maybe_raise("representation") + self.calls.append(("rep", (ws, pid))) + return self.representation_text + + def search(self, ws, query, *, limit=5): + self._maybe_raise("search") + self.calls.append(("search", (ws, query, limit))) + return [{"content": "found", "peer_id": "deerflow", "session_id": "df-t", "created_at": "2026-01-01"}] + + def close(self): + self.closed = True + + +def _manager(**backend_config): + mgr = HonchoMemoryManager.from_config({"base_url": "http://honcho.test", **backend_config}) + fake = _FakeClient() + mgr._client = fake + return mgr, fake + + +def _msg(msg_type, content): + return SimpleNamespace(type=msg_type, content=content) + + +class TestHonchoClientTransport: + def test_constructor_accepts_transport_for_tests(self): + """Tests inject httpx.MockTransport through the constructor (Mem0Client + precedent) instead of rebuilding and overwriting client._http.""" + seen: list[str] = [] + + def handler(request): + seen.append(request.url.path) + return httpx.Response(200) + + cfg = HonchoConfig.from_backend_config({"base_url": "http://honcho.test"}) + client = HonchoClient(cfg, transport=httpx.MockTransport(handler)) + client.get_or_create_peer("w1", "p1") + assert seen == ["/v3/workspaces/w1/peers"] + + +class TestHonchoManagerWrite: + def test_add_maps_messages_to_peers(self): + mgr, fake = _manager(workspace_overrides={"u1": "shared"}, user_peer_overrides={"u1": "alice"}) + mgr.add("t-1", [_msg("human", "please check the deploy"), _msg("ai", "deploy is green"), _msg("tool", "ignored")], user_id="u1") + # Session ids share the user-id path's collision-resistant derivation; + # computed, not hardcoded (see test_add_prefix_workspace_for_unmapped_user). + sid = f"df-{_stable_id('t-1')}" + assert ("messages", ("shared", sid, (("alice", "please check the deploy"), ("deerflow", "deploy is green")))) in fake.calls + assert ("set_peers", ("shared", sid, ("alice", "deerflow"))) in fake.calls + + def test_add_without_user_is_noop(self): + mgr, fake = _manager() + mgr.add("t-1", [_msg("human", "hello")], user_id=None) + assert fake.calls == [] + + def test_add_with_empty_string_user_is_noop(self): + mgr, fake = _manager() + mgr.add("t-1", [_msg("human", "hello")], user_id="") + assert fake.calls == [] + + def test_add_prefix_workspace_for_unmapped_user(self): + mgr, fake = _manager() + mgr.add("t-2", [_msg("human", "x")], user_id="bob@example.com") + # Computed via the same _stable_id formula the manager uses, not + # hardcoded, so this test doesn't silently drift from the real + # collision-resistant derivation (see TestHonchoIdentityDerivation). + expected_peer = _stable_id("bob@example.com") + assert fake.calls[0] == ("peer", (f"deerflow-u-{expected_peer}", expected_peer)) + + def test_add_swallows_backend_errors(self): + mgr, fake = _manager() + fake.raise_on = "messages" + mgr.add("t-3", [_msg("human", "x")], user_id="u1") # must not raise + + def test_add_swallows_non_honcho_exceptions(self): + """The boundary except is broad (except Exception), not narrowly typed + to HonchoRequestError -- any client failure (e.g. a bug surfacing as a + bare RuntimeError, or the non-JSON-response case in TestHonchoClient) + must not escape add() and crash MemoryMiddleware.after_agent.""" + mgr, fake = _manager() + fake.raise_on = "messages" + fake.raise_exc_cls = RuntimeError + mgr.add("t-6", [_msg("human", "x")], user_id="u1") # must not raise + + def test_add_truncates_long_content(self): + mgr, fake = _manager(message_char_limit=10) + mgr.add("t-4", [_msg("human", "0123456789ABCDEF")], user_id="u1") + sent = [c for c in fake.calls if c[0] == "messages"][0][1][2][0][1] + assert len(sent) == 10 + + def test_add_normalizes_list_content(self): + mgr, fake = _manager() + mgr.add("t-5", [_msg("human", [{"type": "text", "text": "part1"}, {"type": "text", "text": "part2"}])], user_id="u1") + sent = [c for c in fake.calls if c[0] == "messages"][0][1][2][0][1] + assert "part1" in sent and "part2" in sent + + def test_session_ids_resist_sanitize_collisions(self): + """Same lossy-sanitization hazard as user ids, one tier down: thread ids + "t.1" and "t-1" both sanitize to "t-1", so bare sanitize_id would merge + two threads' histories into one Honcho session. Session ids must use the + same collision-resistant _stable_id derivation as the user-id path.""" + mgr, fake = _manager() + mgr.add("t.1", [_msg("human", "x")], user_id="u1") + mgr.add("t-1", [_msg("human", "y")], user_id="u1") + session_ids = {c[1][1] for c in fake.calls if c[0] == "session"} + assert len(session_ids) == 2 + + +class TestHonchoManagerRead: + def test_get_context_returns_representation(self): + mgr, fake = _manager(workspace_overrides={"u1": "shared"}) + assert "concise answers" in mgr.get_context("u1") + + def test_get_context_without_user_is_empty(self): + mgr, _ = _manager() + assert mgr.get_context(None) == "" + + def test_get_context_with_empty_string_user_is_empty(self): + mgr, fake = _manager() + assert mgr.get_context("") == "" + assert fake.calls == [] + + def test_get_context_truncates(self): + mgr, fake = _manager(max_injection_chars=10) + fake.representation_text = "y" * 100 + assert len(mgr.get_context("u1")) <= 10 + + def test_get_context_fail_open_by_default(self): + mgr, fake = _manager() + fake.raise_on = "representation" + assert mgr.get_context("u1") == "" + + def test_get_context_fail_closed_raises_contract_error(self): + mgr, fake = _manager(failure_policy={"read": "fail_closed"}) + fake.raise_on = "representation" + with pytest.raises(MemoryManagerError): + mgr.get_context("u1") + + def test_get_context_fail_open_swallows_non_honcho_exceptions(self): + mgr, fake = _manager() + fake.raise_on = "representation" + fake.raise_exc_cls = RuntimeError + assert mgr.get_context("u1") == "" + + def test_get_context_fail_closed_wraps_non_honcho_exceptions(self): + mgr, fake = _manager(failure_policy={"read": "fail_closed"}) + fake.raise_on = "representation" + fake.raise_exc_cls = RuntimeError + with pytest.raises(MemoryManagerError): + mgr.get_context("u1") + + def test_search_maps_results(self): + mgr, _ = _manager() + results = mgr.search("deploy", top_k=3, user_id="u1") + assert results[0]["content"] == "found" + + def test_search_without_user_is_empty(self): + mgr, _ = _manager() + assert mgr.search("q", user_id=None) == [] + + def test_supports_search_flag_matches_override(self): + mgr, _ = _manager() + assert type(mgr).supports_search is True + + def test_search_fail_open_by_default(self): + mgr, fake = _manager() + fake.raise_on = "search" + assert mgr.search("q", user_id="u1") == [] + + def test_search_fail_closed_raises_contract_error(self): + """search() is a recall op (the tool-mode memory_search path): it must + honor failure_policy.read like get_context() does, not silently return + [] and mask an outage from the model and the operator.""" + mgr, fake = _manager(failure_policy={"read": "fail_closed"}) + fake.raise_on = "search" + with pytest.raises(MemoryManagerError): + mgr.search("q", user_id="u1") + + def test_search_fail_closed_wraps_non_honcho_exceptions(self): + mgr, fake = _manager(failure_policy={"read": "fail_closed"}) + fake.raise_on = "search" + fake.raise_exc_cls = RuntimeError + with pytest.raises(MemoryManagerError): + mgr.search("q", user_id="u1") + + def test_get_memory_minimal_shape(self): + mgr, _ = _manager() + doc = mgr.get_memory(user_id="u1") + assert doc["facts"] == [] + assert doc["user"]["workContext"]["summary"] + assert doc["lastUpdated"] + + def test_get_memory_without_user_is_empty_with_no_calls(self): + mgr, fake = _manager() + doc = mgr.get_memory(user_id=None) + assert doc["facts"] == [] + assert doc["user"] == {} + assert doc["history"] == {} + assert doc["lastUpdated"] + assert fake.calls == [] + + def test_get_memory_fail_open_by_default(self): + mgr, fake = _manager() + fake.raise_on = "representation" + doc = mgr.get_memory(user_id="u1") + assert doc["facts"] == [] + assert doc["user"] == {} + + def test_get_memory_fail_closed_raises_contract_error(self): + """get_memory() backs the /memory gateway endpoint — a recall op, so it + follows failure_policy.read like get_context() and search().""" + mgr, fake = _manager(failure_policy={"read": "fail_closed"}) + fake.raise_on = "representation" + with pytest.raises(MemoryManagerError): + mgr.get_memory(user_id="u1") + + +class TestHonchoManagerAsync: + def test_async_entrypoints_delegate(self): + mgr, fake = _manager() + + async def run(): + await mgr.aadd("t-1", [_msg("human", "x")], user_id="u1") + ctx = await mgr.aget_context("u1") + hits = await mgr.asearch("q", user_id="u1") + return ctx, hits + + ctx, hits = asyncio.run(run()) + assert "concise" in ctx and hits + + +class TestHonchoManagerLifecycle: + def test_shutdown_flush_true(self): + mgr, _ = _manager() + assert mgr.shutdown_flush(1.0) is True + + def test_tool_mode_accepted(self): + mgr = HonchoMemoryManager.from_config({"base_url": "http://honcho.test"}, mode="tool") + assert mgr.mode == "tool" + + def test_requires_passive_writes_in_tool_mode(self): + """Honcho's only write path is passive add(); fact CRUD is intentionally + unsupported, so tool mode must keep MemoryMiddleware writes flowing to + Honcho's deriver alongside model-directed search(). Mirrors mem0's + identical ClassVar (mem0_manager.py).""" + assert HonchoMemoryManager.requires_passive_writes_in_tool_mode is True + + def test_close_releases_http_client(self): + """close() is the gateway shutdown hook (manager.py base default is a + no-op); Honcho must override it to release the underlying HTTP client, + mirroring mem0_manager.py's identical close().""" + mgr, fake = _manager() + mgr.close() + assert fake.closed is True + + +class TestHonchoIdentityDerivation: + """Pins the collision-resistant default (non-override) identity derivation. + + ``sanitize_id`` alone is lossy: distinct raw ids can sanitize to the same + string, which would merge two different users' memory into one Honcho + workspace/peer if used bare. ``_stable_id`` (workspace/peer default path) + must keep such inputs apart. + """ + + def test_default_workspace_and_peer_resist_sanitize_collisions(self): + mgr, _ = _manager() + raw_a = "user.name@example.com" + raw_b = "user-name@example.com" + # Precondition: these two distinct raw ids really do collide under + # plain sanitize_id -- otherwise this test would not exercise the bug + # the collision-resistant derivation fixes. + assert sanitize_id(raw_a) == sanitize_id(raw_b) + + ws_a, ws_b = mgr._workspace(raw_a), mgr._workspace(raw_b) + peer_a, peer_b = mgr._user_peer(raw_a), mgr._user_peer(raw_b) + assert ws_a != ws_b + assert peer_a != peer_b + + def test_user_peer_never_empty_for_degenerate_raw_id(self): + mgr, _ = _manager() + # "!!!" sanitizes to "" (every character stripped); the default + # derivation must still produce a non-empty, usable peer/workspace id. + assert sanitize_id("!!!") == "" + peer = mgr._user_peer("!!!") + assert peer != "" + ws = mgr._workspace("!!!") + assert ws is not None + assert ws != mgr._config.workspace_prefix + + +class TestFactoryDiscovery: + def test_manager_class_resolves(self): + from deerflow.agents.memory.backends.honcho import MANAGER_CLASS + + assert MANAGER_CLASS is HonchoMemoryManager diff --git a/config.example.yaml b/config.example.yaml index 53e2e8d13..7d8a0105f 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -1727,6 +1727,18 @@ memory: # For a host-installed OpenViking used by Docker DeerFlow, set base_url to # http://host.docker.internal:1933 and allow_insecure_http: true. The bundled # optional Compose overlay uses the internal http://openviking:1933 address. + # + # Honcho example (replace this DeerMem backend_config block when + # manager_class is honcho). This remote HTTP adapter uses Honcho's server-side + # deriver to build user-model memory representations; no local LLM calls. + # + # backend_config: + # base_url: http://localhost:8000 + # # api_key: $HONCHO_API_KEY # hosted Honcho; plain-http + api_key needs allow_insecure_http: true + # workspace_prefix: deerflow-u- # one isolated workspace per user id + # # workspace_overrides: {} # map specific user ids to custom workspaces + # # user_peer_overrides: {} # map specific user ids to custom peer names + # assistant_peer: deerflow backend_config: storage_path: "" # empty = deer-flow base_dir (factory injects absolute runtime_home); a non-empty path is the root DIRECTORY (per-user memory under {storage_path}/users/{uid}/memory.json) storage_class: file # file (default) or a dotted MemoryStorage class path; invalid persistent backends fail fast