diff --git a/README.md b/README.md index b19aaa34b..0bd6f8d29 100644 --- a/README.md +++ b/README.md @@ -1247,6 +1247,8 @@ Memory updates now skip duplicate fact entries at apply time, so repeated prefer In the default DeerMem `middleware` mode, automatic extraction now classifies every proposed fact by scope, durability, and authority before a deterministic write gate accepts it. Only durable, descriptive user-level facts are stored; current-thread or project constraints and one-time action permissions stay in conversation state. User-global summaries require both user scope and descriptive authority, contradiction removals are scope-gated, and a replacement-dependent removal is applied only when its replacement actually survives validation and storage. These classification labels are extraction-only metadata, add no extra LLM call, and are not written into the fact files. The explicit CRUD tools in `memory.mode: tool` remain a separate, model-directed path. Deployments that override the bundled DeerMem prompts via `memory.backend_config.prompts_dir` must add the new classification fields to their custom templates (the `memory_update` fact/summary/removal formats and the `consolidation` consolidated-fact schema): the write gate fails closed, so an un-migrated template stops every extraction-driven fact, summary, and removal write, surfacing only through the `rejected_by_scope_gate` metrics and the high-rejection-rate warning. +When a fact scope reaches `max_facts`, DeerMem still uses the historical confidence-only eviction order by default. Operators can opt in to `memory.backend_config.fact_eviction_policy: hybrid-v1`, which combines bounded confidence (65%), explicit-confirmation freshness (25%), and query-driven access heat (10%). Hybrid signal metadata is collected only while hybrid-v1 or shadow mode is enabled. Explicit confirmation is returned as `factsToReinforce` by the existing memory-update LLM call and is accepted only when deterministic message processing also detects a user reinforcement signal; it also resets the fact's staleness-review clock. This deterministic gate is batch-level: it establishes only that a human message among the last six filtered messages in the current extraction batch matched a reinforcement pattern. The LLM-selected `factsToReinforce` ID supplies the fact binding; DeerMem does not independently verify a signal-to-fact correspondence. Repeated extraction or automatic injection never confirms a fact. Custom `memory_update` prompts should add the optional `factsToReinforce` array to participate in confirmation freshness. Access heat is stored in a separate decaying sidecar and increases only when `memory_search` actually returns the fact, so reads do not rewrite canonical Markdown or its `updatedAt`. Hybrid mode also reserves a bounded minimum of correction slots (10% of the cap, at most 10; unused slots return to normal competition). Capacity deletion remains physical, but a bounded metadata-only audit records fact IDs, categories, policy scores, and reasons without copying fact content. `fact_eviction_shadow_enabled: true` evaluates hybrid-v1 alongside the default policy without changing actual retention. This feature adds no LLM invocation and can be rolled back by selecting `confidence`. + File-backed memory now separates global user context from agent facts. Each user has one `memory.json` containing only the project-independent `user` and `history` summaries; every fact is a canonical Markdown file below `agents/{agent_name}/facts/`. Existing lead-agent middleware, API, Settings, import/export, and embedded-client calls that omit `agent_name` resolve inside DeerMem to the reserved `__default__` bucket. That bucket is outside the valid custom-agent name grammar, so a real custom agent named `lead-agent` has a separate fact repository and deleting a custom agent cannot delete a memory-only directory without `config.yaml`. Public agent identifiers are case-insensitive and canonicalized to lowercase. Runtime/API readers still receive a compatibility `facts` array for the selected/default agent, so the frontend does not read agent facts from `memory.json`; structured Markdown `source` metadata is projected to the historical string field at the MemoryManager boundary. An unscoped Clear All first migrates facts from unread legacy per-agent JSON without adopting its soon-to-be-cleared summaries, then removes shared summaries and facts from every agent bucket while preserving agent configuration files, so a later read cannot resurrect skipped legacy facts; an explicitly agent-scoped clear removes only that agent's facts. On first normal read, old facts embedded in the user JSON are migrated automatically to `__default__`; facts written to the earlier implicit `lead-agent` bucket are also moved when that directory is not a real custom agent. Migration and normal writes notify the configured retrieval adapter only after durable storage locks are released. DeerMem uses a scope-aware SQLite FTS5/BM25 adapter by default, stores only rebuildable derived index data under `.retrieval/`, and rebuilds it in the background during Gateway startup or lazily on the first scoped search. A corrupt derived index is recreated automatically. Set `memory.backend_config.retrieval_adapter` to an empty string to disable it and use the local substring fallback. Chinese tokenization is optional; install the backend `memory-zh` extra (`uv sync --extra memory-zh`) for jieba-assisted sub-phrase search. Journaled writes, a shared user lock, and optimistic user-memory revisions prevent silent lost updates. Memory injection follows the configured operation mode. In `middleware` mode, DeerMem injects the user-global summaries and the selected agent's facts. Custom-agent bootstrap conversations use that agent's fact bucket as well, so setup details do not leak into the default agent's memory. In `tool` mode, the automatic `` block contains only the global `user` and `history` summaries; agent facts are retrieved explicitly through `memory_search`, avoiding duplicate automatic and tool-returned fact context. Setting `memory.injection_enabled: false` still disables the entire block in either mode. diff --git a/README_zh.md b/README_zh.md index 566c56ccd..b80f832ac 100644 --- a/README_zh.md +++ b/README_zh.md @@ -676,6 +676,8 @@ DeerFlow 不只是“会说它能做”，它是真的有一台自己的“电 默认 DeerMem `middleware` 模式会先判断候选信息的作用域、持久性和授权属性，再由确定性写入门决定是否保存。只有稳定、描述性的用户级事实能进入长期 memory；当前对话或项目的约束、一次性操作授权仍留在对话状态中。用户全局 summary 必须同时具有用户级作用域和描述性授权属性，基于矛盾的删除也会经过作用域保护；如果删除依赖一条替代事实，只有替代事实真正通过校验并保留下来后才执行删除。这些分类字段只用于本次抽取，不写入 fact 文件，也不增加 LLM 调用次数。`memory.mode: tool` 的显式 CRUD 仍是独立的模型直写路径。如果通过 `memory.backend_config.prompts_dir` 覆盖了内置抽取模板，必须同步在自定义模板中加入新的分类字段（`memory_update` 的 fact/summary/removal 格式与 `consolidation` 的合并 fact 结构）：写入门是 fail closed 的，未迁移的旧模板会导致所有抽取驱动的 fact、summary 与删除写入停止，只能通过 `rejected_by_scope_gate` 指标和高拒绝率告警发现。 +当一个作用域的 fact 达到 `max_facts` 时，DeerMem 默认仍沿用仅按 confidence 排序的旧策略。可以显式设置 `memory.backend_config.fact_eviction_policy: hybrid-v1`，改用有界综合分：confidence 65%、用户明确确认的新鲜度 25%、查询召回热度 10%。只有开启 hybrid-v1 或 shadow 模式时才会收集这两类信号元数据。确认由已有的 memory-update LLM 调用返回 `factsToReinforce`，但只有确定性消息检测也发现用户 reinforcement 信号时才会更新，并同时重置该 fact 的 staleness review 时钟。这个确定性门禁是批次级的：它只能证明当前抽取批次最后六条已过滤消息中的某条用户消息命中了 reinforcement 模式。具体 fact 由 LLM 选择的 `factsToReinforce` ID 绑定；DeerMem 不会另外校验该信号与 fact 的一一对应关系。重复抽取、自动注入和单纯召回都不会确认 fact。自定义 `memory_update` prompt 如果希望参与确认新鲜度，需要加入可选的 `factsToReinforce` 数组。召回热度单独保存在衰减 sidecar 中，只有 `memory_search` 真正返回的 fact 才增加，不会重写 canonical Markdown 或污染 `updatedAt`。Hybrid 模式还为 correction 保留有限的最低槽位（容量的 10%，最多 10 个；未使用的槽位会释放给其他类别）。容量删除仍是物理删除，但会留下不含正文的有界审计记录。启用 `fact_eviction_shadow_enabled` 可以在不改变实际保留结果的情况下比较 hybrid-v1；整个功能不增加 LLM 调用，切回 `confidence` 即可回滚。 + ## 推荐模型 DeerFlow 对模型没有强绑定，只要实现了 OpenAI 兼容 API 的 LLM，理论上都可以接入。不过在下面这些能力上表现更强的模型，通常会更适合 DeerFlow： diff --git a/backend/app/gateway/routers/memory.py b/backend/app/gateway/routers/memory.py index 2046ee546..b69c12573 100644 --- a/backend/app/gateway/routers/memory.py +++ b/backend/app/gateway/routers/memory.py @@ -330,8 +330,8 @@ async def create_memory_fact_endpoint(request: FactCreateRequest, http_request: raise HTTPException(status_code=500, detail="Failed to create memory fact.") from exc if fact_id is None: - # max_facts cap evicted the new (lower-confidence) fact; it was not stored. - raise HTTPException(status_code=409, detail="Fact was not stored because memory.max_facts kept higher-confidence facts") + # The configured max_facts policy evicted the new fact; it was not stored. + raise HTTPException(status_code=409, detail="Fact was not stored because the configured memory.max_facts capacity policy evicted it") return MemoryResponse(**memory_data) diff --git a/backend/packages/harness/deerflow/agents/memory/AGENTS.md b/backend/packages/harness/deerflow/agents/memory/AGENTS.md index 37e513aa4..d014508b5 100644 --- a/backend/packages/harness/deerflow/agents/memory/AGENTS.md +++ b/backend/packages/harness/deerflow/agents/memory/AGENTS.md @@ -56,6 +56,7 @@ - `memory.mode: tool` skips `MemoryMiddleware` and registers `memory_search`, `memory_add`, `memory_update`, and `memory_delete` on the agent. The model decides when to search, add, update, or delete facts; this is opt-in/experimental and should not be described as better than middleware mode without eval evidence. - Both modes share `FileMemoryStorage`, per-user/per-agent isolation, manual CRUD primitives, and the updater backend. Injection is mode-aware: middleware mode injects global `user`/`history` summaries plus the selected agent's facts, while tool mode injects only the global summaries and leaves every agent fact behind `memory_search` to avoid duplicating automatically injected and retrieval-returned context. `memory.injection_enabled: false` suppresses the complete block in either mode. - Middleware extraction classifies proposed facts with extraction-only `scope`/`durability`/`authority` labels. `_apply_updates` accepts only `user` + `durable` + `descriptive` new/consolidated facts, accepts only wholly user-scoped summary prose with `authority=descriptive`, and rejects missing labels per item without aborting unrelated updates. Contradiction removals use object entries with `id`, `scope`, `reason`, and optional zero-based `replacementFactIndex`; task/project removals fail closed, and a paired removal runs only when the referenced replacement survives the scope/confidence gates, deduplication, and max-fact trim under another fact ID. The labels are not persisted, so no storage migration is required. Staleness removals retain their independent candidate/cap guardrails, while tool-mode CRUD remains outside this extraction gate. Custom `memory.backend_config.prompts_dir` templates (including per-agent overrides) must carry the same classification fields; an un-migrated template makes the fail-closed gate reject every extraction-driven write, observable only through `rejected_by_scope_gate` and the >60% fact-rejection warning. +- Capacity eviction is centralized in `deermem/core/eviction.py` for automatic extraction, manual/tool fact creation, and import. `confidence` remains the default policy. Opt-in `hybrid-v1` uses bounded 0.65 confidence + 0.25 explicit-confirmation freshness + 0.10 query-access heat, with configurable half-lives and a bounded minimum correction reserve. Confirmation/access metadata is collected only when hybrid-v1 or shadow mode is active. The existing update LLM may return `factsToReinforce`, but `_apply_updates` updates `lastConfirmedAt`/`confirmationCount` only when deterministic message processing also detected `reinforcement`; a valid `lastConfirmedAt` also resets the staleness-review clock. That deterministic gate is batch-level: it matches a human message among the last six filtered messages in the current extraction batch, while the LLM-provided ID supplies fact binding without an independent signal-to-fact correspondence check. Duplicate extraction, prompt injection, and search alone never confirm. Only facts actually returned by `DeerMem.search()` increment the decaying usage sidecar; `get_context()` never does, and confidence-only capacity selection does not read the usage sidecar. Sidecars live under the agent `.metadata/` directory so usage does not mutate canonical Markdown timestamps/revisions. Capacity audits are bounded and metadata-only, are written only after canonical persistence succeeds, and user delete/clear removes matching usage/audit data. Shadow mode computes hybrid disagreement while continuing to execute confidence-only. - Middleware mode queue debounces (30s default), batches updates, and commits global summaries plus the selected/default agent's fact delta through a user-level lock, optimistic user-memory revisions, per-fact revisions, and a recoverable target-file journal. Only explicitly marked point operations may rebase a stale shared revision, and only while every addressed fact still satisfies its original absent/revision precondition. Snapshot-derived clear/trim/consolidation operations instead reload the complete document and recompute their intent on a manifest conflict, with a bounded retry. Typed manifest/fact conflict subclasses keep that decision independent of exception text, and same-ID creates and stale same-fact writes fail. Scope-lock objects are weakly cached so inactive users do not grow a process-lifetime map. Cache validation does not scale with the fact-file count: its token combines the shared JSON's `(mtime_ns, size, revision)`, so the persisted revision invalidates stale caches even when a coarse-mtime filesystem reports identical metadata for same-size writes; direct out-of-band Markdown edits require `reload()`. Atomic replacement also syncs the parent directory on POSIX so the rename is durable. DeerMem translates private storage conflict/corruption exceptions to the backend-neutral MemoryManager contract; the Gateway maps them to HTTP 409 and a stable HTTP 500 response respectively. A normal default-manager read automatically migrates legacy facts from the global JSON into `__default__`; it also adopts the earlier implicit `lead-agent` fact bucket only when that directory has no custom-agent `config.yaml`, and rejects unexpected files instead of deleting them. The v1-to-v2 migration is one-way for the running application: operators must stop DeerFlow and snapshot the configured storage root before upgrade. Before any destructive v2 write, every migrated JSON source is durably retained as `{manifest_filename}.v1.bak`; a missing-write or mismatched existing backup aborts without modifying v1 data. Legacy per-agent JSON is deleted only after its non-empty summaries are safely adopted or confirmed identical; summary conflicts keep the source file and fail loudly. - **Proactive Markdown migration CLI**: from `backend/`, run `PYTHONPATH=. python scripts/migrate_memory_markdown.py --all-users --dry-run` to audit and omit `--dry-run` to migrate before serving traffic. Use repeated `--user-id` values when selecting exact original identities, especially standalone raw IDs containing `@` or other characters that are normalized in directory names; `--storage-path` selects a non-default DeerMem root. The CLI reuses `FileMemoryStorage.migrate`, is idempotent, continues across per-user failures, and exits non-zero if any user fails. It is optional because the first normal read still performs the same migration automatically. - `retrieval_adapter` owns indexing and retrieval. `fts5` is the DeerMem default and uses a persistent derived SQLite index under `.retrieval/`; an empty value disables the adapter and selects `substring_fallback`. File storage sends upsert/remove notifications for normal writes and both explicit and lazy migrations after releasing durable storage locks, then delegates search. Gateway startup schedules `DeerMem.warm_retrieval()` as a background full rebuild so readiness is not delayed, while a first search lazily rebuilds its exact scope until warm-up completes. Individual malformed facts are logged and skipped without triggering repeated full scans; only a fatal adapter rebuild failure keeps lazy retry enabled. During shutdown, the Gateway waits at most one second for this derived rebuild and leaves the full configured timeout to the canonical memory flush; if the rebuild is still active, its adapter remains open until process exit. Adapter failures mark the scope dirty and fall back to canonical substring search until rebuilding succeeds. `FileMemoryStorage` owns and closes the adapter so higher layers do not reach into private storage state. @@ -89,6 +90,11 @@ Focused regression coverage for the updater lives in `backend/tests/test_memory_ - `shutdown_flush_timeout_seconds` - Hard budget (seconds) reserved for draining the memory backend's pending-update buffer on Gateway graceful shutdown (default: 30; 1–300). Each pending item does one LLM call, so large IM batches may need more. The Gateway lifespan calls `MemoryManager.shutdown_flush(timeout)` after channels/scheduler stop and after waiting at most one additional second for the derived retrieval warm-up; the backend short-circuits on an idle buffer, so the host calls it unconditionally (no pending/processing gate). The retrieval wait does not reduce this canonical flush budget. The combined shutdown hooks, brief retrieval wait, flush budget, and scheduling margin must fit inside the pod's K8s `terminationGracePeriodSeconds` (gateway Helm chart default: 45s) or K8s SIGKILLs the drain mid-flight. - `model_name` - LLM for updates (null = default model) - `max_facts` / `fact_confidence_threshold` - Fact storage limits (100 / 0.7) +- `fact_eviction_policy` / `fact_eviction_shadow_enabled` - Capacity policy (`confidence` default; opt-in `hybrid-v1`) and non-enforcing hybrid comparison audit +- `eviction_confidence_weight` / `eviction_confirmation_weight` / `eviction_access_weight` - Hybrid weights (0.65 / 0.25 / 0.10; must sum to 1.0) +- `eviction_confirmation_half_life_days` / `eviction_access_half_life_days` - Confirmation and query-heat decay windows (90 / 30 days) +- `eviction_correction_reserved_fraction` / `eviction_correction_reserved_max` - Bounded minimum correction capacity (0.10 / 10; unused slots are released) +- `eviction_audit_max_entries` - Metadata-only capacity audit bound per user/agent scope (200; 0 disables) - `max_injection_tokens` - Token limit for prompt injection (2000) - `token_counting` - Token counting strategy for the injection budget: `tiktoken` (default, accurate but may download BPE data from a public endpoint on first use — can block for a long time in network-restricted environments, see issues #3402/#3429) or `char` (network-free CJK-aware char estimate, never touches tiktoken) - `staleness_review_enabled` - Enable proactive staleness pruning of aged facts (default: `true`; only triggers when aged candidates exist) diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deer_mem.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deer_mem.py index 0f84a2f87..498220425 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deer_mem.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deer_mem.py @@ -32,6 +32,7 @@ from pydantic import PrivateAttr from deerflow.agents.memory.manager import MemoryConflictError, MemoryCorruptionError, MemoryManager from .deermem.config import DeerMemConfig +from .deermem.core.eviction import EVICTION_POLICY_HYBRID_V1 from .deermem.core.llm import build_llm from .deermem.core.message_processing import ( SIGNAL_NAMES, @@ -340,9 +341,25 @@ class DeerMem(MemoryManager): return [] resolved_agent_name = _resolve_agent_name(agent_name) indexed = self._fts5_search(query, top_k=top_k, user_id=user_id, agent_name=resolved_agent_name, category=category) - if indexed: - return indexed - return self._substring_search(query, top_k=top_k, user_id=user_id, agent_name=resolved_agent_name, category=category) + results = indexed or self._substring_search( + query, + top_k=top_k, + user_id=user_id, + agent_name=resolved_agent_name, + category=category, + ) + if results and (self._config.fact_eviction_policy == EVICTION_POLICY_HYBRID_V1 or self._config.fact_eviction_shadow_enabled): + try: + self._storage.record_fact_accesses( + [str(fact["id"]) for fact in results if fact.get("id")], + agent_name=resolved_agent_name, + user_id=user_id, + ) + except Exception: + # Usage is an eviction hint, never canonical memory. A sidecar + # write failure must not make memory_search lose its results. + logger.warning("Failed to record memory-search access heat", exc_info=True) + return results def _fts5_search( self, diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/config.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/config.py index 891ce1844..5ae6bdc37 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/config.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/config.py @@ -18,6 +18,7 @@ DeerMem default); tracing is via the base ``MemoryManager.callbacks`` field from __future__ import annotations import logging +import math from pathlib import Path from typing import Any, Literal @@ -86,6 +87,27 @@ class DeerMemConfig(BaseModel): ) # ── Facts ──────────────────────────────────────────────────────────── max_facts: int = Field(default=100, ge=10, le=500, description="Maximum number of facts to store.") + fact_eviction_policy: Literal["confidence", "hybrid-v1"] = Field( + default="confidence", + description=("Capacity-eviction policy. 'confidence' preserves the historical behavior; 'hybrid-v1' combines confidence, explicit-confirmation freshness, and query-driven access heat with bounded correction slots."), + ) + fact_eviction_shadow_enabled: bool = Field( + default=False, + description=("When true, also compute hybrid-v1 during confidence-policy trims and include its disagreement in the metadata-only eviction audit."), + ) + eviction_confidence_weight: float = Field(default=0.65, ge=0.0, le=1.0) + eviction_confirmation_weight: float = Field(default=0.25, ge=0.0, le=1.0) + eviction_access_weight: float = Field(default=0.10, ge=0.0, le=1.0) + eviction_confirmation_half_life_days: int = Field(default=90, ge=1, le=3650) + eviction_access_half_life_days: int = Field(default=30, ge=1, le=3650) + eviction_correction_reserved_fraction: float = Field(default=0.10, ge=0.0, le=1.0) + eviction_correction_reserved_max: int = Field(default=10, ge=0, le=100) + eviction_audit_max_entries: int = Field( + default=200, + ge=0, + le=10000, + description="Maximum metadata-only capacity-eviction audit events per user/agent scope; 0 disables the audit.", + ) fact_confidence_threshold: float = Field( default=0.7, ge=0.0, @@ -298,6 +320,9 @@ class DeerMemConfig(BaseModel): f"storage_path as a root DIRECTORY (per-user memory under " f"{{storage_path}}/users/{{uid}}/memory.json). Point it at a directory." ) + weight_sum = self.eviction_confidence_weight + self.eviction_confirmation_weight + self.eviction_access_weight + if not math.isclose(weight_sum, 1.0, rel_tol=0.0, abs_tol=1e-9): + raise ValueError("DeerMem eviction weights must sum to 1.0") return self @classmethod diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/eviction.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/eviction.py new file mode 100644 index 000000000..1f2e69c41 --- /dev/null +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/eviction.py @@ -0,0 +1,246 @@ +"""Deterministic, explainable capacity policies for canonical memory facts.""" + +from __future__ import annotations + +import math +from dataclasses import dataclass +from datetime import UTC, datetime +from typing import Any + +EVICTION_POLICY_CONFIDENCE = "confidence" +EVICTION_POLICY_HYBRID_V1 = "hybrid-v1" + + +@dataclass(frozen=True) +class FactEvictionScore: + """One fact's bounded policy score and its explainable components.""" + + value: float + components: dict[str, float] + + +@dataclass(frozen=True) +class EvictedFact: + """Metadata-only record for a fact removed by the capacity limit.""" + + fact_id: str + category: str + score: float + components: dict[str, float] + + +@dataclass(frozen=True) +class FactEvictionDecision: + """Complete result of applying a capacity policy to one snapshot.""" + + kept: list[dict[str, Any]] + evicted: list[EvictedFact] + scores: dict[str, FactEvictionScore] + policy: str + reserved_correction_slots: int = 0 + + +def _bounded_number(value: Any, *, default: float = 0.0) -> float: + if value is None or isinstance(value, bool): + return default + try: + number = float(value) + except (TypeError, ValueError): + return default + if not math.isfinite(number): + return default + return max(0.0, min(number, 1.0)) + + +def _parse_datetime(value: Any) -> datetime | None: + if not isinstance(value, str) or not value.strip(): + return None + try: + parsed = datetime.fromisoformat(value.strip().replace("Z", "+00:00")) + except ValueError: + return None + if parsed.tzinfo is None: + parsed = parsed.replace(tzinfo=UTC) + return parsed.astimezone(UTC) + + +def _decay(*, elapsed_days: float, half_life_days: float) -> float: + if half_life_days <= 0: + return 0.0 + return 2 ** (-max(0.0, elapsed_days) / half_life_days) + + +def _confirmation_freshness( + fact: dict[str, Any], + *, + now: datetime, + half_life_days: float, +) -> float: + confirmed_at = _parse_datetime(fact.get("lastConfirmedAt")) + if confirmed_at is not None: + elapsed = (now - confirmed_at).total_seconds() / 86400 + return _decay(elapsed_days=elapsed, half_life_days=half_life_days) + + created_at = _parse_datetime(fact.get("createdAt")) + if created_at is None: + return 0.0 + elapsed = (now - created_at).total_seconds() / 86400 + # Creation is weaker evidence than an explicit user confirmation. + return 0.5 * _decay(elapsed_days=elapsed, half_life_days=half_life_days) + + +def _normalized_access_heat( + usage: dict[str, Any] | None, + *, + now: datetime, + half_life_days: float, +) -> float: + if not isinstance(usage, dict): + return 0.0 + last_accessed_at = _parse_datetime(usage.get("lastAccessedAt")) + raw_heat = usage.get("accessHeat") + if last_accessed_at is None or raw_heat is None or isinstance(raw_heat, bool): + return 0.0 + try: + heat = float(raw_heat) + except (TypeError, ValueError): + return 0.0 + if not math.isfinite(heat) or heat <= 0: + return 0.0 + elapsed = (now - last_accessed_at).total_seconds() / 86400 + decayed_heat = heat * _decay(elapsed_days=elapsed, half_life_days=half_life_days) + return min(1.0, math.log1p(decayed_heat) / math.log(9)) + + +def _score_fact( + fact: dict[str, Any], + *, + policy: str, + usage: dict[str, Any] | None, + now: datetime, + confidence_weight: float, + confirmation_weight: float, + access_weight: float, + confirmation_half_life_days: float, + access_half_life_days: float, +) -> FactEvictionScore: + confidence = _bounded_number(fact.get("confidence"), default=0.5) + if policy == EVICTION_POLICY_CONFIDENCE: + return FactEvictionScore( + value=confidence, + components={"confidence": confidence}, + ) + if policy != EVICTION_POLICY_HYBRID_V1: + raise ValueError(f"Unknown fact eviction policy: {policy!r}") + + confirmation = _confirmation_freshness( + fact, + now=now, + half_life_days=confirmation_half_life_days, + ) + access = _normalized_access_heat( + usage, + now=now, + half_life_days=access_half_life_days, + ) + value = confidence_weight * confidence + confirmation_weight * confirmation + access_weight * access + return FactEvictionScore( + value=value, + components={ + "confidence": confidence, + "confirmationFreshness": confirmation, + "accessHeat": access, + }, + ) + + +def select_facts_for_capacity( + facts: list[dict[str, Any]], + *, + max_facts: int, + policy: str, + usage: dict[str, dict[str, Any]] | None = None, + now: datetime | None = None, + confidence_weight: float = 0.65, + confirmation_weight: float = 0.25, + access_weight: float = 0.10, + confirmation_half_life_days: float = 90, + access_half_life_days: float = 30, + correction_reserved_fraction: float = 0.10, + correction_reserved_max: int = 10, +) -> FactEvictionDecision: + """Select facts under the configured cap without mutating the snapshot. + + ``confidence`` exactly preserves the historical ranking. ``hybrid-v1`` + combines three bounded signals and reserves only the minimum number of + correction slots; unused slots immediately return to ordinary competition. + """ + evaluated_at = (now or datetime.now(UTC)).astimezone(UTC) + usage = usage or {} + indexed_facts = list(enumerate(facts)) + scores: dict[str, FactEvictionScore] = {} + for index, fact in indexed_facts: + fact_id = str(fact.get("id") or f"__missing_{index}") + scores[fact_id] = _score_fact( + fact, + policy=policy, + usage=usage.get(fact_id), + now=evaluated_at, + confidence_weight=confidence_weight, + confirmation_weight=confirmation_weight, + access_weight=access_weight, + confirmation_half_life_days=confirmation_half_life_days, + access_half_life_days=access_half_life_days, + ) + + if len(indexed_facts) <= max_facts: + return FactEvictionDecision( + kept=list(facts), + evicted=[], + scores=scores, + policy=policy, + ) + + ranked = sorted( + indexed_facts, + key=lambda item: (-scores[str(item[1].get("id") or f"__missing_{item[0]}")].value, item[0]), + ) + reserved_count = 0 + selected_indexes: set[int] = set() + if policy == EVICTION_POLICY_HYBRID_V1 and max_facts > 0: + correction_slots = min( + correction_reserved_max, + math.ceil(max_facts * correction_reserved_fraction), + ) + corrections = [item for item in ranked if str(item[1].get("category") or "").strip().lower() == "correction"] + reserved_count = min(correction_slots, len(corrections), max_facts) + selected_indexes.update(index for index, _fact in corrections[:reserved_count]) + + for index, _fact in ranked: + if len(selected_indexes) >= max(0, max_facts): + break + selected_indexes.add(index) + + kept = [fact for index, fact in ranked if index in selected_indexes] + evicted: list[EvictedFact] = [] + for index, fact in reversed(ranked): + if index in selected_indexes: + continue + fact_id = str(fact.get("id") or f"__missing_{index}") + score = scores[fact_id] + evicted.append( + EvictedFact( + fact_id=fact_id, + category=str(fact.get("category") or "context"), + score=score.value, + components=dict(score.components), + ) + ) + + return FactEvictionDecision( + kept=kept, + evicted=evicted, + scores=scores, + policy=policy, + reserved_correction_slots=reserved_count, + ) diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/message_patterns/reinforcement.yaml b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/message_patterns/reinforcement.yaml index 0c7bb3108..b68dc4198 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/message_patterns/reinforcement.yaml +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/message_patterns/reinforcement.yaml @@ -34,3 +34,4 @@ - '(?:对[，,]?\s*)?就是这个意思(?:[。！？!?.]|$)' - '正是我想要的(?:[。！？!?.]|$)' - '继续保持(?:[。！？!?.]|$)' +- '(?:以后|今后)(?:都|一直)?保持这样(?:[。！？!?.]|$)' diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/message_processing.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/message_processing.py index 5d58e7849..0fb2a40af 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/message_processing.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/message_processing.py @@ -270,8 +270,8 @@ def detect_signals( ) -> set[str]: """Detect signal classes in the recent conversation turns. - Returns the set of signal names whose patterns match any of the last 6 - human turns. This generalizes :func:`detect_correction` / + Returns the set of signal names whose patterns match a human message among + the last 6 filtered messages. This generalizes :func:`detect_correction` / :func:`detect_reinforcement` (which remain for backward compatibility) to the full signal set. The window stays ``messages[-6:]``. """ diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/paths.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/paths.py index 59c8211a5..37fd6a7ef 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/paths.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/paths.py @@ -108,6 +108,22 @@ def agent_facts_directory(memory_path: Path, agent_name: str) -> Path: return memory_path.parent / "agents" / agent_name.lower() / "facts" +def agent_metadata_directory(memory_path: Path, agent_name: str) -> Path: + """Return the non-canonical usage/audit sidecar root for one agent.""" + validate_agent_name(agent_name) + return memory_path.parent / "agents" / agent_name.lower() / ".metadata" + + +def agent_usage_path(memory_path: Path, agent_name: str) -> Path: + """Return the lightweight query-access sidecar path for one agent.""" + return agent_metadata_directory(memory_path, agent_name) / "fact-usage.json" + + +def agent_eviction_audit_path(memory_path: Path, agent_name: str) -> Path: + """Return the bounded metadata-only capacity audit path for one agent.""" + return agent_metadata_directory(memory_path, agent_name) / "eviction-audit.json" + + def fact_file_path(memory_path: Path, fact_id: str, *, agent_name: str) -> Path: """Return the sharded Markdown path for one agent-owned fact.""" if not fact_id or not re.fullmatch(r"[A-Za-z0-9_-]+", fact_id): diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/memory_update.chat.yaml b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/memory_update.chat.yaml index f5dca0352..82e491079 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/memory_update.chat.yaml +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/memory_update.chat.yaml @@ -109,6 +109,9 @@ messages: "newFacts": [ {{ "content": "...", "category": "preference|knowledge|context|behavior|goal|correction", "confidence": 0.0-1.0, "expected_valid_days": 90, "scope": "user|thread|project", "durability": "durable|temporary", "authority": "descriptive|transactional" }} ], + "factsToReinforce": [ + {{ "id": "fact_id_1", "scope": "user|thread|project", "reason": "explicit user confirmation of this existing fact" }} + ], "factsToRemove": [ {{ "id": "fact_id_1", "scope": "user|thread|project", "reason": "explicit user-level contradiction or retraction", "replacementFactIndex": 0 }} ], @@ -128,6 +131,7 @@ messages: - Include specific metrics, version numbers, and proper nouns in facts - Only add facts that are clearly stated (0.9+) or strongly implied (0.7+) - Use category "correction" for explicit agent mistakes or user corrections; assign confidence >= 0.95 when the correction is explicit + - Add an existing fact to factsToReinforce only when the new conversation explicitly confirms that same durable fact is still true. Include its exact id, scope, and reason. Do not reinforce a fact merely because it was shown in Current Memory State, was automatically injected, was retrieved, or was extracted again. - Include "sourceError" only for explicit correction facts when the prior mistake or wrong approach is clearly stated; omit it otherwise - Remove an existing fact only when the user explicitly contradicts or retracts it at user scope. A thread/project-local exception does not contradict a user-level fact. - factsToRemove entries MUST include scope and reason. Use replacementFactIndex when a removal depends on a replacement in newFacts; the zero-based index must identify that replacement. Omit replacementFactIndex only for a pure user-level retraction with no replacement. diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py index af14565ac..397434641 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py @@ -16,6 +16,7 @@ import hashlib import importlib import json import logging +import math import os import shutil import threading @@ -31,7 +32,18 @@ from typing import Any, Protocol import yaml from ..config import DeerMemConfig -from .paths import DEFAULT_AGENT_BUCKET, agent_facts_directory, fact_file_path, memory_file_path, safe_user_id, validate_agent_name +from .eviction import FactEvictionDecision +from .paths import ( + DEFAULT_AGENT_BUCKET, + agent_eviction_audit_path, + agent_facts_directory, + agent_metadata_directory, + agent_usage_path, + fact_file_path, + memory_file_path, + safe_user_id, + validate_agent_name, +) logger = logging.getLogger(__name__) @@ -207,6 +219,12 @@ def _normalize_fact( normalized["scope"] = copy.deepcopy(scope) _require_string_list(normalized, "topics") _require_string_list(normalized, "consolidatedFrom") + last_confirmed_at = normalized.get("lastConfirmedAt") + if last_confirmed_at is not None and not isinstance(last_confirmed_at, str): + raise ValueError("fact.lastConfirmedAt must be a string when present") + confirmation_count = normalized.get("confirmationCount") + if confirmation_count is not None and (isinstance(confirmation_count, bool) or not isinstance(confirmation_count, int) or confirmation_count < 1): + raise ValueError("fact.confirmationCount must be an integer >= 1 when present") revision = normalized.get("revision", 1) if isinstance(revision, bool) or not isinstance(revision, int) or revision < 1: raise ValueError("fact.revision must be an integer >= 1") @@ -412,6 +430,46 @@ class MemoryStorage(abc.ABC): """Clear global summaries and every agent fact bucket for one user.""" raise NotImplementedError + def get_fact_usage( + self, + *, + agent_name: str, + user_id: str | None = None, + ) -> dict[str, dict[str, Any]]: + """Return optional query-usage sidecar data for capacity scoring.""" + return {} + + def record_fact_accesses( + self, + fact_ids: list[str], + *, + agent_name: str, + user_id: str | None = None, + accessed_at: datetime | None = None, + ) -> None: + """Record actual query hits; alternative providers may override.""" + + def record_capacity_eviction( + self, + decision: FactEvictionDecision, + *, + max_facts: int, + agent_name: str, + user_id: str | None = None, + occurred_at: datetime | None = None, + shadow_decision: FactEvictionDecision | None = None, + ) -> None: + """Persist optional metadata-only eviction evidence.""" + + def clear_fact_metadata( + self, + *, + agent_name: str, + user_id: str | None = None, + fact_ids: list[str] | None = None, + ) -> None: + """Remove sidecar data for selected facts, or the whole scope.""" + def close(self) -> None: """Release optional storage resources.""" @@ -430,6 +488,234 @@ class FileMemoryStorage(MemoryStorage): if self._retrieval is not None: self._retrieval.close() + @staticmethod + def _read_json_sidecar(path: Path, *, expected_type: type[dict] | type[list]) -> dict[str, Any] | list[Any]: + if not path.exists(): + return expected_type() + try: + value = json.loads(path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + logger.warning("Ignoring malformed DeerMem sidecar %s", path, exc_info=True) + return expected_type() + if not isinstance(value, expected_type): + logger.warning("Ignoring DeerMem sidecar with invalid root type: %s", path) + return expected_type() + return value + + def get_fact_usage( + self, + *, + agent_name: str, + user_id: str | None = None, + ) -> dict[str, dict[str, Any]]: + path = self._get_memory_file_path(agent_name, user_id=user_id) + key = self._cache_key(agent_name, user_id=user_id) + with ( + self._scope_lock(key), + _process_file_lock( + path.parent / ".memory.lock", + float(getattr(self._config, "file_lock_timeout_seconds", 10)), + ), + ): + raw = self._read_json_sidecar( + agent_usage_path(path, agent_name), + expected_type=dict, + ) + return {str(fact_id): copy.deepcopy(entry) for fact_id, entry in raw.items() if isinstance(fact_id, str) and isinstance(entry, dict)} + + def record_fact_accesses( + self, + fact_ids: list[str], + *, + agent_name: str, + user_id: str | None = None, + accessed_at: datetime | None = None, + ) -> None: + unique_ids = list(dict.fromkeys(fact_id for fact_id in fact_ids if fact_id)) + if not unique_ids: + return + now = (accessed_at or datetime.now(UTC)).astimezone(UTC) + path = self._get_memory_file_path(agent_name, user_id=user_id) + sidecar_path = agent_usage_path(path, agent_name) + key = self._cache_key(agent_name, user_id=user_id) + with ( + self._scope_lock(key), + _process_file_lock( + path.parent / ".memory.lock", + float(getattr(self._config, "file_lock_timeout_seconds", 10)), + ), + ): + # Coordinate with deletion under the same lock. If a search raced + # with an explicit delete, never recreate usage metadata for a fact + # whose canonical file is already gone. + existing_ids = [fact_id for fact_id in unique_ids if fact_file_path(path, fact_id, agent_name=agent_name).exists()] + if not existing_ids: + return + raw = self._read_json_sidecar(sidecar_path, expected_type=dict) + usage = raw if isinstance(raw, dict) else {} + for fact_id in existing_ids: + previous = usage.get(fact_id) + previous = previous if isinstance(previous, dict) else {} + previous_heat = previous.get("accessHeat", 0.0) + try: + heat = float(previous_heat) + except (TypeError, ValueError): + heat = 0.0 + if not math.isfinite(heat) or heat < 0: + heat = 0.0 + last_accessed = previous.get("lastAccessedAt") + try: + parsed = datetime.fromisoformat(str(last_accessed).replace("Z", "+00:00")) + if parsed.tzinfo is None: + parsed = parsed.replace(tzinfo=UTC) + elapsed_days = max(0.0, (now - parsed.astimezone(UTC)).total_seconds() / 86400) + heat *= 2 ** (-elapsed_days / self._config.eviction_access_half_life_days) + except (TypeError, ValueError): + heat = 0.0 + raw_count = previous.get("accessCount", 0) + count = raw_count if isinstance(raw_count, int) and not isinstance(raw_count, bool) and raw_count >= 0 else 0 + usage[fact_id] = { + "accessHeat": heat + 1.0, + "accessCount": count + 1, + "lastAccessedAt": now.isoformat().removesuffix("+00:00") + "Z", + } + _atomic_write( + sidecar_path, + json.dumps(usage, ensure_ascii=False, indent=2).encode("utf-8"), + ) + + def record_capacity_eviction( + self, + decision: FactEvictionDecision, + *, + max_facts: int, + agent_name: str, + user_id: str | None = None, + occurred_at: datetime | None = None, + shadow_decision: FactEvictionDecision | None = None, + ) -> None: + if not decision.evicted or self._config.eviction_audit_max_entries == 0: + return + now = (occurred_at or datetime.now(UTC)).astimezone(UTC) + path = self._get_memory_file_path(agent_name, user_id=user_id) + sidecar_path = agent_eviction_audit_path(path, agent_name) + event: dict[str, Any] = { + "occurredAt": now.isoformat().removesuffix("+00:00") + "Z", + "reason": "capacity", + "policyVersion": decision.policy, + "maxFacts": max_facts, + "reservedCorrectionSlots": decision.reserved_correction_slots, + "evicted": [ + { + "factId": item.fact_id, + "category": item.category, + "score": item.score, + "components": copy.deepcopy(item.components), + } + for item in decision.evicted + ], + } + if shadow_decision is not None: + actual_ids = {item.fact_id for item in decision.evicted} + shadow_ids = {item.fact_id for item in shadow_decision.evicted} + event["shadow"] = { + "policyVersion": shadow_decision.policy, + "wouldEvict": sorted(shadow_ids), + "disagrees": actual_ids != shadow_ids, + } + key = self._cache_key(agent_name, user_id=user_id) + with ( + self._scope_lock(key), + _process_file_lock( + path.parent / ".memory.lock", + float(getattr(self._config, "file_lock_timeout_seconds", 10)), + ), + ): + raw = self._read_json_sidecar(sidecar_path, expected_type=list) + events = raw if isinstance(raw, list) else [] + events.append(event) + events = events[-self._config.eviction_audit_max_entries :] + _atomic_write( + sidecar_path, + json.dumps(events, ensure_ascii=False, indent=2).encode("utf-8"), + ) + + def clear_fact_metadata( + self, + *, + agent_name: str, + user_id: str | None = None, + fact_ids: list[str] | None = None, + ) -> None: + path = self._get_memory_file_path(agent_name, user_id=user_id) + metadata_path = agent_metadata_directory(path, agent_name) + key = self._cache_key(agent_name, user_id=user_id) + with ( + self._scope_lock(key), + _process_file_lock( + path.parent / ".memory.lock", + float(getattr(self._config, "file_lock_timeout_seconds", 10)), + ), + ): + if fact_ids is None: + if metadata_path.exists(): + shutil.rmtree(metadata_path) + return + removed_ids = set(fact_ids) + if not removed_ids: + return + usage_path = agent_usage_path(path, agent_name) + raw_usage = self._read_json_sidecar(usage_path, expected_type=dict) + usage = raw_usage if isinstance(raw_usage, dict) else {} + filtered_usage = {fact_id: value for fact_id, value in usage.items() if fact_id not in removed_ids} + if filtered_usage: + _atomic_write( + usage_path, + json.dumps(filtered_usage, ensure_ascii=False, indent=2).encode("utf-8"), + ) + else: + usage_path.unlink(missing_ok=True) + + audit_path = agent_eviction_audit_path(path, agent_name) + raw_events = self._read_json_sidecar(audit_path, expected_type=list) + events = raw_events if isinstance(raw_events, list) else [] + filtered_events: list[dict[str, Any]] = [] + for raw_event in events: + if not isinstance(raw_event, dict): + continue + event = copy.deepcopy(raw_event) + evicted = event.get("evicted") + if not isinstance(evicted, list): + continue + event["evicted"] = [item for item in evicted if isinstance(item, dict) and isinstance(item.get("factId"), str) and item["factId"] not in removed_ids] + shadow = event.get("shadow") + if isinstance(shadow, dict): + would_evict = shadow.get("wouldEvict") + if not isinstance(would_evict, list): + event.pop("shadow", None) + else: + shadow["wouldEvict"] = [fact_id for fact_id in would_evict if isinstance(fact_id, str) and fact_id not in removed_ids] + elif "shadow" in event: + event.pop("shadow") + if isinstance(event.get("shadow"), dict): + shadow = event["shadow"] + actual_ids = {item.get("factId") for item in event.get("evicted", []) if isinstance(item, dict) and isinstance(item.get("factId"), str)} + shadow_ids = {fact_id for fact_id in shadow["wouldEvict"] if isinstance(fact_id, str)} + shadow["disagrees"] = actual_ids != shadow_ids + if event.get("evicted"): + filtered_events.append(event) + if filtered_events: + _atomic_write( + audit_path, + json.dumps(filtered_events, ensure_ascii=False, indent=2).encode("utf-8"), + ) + else: + audit_path.unlink(missing_ok=True) + try: + metadata_path.rmdir() + except OSError: + pass + @staticmethod def _cache_key(agent_name: str | None = None, *, user_id: str | None = None) -> tuple[str | None, str | None]: return (user_id, agent_name) @@ -1073,6 +1359,7 @@ class FileMemoryStorage(MemoryStorage): key = self._cache_key(agent_name, user_id=user_id) lock_path = path.parent / ".memory.lock" notifications: list[RetrievalNotification] = [] + deleted_metadata_ids: list[str] = [] try: if not isinstance(memory_data, dict): raise ValueError("memory_data must be an object") @@ -1091,6 +1378,7 @@ class FileMemoryStorage(MemoryStorage): if len(ids) != len(set(ids)): raise ValueError("Duplicate fact ids are not allowed") old_ids = set(self._agent_entries(path, agent_name, user_id=user_id)) if agent_name is not None else set() + deleted_metadata_ids = sorted(old_ids - set(ids)) summaries = None if agent_name is None: summaries = {"user": memory_data.get("user", {}), "history": memory_data.get("history", {})} @@ -1099,7 +1387,7 @@ class FileMemoryStorage(MemoryStorage): user_id=user_id, agent_name=agent_name, upserts=copy.deepcopy(facts_raw), - deletes=sorted(old_ids - set(ids)), + deletes=deleted_metadata_ids, summaries=summaries, expected_revision=expected_revision, ) @@ -1114,6 +1402,12 @@ class FileMemoryStorage(MemoryStorage): return False self._dispatch_retrieval_notifications(notifications, user_id=user_id, agent_name=agent_name) + if agent_name is not None and deleted_metadata_ids: + self.clear_fact_metadata( + agent_name=agent_name, + user_id=user_id, + fact_ids=deleted_metadata_ids, + ) return True def clear_all(self, *, user_id: str | None = None) -> dict[str, Any]: @@ -1121,6 +1415,7 @@ class FileMemoryStorage(MemoryStorage): path = self._get_memory_file_path(user_id=user_id) key = self._cache_key(user_id=user_id) notifications_by_agent: list[ScopedRetrievalNotifications] = [] + metadata_agents: list[str] = [] with ( self._scope_lock(key), _process_file_lock( @@ -1134,6 +1429,7 @@ class FileMemoryStorage(MemoryStorage): for agent_dir in sorted(child for child in agents_root.iterdir() if child.is_dir()): agent_name = agent_dir.name validate_agent_name(agent_name) + metadata_agents.append(agent_name) legacy_path = self._legacy_agent_memory_path(path, agent_name) if legacy_path.exists(): _, _, migration_notifications = self._migrate_locked( @@ -1175,6 +1471,8 @@ class FileMemoryStorage(MemoryStorage): for agent_name, notifications in notifications_by_agent: self._dispatch_retrieval_notifications(notifications, user_id=user_id, agent_name=agent_name) + for agent_name in metadata_agents: + self.clear_fact_metadata(agent_name=agent_name, user_id=user_id) return self.reload(DEFAULT_AGENT_BUCKET, user_id=user_id) @staticmethod @@ -1301,13 +1599,20 @@ class FileMemoryStorage(MemoryStorage): self._dispatch_retrieval_notifications(notifications, user_id=user_id, agent_name=agent_name) if memory_file is None: # defensive: the bounded loop either commits or raises raise MemoryStorageError("Memory repository change did not produce a result") + deleted_fact_ids = [str(value) for action, value, _ in notifications if action == "remove"] + if agent_name is not None and deleted_fact_ids: + self.clear_fact_metadata( + agent_name=agent_name, + user_id=user_id, + fact_ids=deleted_fact_ids, + ) return { "complete": False, "version": memory_file.get("version", DOCUMENT_VERSION), "revision": memory_file.get("revision", 0), "lastUpdated": memory_file.get("lastUpdated", ""), "upsertedFacts": [copy.deepcopy(value) for action, value, _ in notifications if action == "upsert" and isinstance(value, dict)], - "deletedFactIds": [str(value) for action, value, _ in notifications if action == "remove"], + "deletedFactIds": deleted_fact_ids, } def upsert_fact( diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py index a017cf3f9..8be4a5928 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py @@ -16,6 +16,12 @@ from datetime import UTC, datetime, timedelta from typing import Any from ..config import DeerMemConfig +from .eviction import ( + EVICTION_POLICY_CONFIDENCE, + EVICTION_POLICY_HYBRID_V1, + FactEvictionDecision, + select_facts_for_capacity, +) from .message_processing import detect_signals, extract_message_text from .prompt import ( format_conversation_for_update, @@ -73,20 +79,12 @@ def _coerce_source_confidence(fact: dict[str, Any]) -> float: return max(0.0, min(val, 1.0)) if math.isfinite(val) else 0.5 -def _trim_facts_to_max(facts: list[dict[str, Any]], max_facts: int) -> list[dict[str, Any]]: - """Keep the highest-confidence facts within ``max_facts`` (confidence coerced). - - Confidence is read via :func:`_coerce_source_confidence` so legacy / imported - facts with ``null`` or non-numeric confidence never crash the sort -- the - pre-#4023 ``key=lambda f: f.get("confidence", 0)`` form compared ``None`` / - ``str`` against ``float`` and raised ``TypeError`` once ``len(facts) > - max_facts``. Mirrors upstream's ``_trim_facts_to_max`` (introduced in #4023) - so the vendored copy no longer lags the coercion fix the - monolithic->vendored rename silently dropped. - """ - if len(facts) <= max_facts: - return facts - return sorted(facts, key=_coerce_source_confidence, reverse=True)[:max_facts] +def _next_confirmation_count(fact: dict[str, Any]) -> int: + """Increment a valid prior confirmation count, resetting malformed values.""" + prior_count = fact.get("confirmationCount", 0) + if isinstance(prior_count, bool) or not isinstance(prior_count, int) or prior_count < 0: + return 1 + return prior_count + 1 def _extract_text(content: Any) -> str: @@ -244,6 +242,23 @@ def _normalize_memory_update_data(update_data: dict[str, Any]) -> dict[str, Any] history = update_data.get("history") new_facts = update_data.get("newFacts") facts_to_remove = update_data.get("factsToRemove") + facts_to_reinforce = update_data.get("factsToReinforce") + normalized_facts_to_reinforce: list[dict[str, str]] = [] + if isinstance(facts_to_reinforce, list): + for entry in facts_to_reinforce: + if not isinstance(entry, dict): + continue + raw_id = entry.get("id") + scope = _normalize_gate_label(entry.get("scope")) + reason = entry.get("reason") + if not isinstance(raw_id, str) or not raw_id.strip(): + continue + normalized_entry = {"id": raw_id.strip()} + if scope is not None: + normalized_entry["scope"] = scope + if isinstance(reason, str) and reason.strip(): + normalized_entry["reason"] = reason.strip() + normalized_facts_to_reinforce.append(normalized_entry) normalized_facts_to_remove: list[dict[str, Any]] = [] if isinstance(facts_to_remove, list): for entry in facts_to_remove: @@ -382,6 +397,7 @@ def _normalize_memory_update_data(update_data: dict[str, Any]) -> dict[str, Any] "user": user if isinstance(user, dict) else {}, "history": history if isinstance(history, dict) else {}, "newFacts": normalized_new_facts, + "factsToReinforce": normalized_facts_to_reinforce, "factsToRemove": normalized_facts_to_remove, "staleFactsToRemove": normalized_stale_removals, "staleFactsToExtend": normalized_stale_extensions, @@ -474,7 +490,7 @@ def _raise_if_duplicate_fact_content(memory_data: dict[str, Any], content_key: s def _parse_fact_datetime(raw: str) -> datetime | None: - """Parse an ISO-8601 datetime string from a fact's createdAt field. + """Parse an ISO-8601 datetime string from a fact timestamp field. Returns ``None`` on any parse failure so callers can safely skip malformed facts. """ @@ -564,9 +580,11 @@ def _select_stale_candidates( Each fact's effective review age is determined by ``_effective_fact_staleness_age``: facts with an LLM-assigned ``expected_valid_days`` use that value directly; facts without it fall back - to the global ``staleness_age_days``. Protected categories (default: - ``correction``) are excluded because they represent explicit user feedback - that should not be auto-pruned by age. + to the global ``staleness_age_days``. A valid ``lastConfirmedAt`` resets the + review clock because it is explicit evidence that the fact is still true; + otherwise ``createdAt`` remains the reference. Protected categories + (default: ``correction``) are excluded because they represent explicit user + feedback that should not be auto-pruned by age. """ now = datetime.now(UTC) protected = frozenset(config.staleness_protected_categories) @@ -577,15 +595,15 @@ def _select_stale_candidates( category = fact.get("category", "") if isinstance(category, str) and category in protected: continue - created_at = _parse_fact_datetime(fact.get("createdAt", "")) - if created_at is None: + review_reference = _parse_fact_datetime(fact.get("lastConfirmedAt", "")) or _parse_fact_datetime(fact.get("createdAt", "")) + if review_reference is None: continue effective_age = _effective_fact_staleness_age(fact, config) # now - timedelta(days=effective_age) can overflow datetime.min when # effective_age is a huge persisted value; a window that large means the # fact cannot yet be stale, so skip it rather than aborting the cycle. cutoff = _safe_add_days(now, -effective_age) - if cutoff is not None and created_at < cutoff: + if cutoff is not None and review_reference < cutoff: candidates.append(fact) return candidates @@ -611,14 +629,14 @@ def _build_staleness_section( fid = fact.get("id", "?") cat = html.escape(str(fact.get("category", "context")).strip() or "context", quote=False) conf = _coerce_source_confidence(fact) - created_raw = fact.get("createdAt", "") - created_short = created_raw[:10] if isinstance(created_raw, str) and len(created_raw) >= 10 else created_raw + review_reference = _parse_fact_datetime(fact.get("lastConfirmedAt", "")) or _parse_fact_datetime(fact.get("createdAt", "")) + reviewed_short = review_reference.date().isoformat() if review_reference is not None else "" # quote=False: content is in element-text position (inside # tags, never an attribute value), so only <, >, & can break structure - # leave ' and " untouched. Mirrors the convention in prompt.py #4028. content = html.escape(str(fact.get("content", "")), quote=False) effective_age = _effective_fact_staleness_age(fact, config) - lines.append(f'- [{fid} | {cat} | {conf:.2f} | {created_short} | valid:{effective_age}d] "{content}"') + lines.append(f'- [{fid} | {cat} | {conf:.2f} | {reviewed_short} | valid:{effective_age}d] "{content}"') return load_prompt("staleness_review", prompts_dir=prompts_dir, agent_name=agent_name).format(stale_facts="\n".join(lines)) @@ -809,6 +827,72 @@ class MemoryUpdater: """Reload memory data via the injected storage.""" return self._storage.reload(agent_name, user_id=user_id) + def _select_for_capacity( + self, + facts: list[dict[str, Any]], + *, + agent_name: str | None, + user_id: str | None, + ) -> tuple[list[dict[str, Any]], FactEvictionDecision | None, FactEvictionDecision | None]: + """Apply the configured policy and optionally compute hybrid shadow.""" + if len(facts) <= self._config.max_facts: + return facts, None, None + uses_hybrid_scoring = self._config.fact_eviction_policy == EVICTION_POLICY_HYBRID_V1 or self._config.fact_eviction_shadow_enabled + usage = ( + self._storage.get_fact_usage( + agent_name=agent_name, + user_id=user_id, + ) + if uses_hybrid_scoring and agent_name is not None + else {} + ) + common = { + "max_facts": self._config.max_facts, + "usage": usage, + "confidence_weight": self._config.eviction_confidence_weight, + "confirmation_weight": self._config.eviction_confirmation_weight, + "access_weight": self._config.eviction_access_weight, + "confirmation_half_life_days": self._config.eviction_confirmation_half_life_days, + "access_half_life_days": self._config.eviction_access_half_life_days, + "correction_reserved_fraction": self._config.eviction_correction_reserved_fraction, + "correction_reserved_max": self._config.eviction_correction_reserved_max, + } + decision = select_facts_for_capacity( + facts, + policy=self._config.fact_eviction_policy, + **common, + ) + shadow_decision = None + if self._config.fact_eviction_shadow_enabled and self._config.fact_eviction_policy == EVICTION_POLICY_CONFIDENCE: + shadow_decision = select_facts_for_capacity( + facts, + policy=EVICTION_POLICY_HYBRID_V1, + **common, + ) + return decision.kept, decision, shadow_decision + + def _record_capacity_decision( + self, + decision: FactEvictionDecision | None, + shadow_decision: FactEvictionDecision | None, + *, + agent_name: str | None, + user_id: str | None, + ) -> None: + """Write best-effort audit only after canonical persistence succeeds.""" + if decision is None or agent_name is None: + return + try: + self._storage.record_capacity_eviction( + decision, + max_facts=self._config.max_facts, + agent_name=agent_name, + user_id=user_id, + shadow_decision=shadow_decision, + ) + except Exception: + logger.warning("Failed to record capacity-eviction audit", exc_info=True) + def import_memory_data(self, memory_data: dict[str, Any], agent_name: str | None = None, *, user_id: str | None = None) -> dict[str, Any]: """Persist imported memory data via the injected storage.""" if not isinstance(memory_data, dict): @@ -834,6 +918,11 @@ class MemoryUpdater: for fact in incoming_facts: fact["id"] = str(fact.get("id") or f"fact_{uuid.uuid4().hex}") fact["confidence"] = _coerce_source_confidence(fact) + incoming_facts, capacity_decision, shadow_decision = self._select_for_capacity( + incoming_facts, + agent_name=agent_name, + user_id=user_id, + ) current_by_id = {str(fact.get("id")): fact for fact in current.get("facts", []) if isinstance(fact, dict)} incoming_ids = {str(fact.get("id")) for fact in incoming_facts} self._storage.apply_changes( @@ -848,11 +937,40 @@ class MemoryUpdater: user_id=user_id, expected_manifest_revision=int(current.get("revision") or 0), ) + self._record_capacity_decision( + capacity_decision, + shadow_decision, + agent_name=agent_name, + user_id=user_id, + ) return self._storage.load(agent_name, user_id=user_id) if agent_name is None: memory_data["facts"] = [] + capacity_decision = None + shadow_decision = None + else: + raw_facts = memory_data.get("facts", []) + if not isinstance(raw_facts, list) or any(not isinstance(fact, dict) for fact in raw_facts): + raise ValueError("memory_data.facts") + normalized_facts = [] + for raw_fact in raw_facts: + fact = copy.deepcopy(raw_fact) + fact["id"] = str(fact.get("id") or f"fact_{uuid.uuid4().hex}") + fact["confidence"] = _coerce_source_confidence(fact) + normalized_facts.append(fact) + memory_data["facts"], capacity_decision, shadow_decision = self._select_for_capacity( + normalized_facts, + agent_name=agent_name, + user_id=user_id, + ) if not self._storage.save(memory_data, agent_name, user_id=user_id): raise OSError("Failed to save imported memory data") + self._record_capacity_decision( + capacity_decision, + shadow_decision, + agent_name=agent_name, + user_id=user_id, + ) return self._storage.load(agent_name, user_id=user_id) def clear_memory_data(self, agent_name: str | None = None, *, user_id: str | None = None) -> dict[str, Any]: @@ -871,6 +989,10 @@ class MemoryUpdater: user_id=user_id, expected_manifest_revision=int(current.get("revision") or 0), ) + self._storage.clear_fact_metadata( + agent_name=agent_name, + user_id=user_id, + ) return self.reload_memory_data(agent_name, user_id=user_id) except MemoryManifestRevisionConflict: if attempt == 2: @@ -882,6 +1004,11 @@ class MemoryUpdater: cleared_memory["facts"] = [] if not self._save_memory_to_file(cleared_memory, agent_name, user_id=user_id, expected_revision=int(current.get("revision") or 0)): raise OSError("Failed to save cleared memory data") + if agent_name is not None: + self._storage.clear_fact_metadata( + agent_name=agent_name, + user_id=user_id, + ) return cleared_memory def clear_all_memory_data(self, *, user_id: str | None = None) -> dict[str, Any]: @@ -906,9 +1033,8 @@ class MemoryUpdater: which would couple them to the backend's content normalization and could misreport a storage cap on backends that normalize differently. - The new fact is then trimmed by :func:`_trim_facts_to_max` (highest- - confidence wins, confidence coerced). If the cap evicts the just-added - (lower-confidence) fact, ``fact_id`` is ``None`` so callers report + The new fact is then evaluated by the configured capacity policy. If + the cap evicts the just-added fact, ``fact_id`` is ``None`` so callers report "not stored - cap reached" instead of a dangling id with a false "added" status. This restores both the max_facts cap and the post-trim existence check (upstream's ``create_memory_fact_with_created_fact``), @@ -949,7 +1075,11 @@ class MemoryUpdater: # winner's fact, and is rejected here). _raise_if_duplicate_fact_content(memory_data, candidate_key) updated_memory = dict(memory_data) - updated_memory["facts"] = _trim_facts_to_max([*memory_data.get("facts", []), copy.deepcopy(candidate)], self._config.max_facts) + updated_memory["facts"], capacity_decision, shadow_decision = self._select_for_capacity( + [*memory_data.get("facts", []), copy.deepcopy(candidate)], + agent_name=agent_name, + user_id=user_id, + ) kept_ids = {str(fact.get("id")) for fact in updated_memory["facts"]} deletions = [str(fact.get("id")) for fact in memory_data.get("facts", []) if str(fact.get("id")) not in kept_ids] try: @@ -964,6 +1094,12 @@ class MemoryUpdater: user_id=user_id, expected_manifest_revision=int(memory_data.get("revision") or 0), ) + self._record_capacity_decision( + capacity_decision, + shadow_decision, + agent_name=agent_name, + user_id=user_id, + ) fresh_memory = self.reload_memory_data(agent_name, user_id=user_id) stored = any(fact.get("id") == fact_id for fact in fresh_memory.get("facts", [])) return fresh_memory, (fact_id if stored else None) @@ -981,8 +1117,18 @@ class MemoryUpdater: memory_data = self.get_memory_data(agent_name, user_id=user_id) if attempt == 0 else self.reload_memory_data(agent_name, user_id=user_id) _raise_if_duplicate_fact_content(memory_data, candidate_key) updated_memory = dict(memory_data) - updated_memory["facts"] = _trim_facts_to_max([*memory_data.get("facts", []), copy.deepcopy(candidate)], self._config.max_facts) + updated_memory["facts"], capacity_decision, shadow_decision = self._select_for_capacity( + [*memory_data.get("facts", []), copy.deepcopy(candidate)], + agent_name=agent_name, + user_id=user_id, + ) if self._save_memory_to_file(updated_memory, agent_name, user_id=user_id, expected_revision=int(memory_data.get("revision") or 0)): + self._record_capacity_decision( + capacity_decision, + shadow_decision, + agent_name=agent_name, + user_id=user_id, + ) # If the cap evicted the just-added (lower-confidence) fact, # signal via None so callers don't report a dangling id as # "added". @@ -1231,6 +1377,7 @@ class MemoryUpdater: user_id: str | None = None, *, metrics: dict[str, Any] | None = None, + signals: frozenset[str] = frozenset(), ) -> bool: """Parse the model response, apply updates, and persist memory.""" update_data = _parse_memory_update_response(response_content) @@ -1243,12 +1390,22 @@ class MemoryUpdater: # metric tracks the actual filter rather than a re-derived copy here. if getattr(type(self._storage), "apply_changes", None) is not MemoryStorage.apply_changes: for attempt in range(3): + capacity_decisions: list[tuple[FactEvictionDecision, FactEvictionDecision | None]] = [] # Deep-copy before in-place mutation so a failed commit cannot # corrupt the cached snapshot. On a manifest conflict the # complete extraction result is reapplied to a fresh document; # its trim/consolidation/delete decisions are snapshot-wide and # must never be replayed as disjoint point writes. - updated_memory = self._apply_updates(copy.deepcopy(current_memory), update_data, thread_id, metrics=metrics) + updated_memory = self._apply_updates( + copy.deepcopy(current_memory), + update_data, + thread_id, + metrics=metrics, + signals=signals, + agent_name=agent_name, + user_id=user_id, + capacity_decisions=capacity_decisions, + ) updated_memory = _strip_upload_mentions_from_memory(updated_memory) current_by_id = {str(fact.get("id")): fact for fact in current_memory.get("facts", [])} updated_by_id = {str(fact.get("id")): fact for fact in updated_memory.get("facts", [])} @@ -1271,6 +1428,13 @@ class MemoryUpdater: user_id=user_id, expected_manifest_revision=int(current_memory.get("revision") or 0), ) + for decision, shadow_decision in capacity_decisions: + self._record_capacity_decision( + decision, + shadow_decision, + agent_name=agent_name, + user_id=user_id, + ) return True except MemoryManifestRevisionConflict: if attempt == 2: @@ -1280,14 +1444,33 @@ class MemoryUpdater: raise AssertionError("bounded extracted-update retry did not return or raise") # Deep-copy before in-place mutation so a subsequent save() failure # cannot corrupt the still-cached original object reference. - updated_memory = self._apply_updates(copy.deepcopy(current_memory), update_data, thread_id, metrics=metrics) + capacity_decisions = [] + updated_memory = self._apply_updates( + copy.deepcopy(current_memory), + update_data, + thread_id, + metrics=metrics, + signals=signals, + agent_name=agent_name, + user_id=user_id, + capacity_decisions=capacity_decisions, + ) updated_memory = _strip_upload_mentions_from_memory(updated_memory) - return self._storage.save( + saved = self._storage.save( updated_memory, agent_name, user_id=user_id, expected_revision=int(current_memory.get("revision") or 0), ) + if saved: + for decision, shadow_decision in capacity_decisions: + self._record_capacity_decision( + decision, + shadow_decision, + agent_name=agent_name, + user_id=user_id, + ) + return saved async def aupdate_memory( self, @@ -1530,6 +1713,7 @@ class MemoryUpdater: agent_name=agent_name, user_id=user_id, metrics=metrics, + signals=frozenset(feed_signals), ) if success and not bypass_watermark: # Advance the watermark to the last message fed (the feed is a @@ -1667,6 +1851,10 @@ class MemoryUpdater: thread_id: str | None = None, *, metrics: dict[str, Any] | None = None, + signals: frozenset[str] = frozenset(), + agent_name: str | None = None, + user_id: str | None = None, + capacity_decisions: list[tuple[FactEvictionDecision, FactEvictionDecision | None]] | None = None, ) -> dict[str, Any]: """Apply LLM-generated updates to memory. @@ -1693,6 +1881,30 @@ class MemoryUpdater: def reject_by_scope_gate(kind: str, reason: str) -> None: scope_gate_rejections[kind][reason] += 1 + # Explicit confirmation is distinct from extraction duplication. The + # deterministic gate is batch-level: it proves only that a human + # message among the last six filtered messages matched a reinforcement + # pattern. The LLM remains responsible for binding that signal to an + # existing id, which must be user-scoped and carry a non-empty reason. + tracks_hybrid_signals = config.fact_eviction_policy == EVICTION_POLICY_HYBRID_V1 or config.fact_eviction_shadow_enabled + if tracks_hybrid_signals and "reinforcement" in signals: + reinforced_ids = { + entry["id"] + for entry in update_data.get("factsToReinforce", []) + if isinstance(entry, dict) and entry.get("scope") == "user" and isinstance(entry.get("reason"), str) and entry["reason"].strip() and isinstance(entry.get("id"), str) + } + if reinforced_ids: + current_memory["facts"] = [ + { + **fact, + "lastConfirmedAt": now, + "confirmationCount": _next_confirmation_count(fact), + } + if isinstance(fact, dict) and fact.get("id") in reinforced_ids + else fact + for fact in current_memory.get("facts", []) + ] + # Update user sections user_updates = update_data.get("user", {}) for section in ["workContext", "personalContext", "topOfMind"]: @@ -1888,8 +2100,16 @@ class MemoryUpdater: current_memory["facts"].append(fact_entry) existing_fact_keys.add(fact_key) - # Enforce max facts limit (coerced confidence -- see _trim_facts_to_max). - current_memory["facts"] = _trim_facts_to_max(current_memory["facts"], config.max_facts) + # Enforce one capacity policy across automatic, manual, and import + # writes. Usage comes from a separate sidecar, so scoring never rewrites + # canonical fact timestamps merely because a query recalled them. + current_memory["facts"], capacity_decision, shadow_decision = self._select_for_capacity( + current_memory["facts"], + agent_name=agent_name, + user_id=user_id, + ) + if capacity_decision is not None and capacity_decisions is not None: + capacity_decisions.append((capacity_decision, shadow_decision)) # Remove contradicted facts only after replacements have passed both # gates and survived deduplication/trimming. Task-local contradictions diff --git a/backend/packages/harness/deerflow/agents/memory/tools.py b/backend/packages/harness/deerflow/agents/memory/tools.py index c32e2ff12..40d696a77 100644 --- a/backend/packages/harness/deerflow/agents/memory/tools.py +++ b/backend/packages/harness/deerflow/agents/memory/tools.py @@ -141,9 +141,9 @@ def memory_add_tool( except NotImplementedError: return json.dumps({"error": f"memory backend {type(manager).__name__} does not support create_fact"}) if fact_id is None: - # max_facts cap kept higher-confidence facts and evicted the new one; - # the fact was not stored -- report honestly instead of a dangling id. - return json.dumps({"error": "Fact was not stored because memory.max_facts kept higher-confidence facts"}) + # The configured max_facts policy evicted the new fact; report the + # capacity result instead of a dangling id. + return json.dumps({"error": "Fact was not stored because the configured memory.max_facts capacity policy evicted it"}) return json.dumps({"fact_id": fact_id, "status": "added"}) except ValueError as exc: return json.dumps({"error": str(exc)}) diff --git a/backend/packages/harness/deerflow/client.py b/backend/packages/harness/deerflow/client.py index 1af1d79b3..a69630dcb 100644 --- a/backend/packages/harness/deerflow/client.py +++ b/backend/packages/harness/deerflow/client.py @@ -1415,7 +1415,7 @@ class DeerFlowClient: manager = get_memory_manager() memory_data, fact_id = manager.create_fact(content=content, category=category, confidence=confidence, user_id=get_effective_user_id()) if fact_id is None: - raise ValueError("Fact was not stored because memory.max_facts kept higher-confidence facts") + raise ValueError("Fact was not stored because the configured memory.max_facts capacity policy evicted it") return memory_data def delete_memory_fact(self, fact_id: str) -> dict: diff --git a/backend/tests/test_deermem_self_contained.py b/backend/tests/test_deermem_self_contained.py index 2eddc49f0..5db21ff7d 100644 --- a/backend/tests/test_deermem_self_contained.py +++ b/backend/tests/test_deermem_self_contained.py @@ -24,7 +24,6 @@ from deerflow.agents.memory.backends.deermem.deermem.core.message_processing imp filter_messages_for_memory, ) from deerflow.agents.memory.backends.deermem.deermem.core.storage import FileMemoryStorage -from deerflow.agents.memory.backends.deermem.deermem.core.updater import _trim_facts_to_max from deerflow.agents.memory.manager import MemoryCallbacks @@ -490,28 +489,6 @@ def test_per_user_memory_path_matches_host_safe_user_id(deermem_data_dir): assert user_dirs == [expected_safe], f"safe_user_id diverged from host: {user_dirs}" -def test_trim_facts_to_max_coerces_non_float_confidence(): - """Non-float stored confidence must not crash the max_facts trim sort. - - Regression: the vendored copy used ``key=lambda f: f.get("confidence", 0)`` - which raised TypeError comparing None/str against float once ``len > max_facts`` - (legacy / imported facts with abnormal confidence). This is the #4034 intent - that the module-skipped test files never exercised against the vendored - updater; pinning it here so the rename can't silently drop the coercion again. - """ - facts = [ - {"id": "a", "confidence": None}, - {"id": "b", "confidence": "0.9"}, # numeric string - {"id": "c", "confidence": 0.8}, - {"id": "d", "confidence": "high"}, # non-numeric - ] - # No TypeError; coerced ranking: b("0.9"->0.9) > c(0.8) > a(None->0.5)=d("high"->0.5). - kept = _trim_facts_to_max(facts, max_facts=2) - assert [f["id"] for f in kept] == ["b", "c"] - # Below the cap -> returned unchanged (no sort, no crash). - assert _trim_facts_to_max(facts, max_facts=10) == facts - - def test_create_fact_trims_to_max_and_signals_eviction(deermem_data_dir): """create_fact enforces max_facts and signals eviction via None fact_id. @@ -559,6 +536,278 @@ def test_search_survives_non_float_confidence(deermem_data_dir): assert [r["id"] for r in results] == ["b", "a", "c"] +def test_query_search_records_access_but_default_injection_does_not(deermem_data_dir): + dm = DeerMem( + backend_config={ + "storage_path": str(deermem_data_dir), + "retrieval_adapter": "", + "token_counting": "char", + "fact_eviction_policy": "hybrid-v1", + } + ) + dm.create_fact( + "User prefers concise answers", + category="preference", + confidence=0.8, + user_id="u1", + ) + + dm.get_context(user_id="u1") + assert dm._storage.get_fact_usage(agent_name="__default__", user_id="u1") == {} + + results = dm.search("concise", user_id="u1") + usage = dm._storage.get_fact_usage(agent_name="__default__", user_id="u1") + assert len(results) == 1 + fact_id = results[0]["id"] + assert usage[fact_id]["accessCount"] == 1 + + dm.delete_fact(fact_id, user_id="u1") + dm._storage.record_fact_accesses( + [fact_id], + agent_name="__default__", + user_id="u1", + ) + assert dm._storage.get_fact_usage(agent_name="__default__", user_id="u1") == {} + + +def test_default_confidence_policy_does_not_collect_hybrid_usage(deermem_data_dir): + dm = DeerMem( + backend_config={ + "storage_path": str(deermem_data_dir), + "retrieval_adapter": "", + "token_counting": "char", + } + ) + dm.create_fact("User prefers concise answers", user_id="u1") + + assert dm.search("concise", user_id="u1") + assert dm._storage.get_fact_usage(agent_name="__default__", user_id="u1") == {} + + +def test_hybrid_capacity_keeps_recent_confirmation_over_stale_confidence(deermem_data_dir): + dm = DeerMem( + backend_config={ + "storage_path": str(deermem_data_dir), + "retrieval_adapter": "", + "fact_eviction_policy": "hybrid-v1", + "max_facts": 10, + } + ) + dm.import_memory( + { + "user": {}, + "history": {}, + "facts": [ + *[ + { + "id": f"high_{index}", + "content": f"high {index}", + "category": "preference", + "confidence": 0.99, + "createdAt": "2026-08-12T00:00:00Z", + } + for index in range(8) + ], + { + "id": "stale_high", + "content": "stale high confidence", + "category": "preference", + "confidence": 0.9, + "createdAt": "2025-01-01T00:00:00Z", + }, + { + "id": "confirmed_recently", + "content": "recently confirmed preference", + "category": "preference", + "confidence": 0.7, + "createdAt": "2025-01-01T00:00:00Z", + "lastConfirmedAt": "2026-08-12T00:00:00Z", + }, + ], + }, + user_id="u1", + ) + + memory, created_id = dm.create_fact( + "new useful context", + category="context", + confidence=0.8, + user_id="u1", + ) + + kept_ids = {fact["id"] for fact in memory["facts"]} + assert created_id is not None + assert "confirmed_recently" in kept_ids + assert "stale_high" not in kept_ids + + +def test_hybrid_import_applies_same_capacity_policy(deermem_data_dir): + dm = DeerMem( + backend_config={ + "storage_path": str(deermem_data_dir), + "retrieval_adapter": "", + "fact_eviction_policy": "hybrid-v1", + "max_facts": 10, + } + ) + memory = dm.import_memory( + { + "user": {}, + "history": {}, + "facts": [ + *[ + { + "id": f"high_{index}", + "content": f"high {index}", + "category": "preference", + "confidence": 0.99, + "createdAt": "2026-08-12T00:00:00Z", + } + for index in range(8) + ], + { + "id": "stale_high", + "content": "stale high confidence", + "category": "preference", + "confidence": 0.9, + "createdAt": "2025-01-01T00:00:00Z", + }, + { + "id": "confirmed_recently", + "content": "recently confirmed preference", + "category": "preference", + "confidence": 0.7, + "createdAt": "2025-01-01T00:00:00Z", + "lastConfirmedAt": "2026-08-12T00:00:00Z", + }, + { + "id": "new_context", + "content": "new useful context", + "category": "context", + "confidence": 0.8, + "createdAt": "2026-08-12T00:00:00Z", + }, + ], + }, + user_id="u1", + ) + + kept_ids = {fact["id"] for fact in memory["facts"]} + assert "confirmed_recently" in kept_ids + assert "stale_high" not in kept_ids + + +def test_hybrid_capacity_reserves_correction_and_writes_audit(deermem_data_dir): + dm = DeerMem( + backend_config={ + "storage_path": str(deermem_data_dir), + "retrieval_adapter": "", + "fact_eviction_policy": "hybrid-v1", + "max_facts": 10, + } + ) + dm.import_memory( + { + "user": {}, + "history": {}, + "facts": [ + { + "id": f"preference_{index}", + "content": f"preference {index}", + "category": "preference", + "confidence": 0.99, + "createdAt": "2026-08-12T00:00:00Z", + } + for index in range(10) + ], + }, + user_id="u1", + ) + + memory, correction_id = dm.create_fact( + "Never repeat this corrected behavior", + category="correction", + confidence=0.1, + user_id="u1", + ) + + assert correction_id is not None + assert correction_id in {fact["id"] for fact in memory["facts"]} + memory_path = dm._storage._get_memory_file_path("__default__", user_id="u1") + audit_path = memory_path.parent / "agents" / "__default__" / ".metadata" / "eviction-audit.json" + assert audit_path.is_file() + assert "Never repeat this corrected behavior" not in audit_path.read_text() + + +def test_confidence_policy_shadow_does_not_change_actual_selection(deermem_data_dir): + dm = DeerMem( + backend_config={ + "storage_path": str(deermem_data_dir), + "retrieval_adapter": "", + "fact_eviction_shadow_enabled": True, + "max_facts": 10, + } + ) + facts = [ + *[ + { + "id": f"high_{index}", + "content": f"high {index}", + "category": "preference", + "confidence": 0.99, + "createdAt": "2026-08-12T00:00:00Z", + } + for index in range(8) + ], + { + "id": "stale_high", + "content": "stale high confidence", + "category": "preference", + "confidence": 0.9, + "createdAt": "2025-01-01T00:00:00Z", + }, + { + "id": "confirmed_recently", + "content": "recently confirmed preference", + "category": "preference", + "confidence": 0.7, + "createdAt": "2025-01-01T00:00:00Z", + "lastConfirmedAt": "2026-08-12T00:00:00Z", + }, + ] + dm.import_memory({"user": {}, "history": {}, "facts": facts}, user_id="u1") + + memory, _ = dm.create_fact("new useful context", confidence=0.8, user_id="u1") + + kept_ids = {fact["id"] for fact in memory["facts"]} + assert "stale_high" in kept_ids + assert "confirmed_recently" not in kept_ids + memory_path = dm._storage._get_memory_file_path("__default__", user_id="u1") + audit = (memory_path.parent / "agents" / "__default__" / ".metadata" / "eviction-audit.json").read_text() + assert '"disagrees": true' in audit + + +def test_clear_memory_removes_usage_and_eviction_sidecars(deermem_data_dir): + dm = DeerMem( + backend_config={ + "storage_path": str(deermem_data_dir), + "retrieval_adapter": "", + "max_facts": 10, + } + ) + for index in range(10): + dm.create_fact(f"query fact {index}", confidence=0.9, user_id="u1") + dm.search("query", user_id="u1") + dm.create_fact("capacity rejected", confidence=0.1, user_id="u1") + memory_path = dm._storage._get_memory_file_path("__default__", user_id="u1") + metadata_path = memory_path.parent / "agents" / "__default__" / ".metadata" + assert metadata_path.is_dir() + + dm.clear_memory(user_id="u1") + + assert not metadata_path.exists() + + def test_is_human_clarification_response_matches_host_read(): """The standalone mirror must agree with the host's read_human_input_response so hidden-message filtering doesn't diverge between production (host hook) and diff --git a/backend/tests/test_memory_eviction.py b/backend/tests/test_memory_eviction.py new file mode 100644 index 000000000..8e4165835 --- /dev/null +++ b/backend/tests/test_memory_eviction.py @@ -0,0 +1,400 @@ +"""Capacity-eviction policy tests for DeerMem facts.""" + +from __future__ import annotations + +import json +from datetime import UTC, datetime + +import pytest + +from deerflow.agents.memory.backends.deermem.deermem.config import DeerMemConfig +from deerflow.agents.memory.backends.deermem.deermem.core.eviction import ( + EVICTION_POLICY_CONFIDENCE, + EVICTION_POLICY_HYBRID_V1, + select_facts_for_capacity, +) +from deerflow.agents.memory.backends.deermem.deermem.core.paths import ( + agent_eviction_audit_path, + agent_usage_path, +) +from deerflow.agents.memory.backends.deermem.deermem.core.storage import FileMemoryStorage + +NOW = datetime(2026, 8, 13, tzinfo=UTC) + + +def _fact( + fact_id: str, + *, + confidence: float, + created_at: str = "2026-08-01T00:00:00Z", + category: str = "preference", + confirmed_at: str | None = None, +) -> dict[str, object]: + fact: dict[str, object] = { + "id": fact_id, + "content": fact_id, + "category": category, + "confidence": confidence, + "createdAt": created_at, + } + if confirmed_at is not None: + fact["lastConfirmedAt"] = confirmed_at + return fact + + +def test_confidence_policy_preserves_existing_ranking() -> None: + facts = [ + _fact("low", confidence=0.6), + _fact("high", confidence=0.9), + _fact("mid", confidence=0.7), + ] + + decision = select_facts_for_capacity( + facts, + max_facts=2, + policy=EVICTION_POLICY_CONFIDENCE, + now=NOW, + ) + + assert [fact["id"] for fact in decision.kept] == ["high", "mid"] + assert [item.fact_id for item in decision.evicted] == ["low"] + assert decision.policy == EVICTION_POLICY_CONFIDENCE + + +def test_confidence_policy_coerces_non_float_confidence() -> None: + facts = [ + {"id": "a", "confidence": None}, + {"id": "b", "confidence": "0.9"}, + {"id": "c", "confidence": 0.8}, + {"id": "d", "confidence": "high"}, + ] + + decision = select_facts_for_capacity( + facts, + max_facts=2, + policy=EVICTION_POLICY_CONFIDENCE, + now=NOW, + ) + + assert [fact["id"] for fact in decision.kept] == ["b", "c"] + + +@pytest.mark.parametrize( + "policy", + [EVICTION_POLICY_CONFIDENCE, EVICTION_POLICY_HYBRID_V1], +) +@pytest.mark.parametrize("max_facts", [2, 10]) +def test_capacity_policy_preserves_input_order_at_or_below_cap( + policy: str, + max_facts: int, +) -> None: + facts = [ + _fact("low", confidence=0.6), + _fact("high", confidence=0.9), + ] + + decision = select_facts_for_capacity( + facts, + max_facts=max_facts, + policy=policy, + now=NOW, + ) + + assert decision.kept == facts + assert decision.evicted == [] + + +def test_hybrid_policy_keeps_recently_confirmed_fact_over_stale_high_confidence() -> None: + facts = [ + _fact( + "stale_high", + confidence=0.9, + created_at="2026-02-14T00:00:00Z", + ), + _fact( + "recent_confirmed", + confidence=0.7, + created_at="2026-01-01T00:00:00Z", + confirmed_at="2026-08-06T00:00:00Z", + ), + ] + + decision = select_facts_for_capacity( + facts, + max_facts=1, + policy=EVICTION_POLICY_HYBRID_V1, + now=NOW, + ) + + assert [fact["id"] for fact in decision.kept] == ["recent_confirmed"] + evicted = decision.evicted[0] + assert evicted.fact_id == "stale_high" + assert evicted.components["confidence"] == pytest.approx(0.9) + assert evicted.components["confirmationFreshness"] < 0.13 + + +def test_hybrid_policy_uses_bounded_access_heat() -> None: + facts = [ + _fact("recalled", confidence=0.7), + _fact("unused", confidence=0.75), + ] + usage = { + "recalled": { + "accessHeat": 100_000, + "lastAccessedAt": "2026-08-13T00:00:00Z", + } + } + + decision = select_facts_for_capacity( + facts, + max_facts=1, + policy=EVICTION_POLICY_HYBRID_V1, + usage=usage, + now=NOW, + ) + + assert [fact["id"] for fact in decision.kept] == ["recalled"] + assert decision.scores["recalled"].components["accessHeat"] == pytest.approx(1.0) + + +def test_hybrid_policy_reserves_bounded_correction_capacity() -> None: + facts = [ + *[_fact(f"preference_{index}", confidence=0.99 - index / 100) for index in range(10)], + _fact("correction", confidence=0.1, category="correction"), + ] + + decision = select_facts_for_capacity( + facts, + max_facts=10, + policy=EVICTION_POLICY_HYBRID_V1, + now=NOW, + ) + + kept_ids = {fact["id"] for fact in decision.kept} + assert "correction" in kept_ids + assert len(kept_ids) == 10 + assert decision.reserved_correction_slots == 1 + + +def test_hybrid_policy_releases_unused_correction_slots() -> None: + facts = [_fact(f"preference_{index}", confidence=0.9 - index / 100) for index in range(11)] + + decision = select_facts_for_capacity( + facts, + max_facts=10, + policy=EVICTION_POLICY_HYBRID_V1, + now=NOW, + ) + + assert len(decision.kept) == 10 + assert decision.reserved_correction_slots == 0 + + +def test_hybrid_policy_handles_malformed_timestamps() -> None: + fact = _fact("malformed", confidence=0.8, created_at="not-a-date", confirmed_at="also-not-a-date") + + decision = select_facts_for_capacity( + [fact, _fact("valid", confidence=0.7)], + max_facts=1, + policy=EVICTION_POLICY_HYBRID_V1, + now=NOW, + ) + + assert len(decision.kept) == 1 + assert decision.scores["malformed"].components["confirmationFreshness"] == 0.0 + + +def test_hybrid_policy_config_is_opt_in() -> None: + default = DeerMemConfig() + hybrid = DeerMemConfig(fact_eviction_policy=EVICTION_POLICY_HYBRID_V1) + + assert default.fact_eviction_policy == EVICTION_POLICY_CONFIDENCE + assert hybrid.fact_eviction_policy == EVICTION_POLICY_HYBRID_V1 + assert hybrid.eviction_confidence_weight == pytest.approx(0.65) + assert hybrid.eviction_confirmation_weight == pytest.approx(0.25) + assert hybrid.eviction_access_weight == pytest.approx(0.10) + + +def test_file_storage_records_decayed_access_heat_without_touching_fact(tmp_path) -> None: + config = DeerMemConfig(storage_path=str(tmp_path), retrieval_adapter="") + storage = FileMemoryStorage(config) + memory_path = storage._get_memory_file_path("default", user_id="u") + first = datetime(2026, 8, 1, tzinfo=UTC) + second = datetime(2026, 8, 31, tzinfo=UTC) + assert storage.save( + {"version": "1.0", "facts": [_fact("fact_a", confidence=0.8)]}, + "default", + user_id="u", + ) + + storage.record_fact_accesses(["fact_a"], agent_name="default", user_id="u", accessed_at=first) + storage.record_fact_accesses(["fact_a"], agent_name="default", user_id="u", accessed_at=second) + + usage = storage.get_fact_usage(agent_name="default", user_id="u") + assert usage["fact_a"]["accessHeat"] == pytest.approx(1.5) + assert usage["fact_a"]["accessCount"] == 2 + assert agent_usage_path(memory_path, "default").is_file() + + +def test_file_storage_writes_bounded_metadata_only_eviction_audit(tmp_path) -> None: + config = DeerMemConfig( + storage_path=str(tmp_path), + retrieval_adapter="", + eviction_audit_max_entries=1, + ) + storage = FileMemoryStorage(config) + memory_path = storage._get_memory_file_path("default", user_id="u") + decision = select_facts_for_capacity( + [_fact("kept", confidence=0.9), _fact("evicted", confidence=0.1)], + max_facts=1, + policy=EVICTION_POLICY_HYBRID_V1, + now=NOW, + ) + + storage.record_capacity_eviction( + decision, + max_facts=1, + agent_name="default", + user_id="u", + occurred_at=NOW, + ) + storage.record_capacity_eviction( + decision, + max_facts=1, + agent_name="default", + user_id="u", + occurred_at=NOW, + ) + + events = json.loads(agent_eviction_audit_path(memory_path, "default").read_text()) + assert len(events) == 1 + assert events[0]["evicted"][0]["factId"] == "evicted" + assert "content" not in json.dumps(events[0]) + + +def test_clear_fact_metadata_recomputes_shadow_disagreement(tmp_path) -> None: + config = DeerMemConfig( + storage_path=str(tmp_path), + retrieval_adapter="", + ) + storage = FileMemoryStorage(config) + memory_path = storage._get_memory_file_path("default", user_id="u") + facts = [ + _fact("always_kept", confidence=1.0, confirmed_at="2026-08-13T00:00:00Z"), + _fact("stale_high", confidence=0.9, created_at="2025-01-01T00:00:00Z"), + _fact("recent_low", confidence=0.6, confirmed_at="2026-08-13T00:00:00Z"), + _fact("common_loser", confidence=0.1, created_at="2025-01-01T00:00:00Z"), + ] + actual = select_facts_for_capacity( + facts, + max_facts=2, + policy=EVICTION_POLICY_CONFIDENCE, + now=NOW, + ) + shadow = select_facts_for_capacity( + facts, + max_facts=2, + policy=EVICTION_POLICY_HYBRID_V1, + now=NOW, + ) + storage.record_capacity_eviction( + actual, + max_facts=2, + agent_name="default", + user_id="u", + occurred_at=NOW, + shadow_decision=shadow, + ) + + storage.clear_fact_metadata( + agent_name="default", + user_id="u", + fact_ids=["recent_low", "stale_high"], + ) + + events = json.loads(agent_eviction_audit_path(memory_path, "default").read_text()) + assert {item["factId"] for item in events[0]["evicted"]} == {"common_loser"} + assert set(events[0]["shadow"]["wouldEvict"]) == {"common_loser"} + assert events[0]["shadow"]["disagrees"] is False + + +def test_delete_fact_tolerates_malformed_eviction_audit(tmp_path) -> None: + config = DeerMemConfig( + storage_path=str(tmp_path), + retrieval_adapter="", + ) + storage = FileMemoryStorage(config) + storage.upsert_fact( + _fact("victim", confidence=0.8), + agent_name="default", + user_id="u", + ) + memory_path = storage._get_memory_file_path("default", user_id="u") + audit_path = agent_eviction_audit_path(memory_path, "default") + audit_path.parent.mkdir(parents=True, exist_ok=True) + audit_path.write_text( + json.dumps( + [ + { + "reason": "capacity", + "evicted": None, + "shadow": {"wouldEvict": ["victim"], "disagrees": True}, + } + ] + ) + ) + + storage.delete_fact("victim", agent_name="default", user_id="u") + + assert storage.get_fact("victim", agent_name="default", user_id="u") is None + assert not audit_path.exists() + + +def test_clear_fact_metadata_sanitizes_malformed_audit_fields(tmp_path) -> None: + config = DeerMemConfig( + storage_path=str(tmp_path), + retrieval_adapter="", + ) + storage = FileMemoryStorage(config) + memory_path = storage._get_memory_file_path("default", user_id="u") + audit_path = agent_eviction_audit_path(memory_path, "default") + audit_path.parent.mkdir(parents=True, exist_ok=True) + audit_path.write_text( + json.dumps( + [ + { + "reason": "capacity", + "evicted": [ + {"factId": "keep"}, + {"factId": "removed"}, + {"factId": []}, + "invalid", + ], + "shadow": { + "wouldEvict": ["keep", "removed", {"invalid": True}], + "disagrees": True, + }, + }, + { + "reason": "capacity", + "evicted": [{"factId": "keep_without_shadow"}], + "shadow": {"wouldEvict": None, "disagrees": True}, + }, + ] + ) + ) + + storage.clear_fact_metadata( + agent_name="default", + user_id="u", + fact_ids=["removed"], + ) + + events = json.loads(audit_path.read_text()) + assert events[0]["evicted"] == [{"factId": "keep"}] + assert events[0]["shadow"] == { + "wouldEvict": ["keep"], + "disagrees": False, + } + assert events[1]["evicted"] == [{"factId": "keep_without_shadow"}] + assert "shadow" not in events[1] diff --git a/backend/tests/test_memory_staleness_review.py b/backend/tests/test_memory_staleness_review.py index 1fe50a0ff..7ae0fd43d 100644 --- a/backend/tests/test_memory_staleness_review.py +++ b/backend/tests/test_memory_staleness_review.py @@ -351,6 +351,13 @@ class TestSelectStaleCandidates: assert len(candidates) == 1 assert candidates[0]["id"] == "f1" + def test_recent_explicit_confirmation_resets_review_clock(self): + fact = _make_fact("f1", days_ago=400, expected_valid_days=90) + fact["lastConfirmedAt"] = (datetime.now(UTC) - timedelta(days=7)).isoformat() + memory = _make_memory([fact]) + + assert _select_stale_candidates(memory, _memory_config(staleness_age_days=90)) == [] + def test_huge_evd_does_not_abort_selection(self): # A huge persisted expected_valid_days (10**400) makes the review window # unrepresentable in datetime arithmetic. The fact cannot yet be stale diff --git a/backend/tests/test_memory_tools.py b/backend/tests/test_memory_tools.py index c1ec8195e..6c5b26296 100644 --- a/backend/tests/test_memory_tools.py +++ b/backend/tests/test_memory_tools.py @@ -234,7 +234,7 @@ class TestMemoryAddTool: result_json = memory_add_tool.func(SimpleNamespace(context={}), "low confidence fact", confidence=0.1) result = json.loads(result_json) - assert result == {"error": "Fact was not stored because memory.max_facts kept higher-confidence facts"} + assert result == {"error": "Fact was not stored because the configured memory.max_facts capacity policy evicted it"} assert recorded == ["low confidence fact"] def test_uses_runtime_scope(self, monkeypatch): diff --git a/backend/tests/test_memory_updater.py b/backend/tests/test_memory_updater.py index c5a99ca70..1882bcd9c 100644 --- a/backend/tests/test_memory_updater.py +++ b/backend/tests/test_memory_updater.py @@ -157,6 +157,213 @@ def test_apply_updates_skips_whitespace_only_facts() -> None: assert all(fact["content"].strip() for fact in result["facts"]) +def test_apply_updates_reinforces_existing_fact_only_with_detected_signal() -> None: + updater = _make_updater(config=_memory_config(fact_eviction_policy="hybrid-v1")) + current_memory = _make_memory( + facts=[ + { + "id": "fact_preference", + "content": "User prefers concise answers", + "category": "preference", + "confidence": 0.8, + "createdAt": "2026-01-01T00:00:00Z", + "source": "thread-a", + } + ] + ) + update_data = { + "newFacts": [], + "factsToReinforce": [ + { + "id": "fact_preference", + "scope": "user", + "reason": "The user explicitly confirmed this preference", + } + ], + } + + without_signal = updater._apply_updates(copy.deepcopy(current_memory), update_data) + with_signal = updater._apply_updates( + copy.deepcopy(current_memory), + update_data, + signals=frozenset({"reinforcement"}), + ) + + assert "lastConfirmedAt" not in without_signal["facts"][0] + assert with_signal["facts"][0]["lastConfirmedAt"].endswith("Z") + assert with_signal["facts"][0]["confirmationCount"] == 1 + + confidence_only = _make_updater() + without_hybrid_tracking = confidence_only._apply_updates( + copy.deepcopy(current_memory), + update_data, + signals=frozenset({"reinforcement"}), + ) + assert "lastConfirmedAt" not in without_hybrid_tracking["facts"][0] + + +@pytest.mark.parametrize( + ("prior_count", "expected_count"), + [ + (3, 4), + (0, 1), + (True, 1), + (-1, 1), + ("3", 1), + ], +) +def test_apply_updates_normalizes_prior_confirmation_count(prior_count: object, expected_count: int) -> None: + updater = _make_updater(config=_memory_config(fact_eviction_policy="hybrid-v1")) + current_memory = _make_memory( + facts=[ + { + "id": "fact_preference", + "content": "User prefers concise answers", + "category": "preference", + "confidence": 0.8, + "createdAt": "2026-01-01T00:00:00Z", + "source": "thread-a", + "confirmationCount": prior_count, + } + ] + ) + update_data = { + "newFacts": [], + "factsToReinforce": [ + { + "id": "fact_preference", + "scope": "user", + "reason": "The user explicitly confirmed this preference", + } + ], + } + + result = updater._apply_updates( + current_memory, + update_data, + signals=frozenset({"reinforcement"}), + ) + + assert result["facts"][0]["confirmationCount"] == expected_count + + +def test_parse_memory_update_response_normalizes_reinforcement_entries() -> None: + parsed = _parse_memory_update_response( + '{"user":{},"history":{},"newFacts":[],"factsToReinforce":[{"id":" fact_a ","scope":"USER","reason":" explicit confirmation "},{"id":"fact_b","scope":"thread","reason":"one-off"},{"id":"","scope":"user","reason":"bad"}]}' + ) + + assert parsed["factsToReinforce"] == [ + {"id": "fact_a", "scope": "user", "reason": "explicit confirmation"}, + {"id": "fact_b", "scope": "thread", "reason": "one-off"}, + ] + + +def test_automatic_update_uses_hybrid_capacity_policy() -> None: + updater = _make_updater( + config=_memory_config( + fact_eviction_policy="hybrid-v1", + max_facts=10, + fact_confidence_threshold=0.7, + ) + ) + current_memory = _make_memory( + facts=[ + *[ + { + "id": f"high_{index}", + "content": f"high {index}", + "category": "preference", + "confidence": 0.99, + "createdAt": "2026-08-12T00:00:00Z", + } + for index in range(8) + ], + { + "id": "stale_high", + "content": "stale high confidence", + "category": "preference", + "confidence": 0.9, + "createdAt": "2025-01-01T00:00:00Z", + }, + { + "id": "confirmed_recently", + "content": "recently confirmed preference", + "category": "preference", + "confidence": 0.7, + "createdAt": "2025-01-01T00:00:00Z", + "lastConfirmedAt": "2026-08-12T00:00:00Z", + }, + ] + ) + + result = updater._apply_updates( + current_memory, + { + "newFacts": [ + { + "content": "new useful context", + "category": "context", + "confidence": 0.8, + **_DURABLE_USER_FACT, + } + ] + }, + agent_name="default", + ) + + kept_ids = {fact["id"] for fact in result["facts"]} + assert "confirmed_recently" in kept_ids + assert "stale_high" not in kept_ids + + +def test_confidence_capacity_does_not_read_usage_sidecar() -> None: + storage = _MemoryStorage() + storage.get_fact_usage = MagicMock(return_value={}) + updater = _make_updater( + config=_memory_config(max_facts=1), + storage=storage, + ) + + updater._select_for_capacity( + [ + {"id": "high", "confidence": 0.9}, + {"id": "low", "confidence": 0.8}, + ], + agent_name="default", + user_id="user-a", + ) + + storage.get_fact_usage.assert_not_called() + + +@pytest.mark.parametrize( + "config", + [ + _memory_config(max_facts=1, fact_eviction_policy="hybrid-v1"), + _memory_config(max_facts=1, fact_eviction_shadow_enabled=True), + ], + ids=["hybrid", "shadow"], +) +def test_hybrid_capacity_reads_usage_sidecar(config: DeerMemConfig) -> None: + storage = _MemoryStorage() + storage.get_fact_usage = MagicMock(return_value={}) + updater = _make_updater(config=config, storage=storage) + + updater._select_for_capacity( + [ + {"id": "high", "confidence": 0.9}, + {"id": "low", "confidence": 0.8}, + ], + agent_name="default", + user_id="user-a", + ) + + storage.get_fact_usage.assert_called_once_with( + agent_name="default", + user_id="user-a", + ) + + def test_prepare_update_prompt_preserves_non_ascii_memory_text() -> None: current_memory = _make_memory( facts=[ diff --git a/backend/tests/test_message_processing.py b/backend/tests/test_message_processing.py index d36654eb4..afa87a2f9 100644 --- a/backend/tests/test_message_processing.py +++ b/backend/tests/test_message_processing.py @@ -76,6 +76,11 @@ def test_detect_reinforcement_default_bundled(): assert detect_reinforcement(msgs) is True +def test_detect_reinforcement_explicit_durable_chinese_confirmation(): + msgs = [_human("对，我就是喜欢简洁回答，以后都保持这样。")] + assert detect_reinforcement(msgs) is True + + def test_detect_correction_patterns_override(): custom = [re.compile(r"zzz")] assert detect_correction([_human("zzz here")], patterns=custom) is True diff --git a/config.example.yaml b/config.example.yaml index 7533fda37..5df01467c 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -1769,6 +1769,26 @@ memory: # temperature: # optional max_facts: 100 # Maximum number of facts to store fact_confidence_threshold: 0.7 # Minimum confidence for storing facts + # Capacity eviction defaults to the historical confidence-only ranking. + # Opt in to hybrid-v1 to score confidence (65%), explicit-confirmation + # freshness (25%, 90-day half-life), and query-driven access heat (10%, + # 30-day half-life). Actual memory_search hits count; default injection does + # not. A bounded 10% correction reserve aligns storage with guaranteed + # correction injection. Confirmation detection is batch-level: it checks + # human messages among the last six filtered messages, while the LLM binds + # that signal to a factsToReinforce ID without a second correspondence + # check. Shadow mode audits hybrid disagreements without changing which + # facts confidence-only keeps. + fact_eviction_policy: confidence # confidence | hybrid-v1 + fact_eviction_shadow_enabled: false + eviction_confidence_weight: 0.65 + eviction_confirmation_weight: 0.25 + eviction_access_weight: 0.10 + eviction_confirmation_half_life_days: 90 + eviction_access_half_life_days: 30 + eviction_correction_reserved_fraction: 0.10 + eviction_correction_reserved_max: 10 + eviction_audit_max_entries: 200 # metadata-only events per user/agent scope; 0 disables max_injection_tokens: 2000 # Maximum tokens for memory injection # Token counting strategy for memory-injection budgeting: # tiktoken (default) - accurate, but the encoding BPE data may be