diff --git a/backend/AGENTS.md b/backend/AGENTS.md index 30e8d97b2..5e8f1fc01 100644 --- a/backend/AGENTS.md +++ b/backend/AGENTS.md @@ -580,7 +580,7 @@ The cached value is reused for both the blocking (`runs.wait`) and streaming (`_ - `memory.mode: tool` skips `MemoryMiddleware` and registers `memory_search`, `memory_add`, `memory_update`, and `memory_delete` on the agent. The model decides when to search, add, update, or delete facts; this is opt-in/experimental and should not be described as better than middleware mode without eval evidence. - Both modes share `FileMemoryStorage`, per-user/per-agent isolation, prompt injection, manual CRUD primitives, and the updater backend. - Middleware mode queue debounces (30s default), batches updates, deduplicates per-thread, applies updates atomically (temp file + rename) with cache invalidation, and skips duplicate fact content before append. -- Staleness pass (same LLM invocation as the regular updater, no extra API call): when `staleness_review_enabled` is `true` and at least `staleness_min_candidates` aged facts exist, `_select_stale_candidates` selects facts older than `staleness_age_days` that are not in `staleness_protected_categories` (default: `correction`), surfaces them in the prompt, and the LLM judges each as KEEP or REMOVE. `_apply_updates` enforces the guardrail unconditionally at apply time: it intersects the LLM-returned removal set with `_select_stale_candidates` output before applying the per-cycle cap (`staleness_max_removals_per_cycle`), so protected and non-aged facts can never be deleted regardless of model behavior or the feature flag setting. +- Staleness pass (same LLM invocation as the regular updater, no extra API call): when `staleness_review_enabled` is `true` and at least `staleness_min_candidates` aged facts exist, `_select_stale_candidates` selects facts older than their individual review window (`expected_valid_days`, or the global `staleness_age_days` fallback) that are not in `staleness_protected_categories` (default: `correction`), surfaces them in the prompt with a `valid:Nd` annotation, and the LLM judges each as KEEP, REMOVE, or EXTEND. REMOVE entries go in `staleFactsToRemove`; EXTEND entries go in `staleFactsToExtend` with an `extend_by_days` value, which sets the fact's `expected_valid_days` to `min(days_since_created + extend_by_days, staleness_max_extension_days)`. The LLM assigns `expected_valid_days` when creating a fact; it is clamped at write time to `staleness_age_days × staleness_max_lifetime_multiplier` (creation cap). `_apply_updates` enforces the guardrail unconditionally at apply time: it intersects both the removal and extension sets with `_select_stale_candidates` output before applying the per-cycle cap (`staleness_max_removals_per_cycle`), so protected and non-aged facts can never be targeted regardless of model behavior or the feature flag setting. Facts the LLM proposed for removal are excluded from extension even if the per-cycle cap prevented their actual deletion that cycle. Extensions use an absolute ceiling (`staleness_max_extension_days`) rather than the creation multiplier so a deliberate review decision can advance the window beyond the initial cap while preventing `timedelta` overflow from a malformed `extend_by_days`. - Consolidation pass (same LLM invocation as the regular updater, no extra API call): when `consolidation_enabled` is `true` and at least one category holds `consolidation_min_facts` or more facts, `_select_consolidation_candidates` identifies fragmented categories and surfaces at most `consolidation_max_groups_per_cycle` of them (largest first) in the prompt. The LLM decides which groups to merge and proposes a synthesised fact per group. `_apply_updates` enforces guardrails: source IDs must exist and must not overlap across groups, group size is capped at `consolidation_max_sources`, the merged fact's confidence cannot exceed the source maximum, and facts below `fact_confidence_threshold` are not written. - Next interaction injects selected facts + context into `` tags in the system prompt when `injection_enabled` is true. @@ -607,6 +607,8 @@ Focused regression coverage for the updater lives in `backend/tests/test_memory_ - `staleness_min_candidates` - Minimum aged candidates required to trigger a review cycle (default: 3; range: 1–50) - `staleness_max_removals_per_cycle` - Maximum facts removed in a single cycle; lowest-confidence entries are kept when the LLM requests more (default: 10; range: 1–50) - `staleness_protected_categories` - Fact categories that are never pruned by staleness review (default: `["correction"]`) +- `staleness_max_lifetime_multiplier` - Creation-time cap multiplier for a fact's LLM-assigned `expected_valid_days`: stored value is clamped to `staleness_age_days × multiplier` so the model cannot defer first review indefinitely (default: 20.0; range: 1.0–100.0). Default 20.0 (90 × 20 = 1800 d ≈ 5 years) is generous enough to support the very-stable prompt tier without needing multiple review cycles to escape the cap. +- `staleness_max_extension_days` - Absolute upper bound (in days) on `expected_valid_days` after a lifetime extension (`staleFactsToExtend`). Applied at write time as `min(days_since + extend_by, staleness_max_extension_days)`. Uses an absolute ceiling rather than the multiplier because extensions are deliberate review decisions; prevents `timedelta` overflow and LLM misfire from permanently deferring a fact (default: 3650 = 10 years; range: 90–36500). - `consolidation_enabled` - Enable memory consolidation (default: `true`; no extra API call — runs in the same LLM invocation as the normal memory update) - `consolidation_min_facts` - Minimum facts in a category to trigger consolidation review (default: 8; range: 3–30) - `consolidation_max_groups_per_cycle` - Maximum categories the LLM can merge in one cycle (default: 3; range: 1–10; also controls the LLM's prompt instruction) @@ -734,7 +736,7 @@ Config is env-driven like the others — `MonocleTracingConfig`, built in `get_t - `title` - Auto-title generation (enabled, max_words, max_chars, model_name; null model_name uses fast local fallback, explicit model_name uses the prompt_template LLM path) - `summarization` - Context summarization (enabled, trigger conditions, keep policy) - `subagents.enabled` - Master switch for subagent delegation -- `memory` - Memory system (enabled, storage_path, debounce_seconds, shutdown_flush_timeout_seconds, model_name, max_facts, fact_confidence_threshold, injection_enabled, max_injection_tokens, staleness_review_enabled, staleness_age_days, staleness_min_candidates, staleness_max_removals_per_cycle, staleness_protected_categories) +- `memory` - Memory system (enabled, storage_path, debounce_seconds, shutdown_flush_timeout_seconds, model_name, max_facts, fact_confidence_threshold, injection_enabled, max_injection_tokens, staleness_review_enabled, staleness_age_days, staleness_min_candidates, staleness_max_removals_per_cycle, staleness_protected_categories, staleness_max_lifetime_multiplier, staleness_max_extension_days) **`extensions_config.json`**: - `mcpServers` - Map of server name → config (enabled, type, command, args, env, url, headers, oauth, description, `routing`, `tools`, `tool_call_timeout`). `routing.mode="prefer"` emits `` prompt guidance; if `tool_search` defers the hinted tool, `McpRoutingMiddleware` can also auto-promote matching deferred schemas before the model call. It does not hard-disable other tools. diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/config.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/config.py index 54459823a..6e54a4493 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/config.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/config.py @@ -115,6 +115,39 @@ class DeerMemConfig(BaseModel): default_factory=lambda: ["correction"], description="Fact categories exempt from staleness review.", ) + staleness_max_lifetime_multiplier: float = Field( + default=20.0, + ge=1.0, + le=100.0, + description=( + "Creation-time cap multiplier for a fact's LLM-assigned " + "expected_valid_days. When a new fact is stored, its " + "expected_valid_days is clamped to " + "staleness_age_days * staleness_max_lifetime_multiplier so the " + "model cannot set an initial lifetime so long that the fact is " + "never re-evaluated. Default 20.0 (90 x 20 = 1800 d ~= 5 years) " + "is generous enough to support the 'very stable' prompt tier " + "(core skills, native language) without needing multiple review " + "cycles to escape the cap. Lifetime extensions (staleFactsToExtend) " + "are subject to staleness_max_extension_days instead." + ), + ) + staleness_max_extension_days: int = Field( + default=3650, + ge=90, + le=36500, + description=( + "Absolute upper bound (in days) on expected_valid_days after a " + "lifetime extension (staleFactsToExtend). Applied at write time " + "during staleness review: new_evd = min(days_since + extend_by, " + "staleness_max_extension_days). Separate from the creation-time " + "multiplier cap because extensions are deliberate recalibration " + "decisions and are not subject to the staleness_age_days scale. " + "The ceiling prevents a single LLM misfire from permanently " + "deferring a fact or causing timedelta overflow on the next " + "candidate-selection pass. Default 3650 (10 years)." + ), + ) # ── Memory consolidation ──────────────────────────────────────────── consolidation_enabled: bool = Field( default=False, diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py index 83a165201..d9dd14271 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py @@ -82,6 +82,16 @@ Memory Section Guidelines: * behavior: Working patterns, communication habits, problem-solving approaches * goal: Stated objectives, learning targets, project ambitions * correction: Explicit agent mistakes or user corrections, including the correct approach +- Fact lifetime (``expected_valid_days``, optional integer): + How many days before this fact should be reviewed for possible removal. + The system schedules review automatically; omit when uncertain. + * <= 14: highly transient - active bugs, immediate tasks, today's focus + * 15-60: short-term - current experiments, in-progress side projects, near-term goals + * 60-180: medium-term - current role, active tech stack, ongoing preferences + * 180-365: stable - professional background, established working patterns + * > 365: very stable - core skills, native language, personality traits + Assign the value that semantically fits; values above the server-configured + ceiling are silently reduced on storage. - Confidence levels: * 0.9-1.0: Explicitly stated facts ("I work on X", "My role is Y") * 0.7-0.8: Strongly implied from actions/discussions @@ -114,10 +124,11 @@ Output Format (JSON): "longTermBackground": {{ "summary": "...", "shouldUpdate": true/false }} }}, "newFacts": [ - {{ "content": "...", "category": "preference|knowledge|context|behavior|goal|correction", "confidence": 0.0-1.0 }} + {{ "content": "...", "category": "preference|knowledge|context|behavior|goal|correction", "confidence": 0.0-1.0, "expected_valid_days": 90 }} ], "factsToRemove": ["fact_id_1", "fact_id_2"], "staleFactsToRemove": [{{ "id": "fact_id", "reason": "brief explanation" }}], + "staleFactsToExtend": [{{ "id": "fact_id", "extend_by_days": 365, "reason": "brief explanation" }}], "factsToConsolidate": [ {{ "sourceIds": ["fact_id_1", "fact_id_2"], @@ -155,27 +166,37 @@ Return ONLY valid JSON, no explanation or markdown.""" # rather than relying on passive contradiction from the current conversation. STALENESS_REVIEW_PROMPT = """## Staleness Review -The following facts were created more than {age_days} days ago and may no longer -accurately reflect the user's current situation. Review each one against the full -conversation context and your understanding of the user. +The following facts have reached their individual review window and may no longer +accurately reflect the user's current situation. Each entry shows a ``valid:Nd`` +annotation - the number of days this fact was expected to remain valid before +re-evaluation. Use it to calibrate conservatism: a ``valid:30d`` fact was +considered volatile at creation; a ``valid:365d`` fact was considered stable. {stale_facts} -For each fact, decide KEEP or REMOVE: -- KEEP: Still likely valid — even if not mentioned in this conversation. +For each fact, decide KEEP, REMOVE, or EXTEND: +- KEEP: Still likely valid - even if not mentioned in this conversation. Stable attributes (native language, core expertise, personality traits) often remain true indefinitely. - REMOVE: Outdated, contradicted by recent context, or no longer relevant. Examples: tech-stack migrations, job changes, relocated offices, abandoned projects. +- EXTEND: Keep but recalibrate the review window (see below). Add REMOVE decisions to "staleFactsToRemove" in your output JSON. Each entry must be {{"id": "fact_id", "reason": "brief explanation"}}. The reason should cite what signal in the conversation (or absence thereof) supports the removal. -Be conservative — when in doubt, KEEP. Removing a valid fact is worse than +Optionally, for facts you KEEP and wish to recalibrate, add them to +"staleFactsToExtend" with the number of days from now before the next review: +{{"id": "fact_id", "extend_by_days": 365, "reason": "brief explanation"}} +Use this when the current window seems miscalibrated - e.g. a core skill marked +``valid:30d`` that is clearly stable, or a goal nearing completion whose window +should shrink. Omit facts whose current window already seems appropriate. + +Be conservative - when in doubt, KEEP. Removing a valid fact is worse than keeping a slightly stale one, because the next review cycle will re-evaluate it.""" diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py index 894cc7a4b..1e43c62d2 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py @@ -171,6 +171,15 @@ def _normalize_memory_update_fact(fact: Any) -> dict[str, Any] | None: if normalized_source_error: normalized_fact["sourceError"] = normalized_source_error + # Fact lifetime (expected_valid_days): optional LLM-assigned review window. + # Accept int/float (reject bool which subclasses int), coerce to int, keep + # only positive values; the creation-time cap is applied in _apply_updates. + raw_evd = fact.get("expected_valid_days") + if isinstance(raw_evd, (int, float)) and not isinstance(raw_evd, bool): + evd = int(raw_evd) + if evd > 0: + normalized_fact["expected_valid_days"] = evd + return normalized_fact @@ -216,6 +225,32 @@ def _normalize_memory_update_data(update_data: dict[str, Any]) -> dict[str, Any] } ) + # ── Normalize staleness review lifetime extensions ── + stale_extensions_raw = update_data.get("staleFactsToExtend") + normalized_stale_extensions: list[dict[str, Any]] = [] + if isinstance(stale_extensions_raw, list): + for entry in stale_extensions_raw: + if not isinstance(entry, dict): + continue + fact_id = entry.get("id") + if not isinstance(fact_id, str) or not fact_id: + continue + # extend_by_days: accept int/float (reject bool), coerce to int, keep > 0. + # A fractional value in (0, 1) coerces to 0 and is dropped here so the + # apply path never silently writes a zero-delta extension. + raw_extend = entry.get("extend_by_days") + if isinstance(raw_extend, (int, float)) and not isinstance(raw_extend, bool): + extend_by = int(raw_extend) + if extend_by > 0: + reason = entry.get("reason", "") + normalized_stale_extensions.append( + { + "id": fact_id, + "extend_by_days": extend_by, + "reason": reason if isinstance(reason, str) else "", + } + ) + # ── Normalize consolidation decisions ── consolidation_raw = update_data.get("factsToConsolidate") normalized_consolidation: list[dict[str, Any]] = [] @@ -265,6 +300,7 @@ def _normalize_memory_update_data(update_data: dict[str, Any]) -> dict[str, Any] "newFacts": normalized_new_facts, "factsToRemove": normalized_facts_to_remove, "staleFactsToRemove": normalized_stale_removals, + "staleFactsToExtend": normalized_stale_extensions, "factsToConsolidate": normalized_consolidation, } @@ -357,16 +393,38 @@ def _parse_fact_datetime(raw: str) -> datetime | None: return None +def _effective_fact_staleness_age(fact: dict[str, Any], config: Any) -> int: + """Return the effective staleness review age in days for *fact*. + + Returns the stored ``expected_valid_days`` value directly when present and + valid. The ``staleness_max_lifetime_multiplier`` cap is applied once at + *write time* (when a fact is first created) so the review window is bounded + from the start. Re-applying it here would prevent lifetime-extension + operations from ever moving the review window beyond that initial cap, + defeating the purpose of ``staleFactsToExtend``. Falls back to the global + ``staleness_age_days`` for facts that pre-date this feature or where the + LLM did not provide an estimate. + """ + raw = fact.get("expected_valid_days") + if isinstance(raw, (int, float)) and not isinstance(raw, bool) and raw > 0: + return int(raw) + return config.staleness_age_days + + def _select_stale_candidates( current_memory: dict[str, Any], config: Any, ) -> list[dict[str, Any]]: - """Return facts that are older than ``staleness_age_days`` and not protected. + """Return facts that have exceeded their individual review window. - Protected categories (default: ``correction``) are excluded because they - represent explicit user feedback that should not be auto-pruned by age. + Each fact's effective review age is determined by + ``_effective_fact_staleness_age``: facts with an LLM-assigned + ``expected_valid_days`` use that value directly; facts without it fall back + to the global ``staleness_age_days``. Protected categories (default: + ``correction``) are excluded because they represent explicit user feedback + that should not be auto-pruned by age. """ - cutoff = datetime.now(UTC) - timedelta(days=config.staleness_age_days) + now = datetime.now(UTC) protected = frozenset(config.staleness_protected_categories) candidates: list[dict[str, Any]] = [] for fact in current_memory.get("facts", []): @@ -376,31 +434,41 @@ def _select_stale_candidates( if isinstance(category, str) and category in protected: continue created_at = _parse_fact_datetime(fact.get("createdAt", "")) - if created_at is not None and created_at < cutoff: + if created_at is None: + continue + effective_age = _effective_fact_staleness_age(fact, config) + if created_at < now - timedelta(days=effective_age): candidates.append(fact) return candidates def _build_staleness_section( stale_candidates: list[dict[str, Any]], - age_days: int, + config: Any, ) -> str: - """Format the staleness review prompt section from candidate facts.""" + """Format the staleness review prompt section from candidate facts. + + Each fact line includes a ``valid:Nd`` annotation - the effective review + window for that fact - so the LLM can calibrate its conservatism: a fact + reviewed after 30 days was considered volatile at creation; one reviewed + after 365 days was considered stable. + """ if not stale_candidates: return "" lines: list[str] = [] for fact in stale_candidates: fid = fact.get("id", "?") - cat = html.escape(str(fact.get("category", "context")).strip() or "context") + cat = html.escape(str(fact.get("category", "context")).strip() or "context", quote=False) conf = _coerce_source_confidence(fact) created_raw = fact.get("createdAt", "") created_short = created_raw[:10] if isinstance(created_raw, str) and len(created_raw) >= 10 else created_raw - content = html.escape(str(fact.get("content", ""))) - lines.append(f'- [{fid} | {cat} | {conf:.2f} | {created_short}] "{content}"') - return STALENESS_REVIEW_PROMPT.format( - stale_facts="\n".join(lines), - age_days=age_days, - ) + # quote=False: content is in element-text position (inside + # tags, never an attribute value), so only <, >, & can break structure - + # leave ' and " untouched. Mirrors the convention in prompt.py #4028. + content = html.escape(str(fact.get("content", "")), quote=False) + effective_age = _effective_fact_staleness_age(fact, config) + lines.append(f'- [{fid} | {cat} | {conf:.2f} | {created_short} | valid:{effective_age}d] "{content}"') + return STALENESS_REVIEW_PROMPT.format(stale_facts="\n".join(lines)) # ── Consolidation helpers ─────────────────────────────────────────────── @@ -670,10 +738,7 @@ class MemoryUpdater: if config.staleness_review_enabled: stale_candidates = _select_stale_candidates(current_memory, config) if len(stale_candidates) >= config.staleness_min_candidates: - staleness_section = _build_staleness_section( - stale_candidates, - config.staleness_age_days, - ) + staleness_section = _build_staleness_section(stale_candidates, config) # ── Build consolidation section ── consolidation_section = "" @@ -953,48 +1018,101 @@ class MemoryUpdater: if facts_to_remove: current_memory["facts"] = [f for f in current_memory.get("facts", []) if f.get("id") not in facts_to_remove] - # ── Staleness review removals ── + # ── Staleness review: removals + lifetime extensions ── + # Both operations share one staleness-candidate guardrail pass and one + # candidate_ids set. proposed_remove_ids is hoisted out of the removals + # sub-block so it covers ALL LLM-proposed removals, not just the ones + # the per-cycle cap actually deleted: a fact the LLM wanted to remove + # must never be silently extended even when the cap spares it. stale_removals = update_data.get("staleFactsToRemove", []) - if isinstance(stale_removals, list) and stale_removals: - stale_ids_to_remove = {entry["id"] for entry in stale_removals if isinstance(entry, dict) and "id" in entry} - - # Deterministic guardrail: intersect with actual staleness - # candidates so an LLM slip that emits a protected-category or - # non-aged fact id is silently rejected. Runs unconditionally - # so the apply-layer protection is independent of model behavior - # AND of the staleness_review_enabled flag. - # Guard against legacy / hand-edited facts that predate the id - # field: an aged, non-protected fact with no "id" is a valid - # staleness candidate but has no id to intersect against, so skip - # it here instead of raising KeyError (id-less facts can never be - # targeted by the id-based removal set anyway). + stale_extensions = update_data.get("staleFactsToExtend", []) + has_staleness_ops = (isinstance(stale_removals, list) and stale_removals) or (isinstance(stale_extensions, list) and stale_extensions) + if has_staleness_ops: + # Deterministic guardrail: intersect with actual staleness candidates + # so an LLM slip that emits a protected-category or non-aged fact id + # is silently rejected. Runs unconditionally so the apply-layer + # protection is independent of model behavior AND of the + # staleness_review_enabled flag. Guard against legacy / hand-edited + # facts that predate the id field: an aged, non-protected fact with + # no "id" is a valid staleness candidate but has no id to intersect + # against, so skip it here instead of raising KeyError. candidate_ids = {f["id"] for f in _select_stale_candidates(current_memory, config) if f.get("id") is not None} - stale_ids_to_remove &= candidate_ids - if not stale_ids_to_remove: - # After intersection with candidate set, nothing to remove. - stale_removals = [] - else: - # Safety cap: limit max staleness removals per cycle. - # When the LLM returns more than the cap, keep only the - # lowest-confidence entries up to the limit so the most - # questionable facts are removed first. - max_stale = config.staleness_max_removals_per_cycle - if len(stale_ids_to_remove) > max_stale: - stale_facts = [f for f in current_memory.get("facts", []) if f.get("id") in stale_ids_to_remove] - stale_facts.sort(key=_coerce_source_confidence) - stale_ids_to_remove = {f["id"] for f in stale_facts[:max_stale]} + # ── Removals ── + proposed_remove_ids: set[str] = set() + if isinstance(stale_removals, list) and stale_removals: + proposed_remove_ids = {entry["id"] for entry in stale_removals if isinstance(entry, dict) and "id" in entry} + stale_ids_to_remove = proposed_remove_ids & candidate_ids - current_memory["facts"] = [f for f in current_memory.get("facts", []) if f.get("id") not in stale_ids_to_remove] + if not stale_ids_to_remove: + stale_removals = [] + else: + # Safety cap: limit max staleness removals per cycle. When + # the LLM returns more than the cap, keep only the + # lowest-confidence entries up to the limit so the most + # questionable facts are removed first. + max_stale = config.staleness_max_removals_per_cycle + if len(stale_ids_to_remove) > max_stale: + stale_facts = [f for f in current_memory.get("facts", []) if f.get("id") in stale_ids_to_remove] + stale_facts.sort(key=_coerce_source_confidence) + stale_ids_to_remove = {f["id"] for f in stale_facts[:max_stale]} - # Log removals for observability - for entry in stale_removals: - if isinstance(entry, dict) and entry.get("id") in stale_ids_to_remove: - logger.info( - "Staleness review removed fact %s: %s", - entry["id"], - entry.get("reason", "no reason provided"), - ) + current_memory["facts"] = [f for f in current_memory.get("facts", []) if f.get("id") not in stale_ids_to_remove] + + # Log removals for observability + for entry in stale_removals: + if isinstance(entry, dict) and entry.get("id") in stale_ids_to_remove: + logger.info( + "Staleness review removed fact %s: %s", + entry["id"], + entry.get("reason", "no reason provided"), + ) + + # ── Lifetime extensions ── + # Recalibrate expected_valid_days for facts the LLM chose to keep. + # Eligible facts are stale candidates that the LLM did NOT propose + # for removal - including those that survived only because the + # per-cycle cap prevented their deletion. The new window is + # min(days_since + extend_by_days, staleness_max_extension_days). + # Extensions use an absolute ceiling rather than the creation-time + # multiplier cap: they are deliberate review decisions and must be + # able to advance the window beyond the original creation cap, but + # an absolute bound prevents timedelta overflow and LLM misfire. + if isinstance(stale_extensions, list) and stale_extensions: + # Exclude all LLM-proposed removals, not just the trimmed set, + # so a cap-surviving proposed-removal fact is never extended. + extendable_ids = candidate_ids - proposed_remove_ids + ext_by_id = {e["id"]: e for e in stale_extensions if isinstance(e, dict) and isinstance(e.get("id"), str) and e["id"] in extendable_ids} + if ext_by_id: + now_utc = datetime.now(UTC) + max_ext = config.staleness_max_extension_days + updated_facts: list[dict[str, Any]] = [] + for fact in current_memory.get("facts", []): + fid = fact.get("id") + ext = ext_by_id.get(fid) if fid else None + if ext is not None: + extend_by = ext.get("extend_by_days") + if isinstance(extend_by, (int, float)) and not isinstance(extend_by, bool): + extend_by_int = int(extend_by) # coerce before guard + if extend_by_int > 0: + created = _parse_fact_datetime(fact.get("createdAt", "")) + if created is None: + # Unreachable: _select_stale_candidates already + # excludes facts with unparseable createdAt. + updated_facts.append(fact) + continue + days_since = int((now_utc - created).total_seconds() // 86400) + new_evd = min(days_since + extend_by_int, max_ext) + fact = {**fact, "expected_valid_days": new_evd} + logger.info( + "Staleness review extended fact %s by %d days (new expected_valid_days: %d): %s", + fid, + extend_by_int, + new_evd, + ext.get("reason", "no reason provided"), + ) + updated_facts.append(fact) + current_memory["facts"] = updated_facts # Add new facts existing_fact_keys = {fact_key for fact_key in (_fact_content_key(fact.get("content")) for fact in current_memory.get("facts", [])) if fact_key is not None} @@ -1028,6 +1146,15 @@ class MemoryUpdater: normalized_source_error = source_error.strip() if normalized_source_error: fact_entry["sourceError"] = normalized_source_error + evd = fact.get("expected_valid_days") + if isinstance(evd, int) and not isinstance(evd, bool) and evd > 0: + # Apply the creation-time cap so the LLM cannot assign an + # unbounded lifetime that defers staleness review indefinitely. + # Extensions (staleFactsToExtend) bypass this cap via their own + # staleness_max_extension_days ceiling because they represent a + # deliberate review decision, not an unchecked initial assignment. + creation_cap = int(config.staleness_age_days * config.staleness_max_lifetime_multiplier) + fact_entry["expected_valid_days"] = min(evd, creation_cap) current_memory["facts"].append(fact_entry) if fact_key is not None: existing_fact_keys.add(fact_key) diff --git a/backend/packages/harness/deerflow/config/memory_config.py b/backend/packages/harness/deerflow/config/memory_config.py index 80fb4e4bf..b0ad30e73 100644 --- a/backend/packages/harness/deerflow/config/memory_config.py +++ b/backend/packages/harness/deerflow/config/memory_config.py @@ -41,6 +41,8 @@ _LEGACY_DEERMEM_FIELDS = frozenset( "staleness_min_candidates", "staleness_max_removals_per_cycle", "staleness_protected_categories", + "staleness_max_lifetime_multiplier", + "staleness_max_extension_days", "consolidation_enabled", "consolidation_min_facts", "consolidation_max_groups_per_cycle", diff --git a/backend/tests/test_memory_staleness_review.py b/backend/tests/test_memory_staleness_review.py index b0c9eb034..330d3dc7a 100644 --- a/backend/tests/test_memory_staleness_review.py +++ b/backend/tests/test_memory_staleness_review.py @@ -1,55 +1,72 @@ """Tests for the staleness review feature in the memory updater. Covers: -- Candidate selection (age threshold, protected categories) +- Candidate selection (per-fact expected_valid_days + global fallback, protected categories) - Trigger conditions (min candidates, enabled flag) -- Prompt section formatting +- Prompt section formatting (valid:Nd annotation, html escaping) - Staleness removal in _apply_updates (safety cap, observability) -- Normalization of staleFactsToRemove from LLM responses +- Lifetime extensions (staleFactsToExtend) with staleness_max_extension_days cap +- Creation-time cap (staleness_max_lifetime_multiplier) on new-fact expected_valid_days +- Normalization of staleFactsToRemove / staleFactsToExtend from LLM responses - Integration with _prepare_update_prompt + +The memory module was refactored into a pluggable backend (#4122): staleness +config now lives on ``DeerMemConfig`` and ``MemoryUpdater`` is dependency-injected +``(config, storage, llm)``. These tests construct the updater via DI rather than +patching ``get_memory_config``. """ from datetime import UTC, datetime, timedelta -from unittest.mock import MagicMock, patch +from unittest.mock import MagicMock -import pytest - -pytest.skip( - "Pending full DI migration: staleness config is now on DeerMemConfig (not " - "MemoryConfig); _apply_updates/_select_stale_candidates read self._config. " - "Staleness behavior is covered via DeerMem public API in test_deermem_self_contained.py. " - "Full unit-test migration is a follow-up.", - allow_module_level=True, -) - -from deerflow.agents.memory.backends.deermem.deermem.core.updater import ( # noqa: E402 +from deerflow.agents.memory.backends.deermem.deermem.config import DeerMemConfig +from deerflow.agents.memory.backends.deermem.deermem.core.updater import ( MemoryUpdater, _build_staleness_section, + _effective_fact_staleness_age, _normalize_memory_update_data, _parse_fact_datetime, _select_stale_candidates, ) -from deerflow.config.memory_config import MemoryConfig # noqa: E402 # ── Helpers ──────────────────────────────────────────────────────────────── -def _memory_config(**overrides: object) -> MemoryConfig: - config = MemoryConfig() +def _memory_config(**overrides: object) -> DeerMemConfig: + config = DeerMemConfig() for key, value in overrides.items(): setattr(config, key, value) return config +class _FakeStorage: + """Minimal in-memory storage stub for DI - load() returns a held dict.""" + + def __init__(self, memory: dict | None = None) -> None: + self._memory = memory or _make_memory([]) + + def load(self, agent_name: str | None = None, *, user_id: str | None = None) -> dict: + return self._memory + + def save(self, memory_data: dict, agent_name: str | None = None, *, user_id: str | None = None) -> bool: + self._memory = memory_data + return True + + +def _make_updater(config: DeerMemConfig | None = None, memory: dict | None = None) -> MemoryUpdater: + return MemoryUpdater(config or _memory_config(), _FakeStorage(memory), llm=None) + + def _make_fact( fact_id: str, content: str = "test content", category: str = "knowledge", confidence: float = 0.9, days_ago: int = 100, + expected_valid_days: int | None = None, ) -> dict: created = (datetime.now(UTC) - timedelta(days=days_ago)).isoformat().replace("+00:00", "Z") - return { + fact: dict = { "id": fact_id, "content": content, "category": category, @@ -57,6 +74,9 @@ def _make_fact( "createdAt": created, "source": "thread-test", } + if expected_valid_days is not None: + fact["expected_valid_days"] = expected_valid_days + return fact def _make_memory(facts: list[dict] | None = None) -> dict: @@ -96,14 +116,53 @@ class TestParseFactDatetime: assert _parse_fact_datetime("") is None def test_invalid_format(self): - assert _parse_fact_datetime("not-a-date") is None + assert _parse_fact_datetime("not a date") is None def test_naive_datetime_gets_utc(self): - """Naive datetime (no tzinfo) should be treated as UTC, not cause TypeError.""" result = _parse_fact_datetime("2025-06-01T12:00:00") assert result is not None assert result.tzinfo is not None - assert result.utcoffset().total_seconds() == 0 + + +# ── _effective_fact_staleness_age ───────────────────────────────────────── + + +class TestEffectiveFactStalenessAge: + def test_returns_stored_evd_when_present(self): + fact = _make_fact("f1", days_ago=100, expected_valid_days=365) + config = _memory_config(staleness_age_days=90) + assert _effective_fact_staleness_age(fact, config) == 365 + + def test_falls_back_to_global_age_when_absent(self): + fact = _make_fact("f1", days_ago=100) + config = _memory_config(staleness_age_days=90) + assert _effective_fact_staleness_age(fact, config) == 90 + + def test_returns_raw_value_above_creation_cap(self): + # Read-time cap is removed; the stored value is returned directly even + # when it exceeds staleness_age_days * multiplier (creation cap only). + fact = _make_fact("f1", days_ago=100, expected_valid_days=999) + config = _memory_config(staleness_age_days=90, staleness_max_lifetime_multiplier=3.0) + assert _effective_fact_staleness_age(fact, config) == 999 + + def test_accepts_float_from_hand_edited_memory(self): + fact = _make_fact("f1", days_ago=100) + fact["expected_valid_days"] = 180.7 + config = _memory_config(staleness_age_days=90) + assert _effective_fact_staleness_age(fact, config) == 180 + + def test_ignores_bool(self): + fact = _make_fact("f1", days_ago=100) + fact["expected_valid_days"] = True + config = _memory_config(staleness_age_days=90) + assert _effective_fact_staleness_age(fact, config) == 90 + + def test_ignores_zero_and_negative(self): + config = _memory_config(staleness_age_days=90) + for bad in (0, -5): + fact = _make_fact("f1", days_ago=100) + fact["expected_valid_days"] = bad + assert _effective_fact_staleness_age(fact, config) == 90 # ── _select_stale_candidates ────────────────────────────────────────────── @@ -111,38 +170,25 @@ class TestParseFactDatetime: class TestSelectStaleCandidates: def test_old_facts_selected(self): - memory = _make_memory( - [ - _make_fact("fact_old", days_ago=100), - _make_fact("fact_new", days_ago=10), - ] - ) + memory = _make_memory([_make_fact("fact_old", days_ago=100), _make_fact("fact_new", days_ago=10)]) config = _memory_config(staleness_age_days=90) candidates = _select_stale_candidates(memory, config) assert len(candidates) == 1 assert candidates[0]["id"] == "fact_old" def test_protected_category_excluded(self): - memory = _make_memory( - [ - _make_fact("fact_correction", category="correction", days_ago=200), - _make_fact("fact_knowledge", category="knowledge", days_ago=200), - ] - ) + memory = _make_memory([_make_fact("fact_old", category="correction", days_ago=100), _make_fact("fact_norm", days_ago=100)]) config = _memory_config(staleness_age_days=90, staleness_protected_categories=["correction"]) candidates = _select_stale_candidates(memory, config) assert len(candidates) == 1 - assert candidates[0]["id"] == "fact_knowledge" + assert candidates[0]["id"] == "fact_norm" def test_custom_protected_categories(self): - memory = _make_memory( - [ - _make_fact("fact_goal", category="goal", days_ago=200), - ] - ) + memory = _make_memory([_make_fact("fact_goal", category="goal", days_ago=100), _make_fact("fact_know", days_ago=100)]) config = _memory_config(staleness_age_days=90, staleness_protected_categories=["goal"]) candidates = _select_stale_candidates(memory, config) - assert len(candidates) == 0 + assert len(candidates) == 1 + assert candidates[0]["id"] == "fact_know" def test_no_facts(self): memory = _make_memory([]) @@ -150,109 +196,111 @@ class TestSelectStaleCandidates: assert _select_stale_candidates(memory, config) == [] def test_all_recent(self): - memory = _make_memory( - [ - _make_fact("fact_a", days_ago=10), - _make_fact("fact_b", days_ago=30), - ] - ) + memory = _make_memory([_make_fact("fact_a", days_ago=10), _make_fact("fact_b", days_ago=20)]) config = _memory_config(staleness_age_days=90) assert _select_stale_candidates(memory, config) == [] + def test_fact_within_evd_not_selected(self): + # days_ago=300 but evd=999 -> within its own review window, not stale. + memory = _make_memory([_make_fact("f1", days_ago=300, expected_valid_days=999)]) + config = _memory_config(staleness_age_days=90) + assert _select_stale_candidates(memory, config) == [] + + def test_fact_past_evd_is_selected(self): + memory = _make_memory([_make_fact("f1", days_ago=400, expected_valid_days=365)]) + config = _memory_config(staleness_age_days=90) + candidates = _select_stale_candidates(memory, config) + assert len(candidates) == 1 + assert candidates[0]["id"] == "f1" + # ── Trigger conditions via _select_stale_candidates + config ───────────── class TestStalenessTriggerConditions: - """The old _should_run_staleness_review was removed; trigger logic is now - inlined in _prepare_update_prompt. We verify the gating conditions here - through _select_stale_candidates + config flags directly.""" - - def test_disabled_means_no_section(self): - memory = _make_memory([_make_fact(f"f{i}", days_ago=100) for i in range(5)]) - config = _memory_config(staleness_review_enabled=False, staleness_age_days=90, staleness_min_candidates=3) - candidates = _select_stale_candidates(memory, config) - # Even though candidates exist, the caller checks enabled flag first - assert config.staleness_review_enabled is False - assert len(candidates) >= config.staleness_min_candidates + """Trigger logic is tested through _prepare_update_prompt, but the candidate + selection that drives it is exercised here directly.""" def test_below_min_candidates(self): - memory = _make_memory([_make_fact("fact_only", days_ago=100)]) + memory = _make_memory([_make_fact("fact_a", days_ago=100), _make_fact("fact_b", days_ago=100)]) config = _memory_config(staleness_review_enabled=True, staleness_age_days=90, staleness_min_candidates=3) candidates = _select_stale_candidates(memory, config) - assert len(candidates) < config.staleness_min_candidates + assert len(candidates) == 2 # below min_candidates -> caller won't build section def test_at_min_candidates(self): memory = _make_memory([_make_fact(f"fact_{i}", days_ago=100) for i in range(3)]) config = _memory_config(staleness_review_enabled=True, staleness_age_days=90, staleness_min_candidates=3) candidates = _select_stale_candidates(memory, config) - assert len(candidates) >= config.staleness_min_candidates + assert len(candidates) == 3 def test_above_min_candidates(self): - memory = _make_memory([_make_fact(f"fact_{i}", days_ago=100) for i in range(10)]) + memory = _make_memory([_make_fact(f"fact_{i}", days_ago=100) for i in range(5)]) config = _memory_config(staleness_review_enabled=True, staleness_age_days=90, staleness_min_candidates=3) candidates = _select_stale_candidates(memory, config) - assert len(candidates) >= config.staleness_min_candidates + assert len(candidates) == 5 -# ── _build_staleness_section ────────────────────────────────────────────── +# ── _build_staleness_section ───────────────────────────────────────────── class TestBuildStalenessSection: def test_empty_candidates(self): - assert _build_staleness_section([], 90) == "" + assert _build_staleness_section([], _memory_config()) == "" def test_includes_fact_details(self): - candidates = [ - _make_fact("fact_vue", "User uses Vue.js", "knowledge", 0.95, days_ago=120), - ] - section = _build_staleness_section(candidates, 90) - assert "fact_vue" in section - assert "User uses Vue.js" in section - assert "0.95" in section - assert "90 days" in section + candidates = [_make_fact("fact_1", "User knows Python", "knowledge", 0.9, days_ago=100)] + section = _build_staleness_section(candidates, _memory_config(staleness_age_days=90)) + assert "Staleness Review" in section + assert "" in section + assert "fact_1" in section + assert "User knows Python" in section + assert "valid:90d" in section # global fallback annotation def test_multiple_facts(self): candidates = [ - _make_fact("fact_a", "Fact A", "knowledge", 0.9, days_ago=100), - _make_fact("fact_b", "Fact B", "preference", 0.8, days_ago=150), + _make_fact("fact_1", "A", days_ago=100), + _make_fact("fact_2", "B", days_ago=200), ] - section = _build_staleness_section(candidates, 90) - assert "fact_a" in section - assert "fact_b" in section - assert "" in section + section = _build_staleness_section(candidates, _memory_config(staleness_age_days=90)) + assert "fact_1" in section + assert "fact_2" in section + + def test_valid_annotation_uses_stored_evd(self): + candidates = [_make_fact("f1", days_ago=100, expected_valid_days=365)] + section = _build_staleness_section(candidates, _memory_config(staleness_age_days=90)) + assert "valid:365d" in section def test_html_special_chars_in_content_are_escaped(self): - """Fact content with XML tags or quotes is HTML-escaped so it cannot - break the surrounding prompt structure.""" + """Fact content with XML special chars is escaped (quote=False: only + <, >, & break element-text structure; " and ' are left untouched, + consistent with the prompt.py convention).""" candidates = [ _make_fact("fact_x", 'Like bold & "quotes"', "knowledge", 0.9, days_ago=100), ] - section = _build_staleness_section(candidates, 90) + section = _build_staleness_section(candidates, _memory_config()) assert "" not in section assert "<b>" in section assert "&" in section - assert """ in section + assert '"quotes"' in section # " left unescaped in element-text position + assert """ not in section def test_closing_tag_in_content_is_escaped(self): - """A closing tag embedded in content must not prematurely - end the prompt XML block.""" candidates = [ _make_fact("fact_y", "bad", "knowledge", 0.8, days_ago=100), ] - section = _build_staleness_section(candidates, 90) + section = _build_staleness_section(candidates, _memory_config()) assert "" not in section assert "</stale_facts>" in section def test_special_chars_in_category_are_escaped(self): - """A category name with XML tags or quotes is HTML-escaped, consistent - with how category is handled in the consolidation section.""" + """Category name XML special chars are escaped; quote=False so only + <, >, & are escaped - " is left untouched in element-text position.""" candidates = [ _make_fact("fact_z", "content", 'pref<"erences>', 0.8, days_ago=100), ] - section = _build_staleness_section(candidates, 90) + section = _build_staleness_section(candidates, _memory_config()) assert 'pref<"erences>' not in section - assert "pref<"erences>" in section + assert 'pref<"erences>' in section # " left unescaped # ── _apply_updates with staleness removals ───────────────────────────────── @@ -260,7 +308,7 @@ class TestBuildStalenessSection: class TestApplyUpdatesStaleness: def test_stale_facts_removed(self): - updater = MemoryUpdater() + updater = _make_updater(_memory_config(max_facts=100, staleness_max_removals_per_cycle=10)) current_memory = _make_memory( [ _make_fact("fact_keep", "User knows Python", days_ago=100), @@ -277,29 +325,16 @@ class TestApplyUpdatesStaleness: ], } - with patch( - "deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", - return_value=_memory_config(max_facts=100, staleness_max_removals_per_cycle=10), - ): - result = updater._apply_updates(current_memory, update_data) + result = updater._apply_updates(current_memory, update_data) assert len(result["facts"]) == 1 assert result["facts"][0]["id"] == "fact_keep" def test_stale_candidate_without_id_does_not_raise(self): """A legacy / hand-edited fact that lacks an ``id`` must not crash the - staleness apply path. - - Regression: ``candidate_ids`` was built with a direct ``f["id"]`` - access over ``_select_stale_candidates`` output, but every other fact - access in the module uses ``f.get("id")``. An aged, non-protected fact - with no ``id`` key (common in legacy / migrated ``memory.json``) is a - valid staleness candidate, so it reached ``f["id"]`` and raised - ``KeyError: 'id'``, aborting the whole memory-update cycle. - """ - updater = MemoryUpdater() + staleness apply path.""" + updater = _make_updater(_memory_config(max_facts=100, staleness_max_removals_per_cycle=10)) aged = (datetime.now(UTC) - timedelta(days=120)).isoformat().replace("+00:00", "Z") - # An aged, non-protected fact deliberately missing the "id" key. idless_fact = {"content": "User uses Vue.js", "category": "knowledge", "confidence": 0.8, "createdAt": aged} current_memory = _make_memory([_make_fact("fact_keep", days_ago=100), idless_fact]) update_data = { @@ -312,21 +347,13 @@ class TestApplyUpdatesStaleness: ], } - with patch( - "deerflow.agents.memory.updater.get_memory_config", - return_value=_memory_config(max_facts=100, staleness_max_removals_per_cycle=10), - ): - # Must not raise KeyError: 'id'. - result = updater._apply_updates(current_memory, update_data) + result = updater._apply_updates(current_memory, update_data) # must not raise KeyError - # The id-less fact survives (it can never be targeted by the id-based - # removal set), and the id-based removal of fact_keep still applies. contents = {f.get("content") for f in result["facts"]} assert "User uses Vue.js" in contents def test_safety_cap_limits_removals(self): - updater = MemoryUpdater() - # 5 stale facts, but cap is 2 → only 2 lowest-confidence should be removed + updater = _make_updater(_memory_config(max_facts=100, staleness_max_removals_per_cycle=2)) current_memory = _make_memory( [ _make_fact("fact_high", confidence=0.95, days_ago=100), @@ -350,13 +377,8 @@ class TestApplyUpdatesStaleness: ], } - with patch( - "deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", - return_value=_memory_config(max_facts=100, staleness_max_removals_per_cycle=2), - ): - result = updater._apply_updates(current_memory, update_data) + result = updater._apply_updates(current_memory, update_data) - # 5 - 2 = 3 facts remain; the 2 lowest-confidence removed assert len(result["facts"]) == 3 remaining_ids = {f["id"] for f in result["facts"]} assert "fact_high" in remaining_ids @@ -364,12 +386,8 @@ class TestApplyUpdatesStaleness: assert "fact_low1" in remaining_ids def test_empty_stale_removals_no_effect(self): - updater = MemoryUpdater() - current_memory = _make_memory( - [ - _make_fact("fact_a", days_ago=100), - ] - ) + updater = _make_updater(_memory_config(max_facts=100)) + current_memory = _make_memory([_make_fact("fact_a", days_ago=100)]) update_data = { "user": {}, "history": {}, @@ -378,37 +396,26 @@ class TestApplyUpdatesStaleness: "staleFactsToRemove": [], } - with patch( - "deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", - return_value=_memory_config(max_facts=100), - ): - result = updater._apply_updates(current_memory, update_data) + result = updater._apply_updates(current_memory, update_data) assert len(result["facts"]) == 1 def test_missing_stale_removals_key_no_effect(self): - """When LLM doesn't return staleFactsToRemove, existing behavior is preserved.""" - updater = MemoryUpdater() + updater = _make_updater(_memory_config(max_facts=100)) current_memory = _make_memory([_make_fact("fact_a")]) update_data = { "user": {}, "history": {}, "newFacts": [], "factsToRemove": [], - # no staleFactsToRemove key } - with patch( - "deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", - return_value=_memory_config(max_facts=100), - ): - result = updater._apply_updates(current_memory, update_data) + result = updater._apply_updates(current_memory, update_data) assert len(result["facts"]) == 1 def test_contradiction_and_staleness_removals_combined(self): - """Both factsToRemove and staleFactsToRemove work together.""" - updater = MemoryUpdater() + updater = _make_updater(_memory_config(max_facts=100, staleness_max_removals_per_cycle=10)) current_memory = _make_memory( [ _make_fact("fact_keep", days_ago=10), @@ -424,20 +431,22 @@ class TestApplyUpdatesStaleness: "staleFactsToRemove": [{"id": "fact_stale", "reason": "old"}], } - with patch( - "deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", - return_value=_memory_config(max_facts=100, staleness_max_removals_per_cycle=10), - ): - result = updater._apply_updates(current_memory, update_data) + result = updater._apply_updates(current_memory, update_data) assert len(result["facts"]) == 1 assert result["facts"][0]["id"] == "fact_keep" def test_protected_category_fact_refused_at_apply(self): - """Regression: LLM hallucinating a correction-category fact id in - staleFactsToRemove must be silently rejected at the apply layer, - even though it appears in the serialized prompt JSON.""" - updater = MemoryUpdater() + updater = _make_updater( + _memory_config( + max_facts=100, + staleness_review_enabled=True, + staleness_age_days=90, + staleness_min_candidates=1, + staleness_max_removals_per_cycle=10, + staleness_protected_categories=["correction"], + ) + ) current_memory = _make_memory( [ _make_fact("fact_stale", category="knowledge", days_ago=200), @@ -455,27 +464,21 @@ class TestApplyUpdatesStaleness: ], } - with patch( - "deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", - return_value=_memory_config( + result = updater._apply_updates(current_memory, update_data) + + assert len(result["facts"]) == 1 + assert result["facts"][0]["id"] == "fact_correction" + + def test_non_aged_fact_refused_at_apply(self): + updater = _make_updater( + _memory_config( max_facts=100, staleness_review_enabled=True, staleness_age_days=90, staleness_min_candidates=1, staleness_max_removals_per_cycle=10, - staleness_protected_categories=["correction"], - ), - ): - result = updater._apply_updates(current_memory, update_data) - - # fact_stale removed, fact_correction kept (protected) - assert len(result["facts"]) == 1 - assert result["facts"][0]["id"] == "fact_correction" - - def test_non_aged_fact_refused_at_apply(self): - """Regression: LLM returning a fresh (non-aged) fact id in - staleFactsToRemove must be silently rejected.""" - updater = MemoryUpdater() + ) + ) current_memory = _make_memory( [ _make_fact("fact_stale", days_ago=200), @@ -489,36 +492,262 @@ class TestApplyUpdatesStaleness: "factsToRemove": [], "staleFactsToRemove": [ {"id": "fact_stale", "reason": "outdated"}, - {"id": "fact_fresh", "reason": "LLM hallucination"}, + {"id": "fact_fresh", "reason": "LLM slip"}, ], } - with patch( - "deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", - return_value=_memory_config( + result = updater._apply_updates(current_memory, update_data) + + # fact_stale removed, fact_fresh kept (not aged) + ids = {f["id"] for f in result["facts"]} + assert ids == {"fact_fresh"} + + def test_guardrail_runs_when_staleness_review_disabled(self): + """The apply-layer staleness guardrail runs unconditionally (independent + of the staleness_review_enabled flag) as defense-in-depth.""" + updater = _make_updater( + _memory_config( max_facts=100, - staleness_review_enabled=True, + staleness_review_enabled=False, staleness_age_days=90, staleness_min_candidates=1, staleness_max_removals_per_cycle=10, - staleness_protected_categories=["correction"], - ), - ): - result = updater._apply_updates(current_memory, update_data) - - # fact_stale removed, fact_fresh kept (not in candidate set) - assert len(result["facts"]) == 1 - assert result["facts"][0]["id"] == "fact_fresh" - - def test_guardrail_runs_when_staleness_review_disabled(self): - """Regression: guardrail must reject invalid ids even when - staleness_review_enabled=False, so the protection is independent - of the feature flag and model behavior.""" - updater = MemoryUpdater() + ) + ) current_memory = _make_memory( [ _make_fact("fact_stale", days_ago=200), - _make_fact("fact_fresh", days_ago=5), + ] + ) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [{"id": "fact_stale", "reason": "outdated"}], + } + + result = updater._apply_updates(current_memory, update_data) + + assert result["facts"] == [] + + +# ── _apply_updates: lifetime extensions (staleFactsToExtend) ────────────── + + +class TestApplyUpdatesStaleFactsExtend: + def test_extension_updates_expected_valid_days(self): + updater = _make_updater(_memory_config(max_facts=100, staleness_age_days=90, staleness_max_lifetime_multiplier=10.0)) + current_memory = _make_memory([_make_fact("fact_stable", days_ago=100)]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [], + "staleFactsToExtend": [{"id": "fact_stable", "extend_by_days": 365, "reason": "core skill"}], + } + + result = updater._apply_updates(current_memory, update_data) + + assert len(result["facts"]) == 1 + fact = result["facts"][0] + assert "expected_valid_days" in fact + # days_since ~= 100; new_evd = days_since + 365. Allow +/-1 to avoid + # flakiness if the test crosses a UTC midnight between _make_fact and + # _apply_updates. + assert abs(fact["expected_valid_days"] - (100 + 365)) <= 1 + + def test_extension_not_capped_by_multiplier(self): + # Extensions bypass the staleness_max_lifetime_multiplier creation cap. + # A large extend_by_days (9999) with multiplier=3.0 (creation cap=270) + # should NOT be clamped to 270. staleness_max_extension_days is set high + # enough (36500) to let the value through and isolate the multiplier check. + updater = _make_updater( + _memory_config( + max_facts=100, + staleness_age_days=90, + staleness_max_lifetime_multiplier=3.0, + staleness_max_extension_days=36500, + ) + ) + current_memory = _make_memory([_make_fact("fact_a", days_ago=100)]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [], + "staleFactsToExtend": [{"id": "fact_a", "extend_by_days": 9999}], + } + + result = updater._apply_updates(current_memory, update_data) + + fact = result["facts"][0] + expected = 100 + 9999 + assert abs(fact.get("expected_valid_days", 0) - expected) <= 1 + + def test_extension_capped_by_max_extension_days(self): + # staleness_max_extension_days provides an absolute safety ceiling for + # extensions, guarding against timedelta overflow and LLM misfires. + updater = _make_updater( + _memory_config( + max_facts=100, + staleness_age_days=90, + staleness_max_lifetime_multiplier=20.0, + staleness_max_extension_days=3650, + ) + ) + current_memory = _make_memory([_make_fact("fact_a", days_ago=100)]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [], + "staleFactsToExtend": [{"id": "fact_a", "extend_by_days": 9999}], + } + + result = updater._apply_updates(current_memory, update_data) + + fact = result["facts"][0] + # days_since + 9999 >> 3650, so the absolute cap kicks in + assert fact.get("expected_valid_days") == 3650 + + def test_huge_extend_by_days_does_not_overflow_next_cycle(self): + # Regression: before staleness_max_extension_days, an LLM-supplied + # extend_by_days=10**9 could store a value that later caused + # OverflowError in timedelta(days=...) during the next + # _select_stale_candidates call, permanently breaking memory updates. + cfg = _memory_config( + max_facts=100, + staleness_age_days=90, + staleness_max_lifetime_multiplier=20.0, + staleness_max_extension_days=3650, + ) + updater = _make_updater(cfg) + current_memory = _make_memory([_make_fact("fact_a", days_ago=100)]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [], + "staleFactsToExtend": [{"id": "fact_a", "extend_by_days": 10**9}], + } + + result = updater._apply_updates(current_memory, update_data) + + # Stored value must be within timedelta-safe range + assert result["facts"][0]["expected_valid_days"] == 3650 + # Next-cycle candidate selection must not raise OverflowError + candidates = _select_stale_candidates(result, cfg) + assert isinstance(candidates, list) + + def test_removed_fact_cannot_be_extended(self): + updater = _make_updater( + _memory_config( + max_facts=100, + staleness_age_days=90, + staleness_max_lifetime_multiplier=10.0, + staleness_max_removals_per_cycle=10, + ) + ) + current_memory = _make_memory( + [ + _make_fact("fact_gone", days_ago=150), + _make_fact("fact_kept", days_ago=150), + ] + ) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [{"id": "fact_gone", "reason": "outdated"}], + "staleFactsToExtend": [ + {"id": "fact_gone", "extend_by_days": 180}, # ignored: removed this cycle + {"id": "fact_kept", "extend_by_days": 180}, + ], + } + + result = updater._apply_updates(current_memory, update_data) + + ids = {f["id"] for f in result["facts"]} + assert "fact_gone" not in ids + assert "fact_kept" in ids + kept = next(f for f in result["facts"] if f["id"] == "fact_kept") + assert "expected_valid_days" in kept + + def test_non_candidate_fact_cannot_be_extended(self): + """A fresh (non-stale) fact must be rejected by the candidate guardrail.""" + updater = _make_updater(_memory_config(max_facts=100, staleness_age_days=90)) + current_memory = _make_memory([_make_fact("fact_fresh", days_ago=5)]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [], + "staleFactsToExtend": [{"id": "fact_fresh", "extend_by_days": 180}], + } + + result = updater._apply_updates(current_memory, update_data) + + assert "expected_valid_days" not in result["facts"][0] + + def test_empty_extensions_no_effect(self): + updater = _make_updater(_memory_config(max_facts=100, staleness_age_days=90)) + current_memory = _make_memory([_make_fact("fact_a", days_ago=100)]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [], + "staleFactsToExtend": [], + } + + result = updater._apply_updates(current_memory, update_data) + + assert len(result["facts"]) == 1 + assert "expected_valid_days" not in result["facts"][0] + + def test_fractional_extend_by_days_below_one_is_silently_skipped(self): + # extend_by_days=0.9 coerces to int 0, which is not a positive extension + # - must be rejected without writing a zero-delta expected_valid_days. + updater = _make_updater(_memory_config(max_facts=100, staleness_age_days=90)) + current_memory = _make_memory([_make_fact("fact_a", days_ago=100)]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToRemove": [], + "staleFactsToExtend": [{"id": "fact_a", "extend_by_days": 0.9}], + } + + result = updater._apply_updates(current_memory, update_data) + + assert "expected_valid_days" not in result["facts"][0] + + def test_proposed_removal_not_extendable_even_when_cap_trims(self): + # When the per-cycle removal cap trims actual deletions, proposed-removal + # facts that survive the cap must still be excluded from extensions. + # f_low (confidence=0.6) and f_high (confidence=0.9) are both proposed + # for removal; cap=1 so only f_low is actually removed. f_high must NOT + # be extended even though it wasn't removed this cycle. + updater = _make_updater( + _memory_config( + max_facts=100, + staleness_age_days=90, + staleness_max_removals_per_cycle=1, + ) + ) + current_memory = _make_memory( + [ + _make_fact("f_low", days_ago=120, confidence=0.6), + _make_fact("f_high", days_ago=120, confidence=0.9), ] ) update_data = { @@ -527,32 +756,81 @@ class TestApplyUpdatesStaleness: "newFacts": [], "factsToRemove": [], "staleFactsToRemove": [ - {"id": "fact_stale", "reason": "LLM hallucination"}, - {"id": "fact_fresh", "reason": "LLM hallucination"}, + {"id": "f_low", "reason": "outdated"}, + {"id": "f_high", "reason": "outdated"}, ], + "staleFactsToExtend": [{"id": "f_high", "extend_by_days": 180}], } - with patch( - "deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", - return_value=_memory_config( - max_facts=100, - staleness_review_enabled=False, - staleness_age_days=90, - staleness_min_candidates=3, - staleness_max_removals_per_cycle=10, - staleness_protected_categories=["correction"], - ), - ): - result = updater._apply_updates(current_memory, update_data) + result = updater._apply_updates(current_memory, update_data) + + ids = {f["id"] for f in result["facts"]} + assert "f_low" not in ids # removed (lowest confidence=0.6 after sort; cap=1) + assert "f_high" in ids # survived cap but must NOT be extended + f_high = next(f for f in result["facts"] if f["id"] == "f_high") + assert "expected_valid_days" not in f_high + + +# ── expected_valid_days on new facts (creation-time cap) ────────────────── + + +class TestNewFactsExpectedValidDays: + def test_stored_on_new_fact(self): + # evd=180 is within the default cap (90 * 20.0 = 1800), so stored as-is. + updater = _make_updater(_memory_config(max_facts=100, fact_confidence_threshold=0.7)) + current_memory = _make_memory([]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [{"content": "User speaks Spanish natively", "category": "knowledge", "confidence": 0.95, "expected_valid_days": 180}], + "factsToRemove": [], + } + + result = updater._apply_updates(current_memory, update_data) - # Guardrail runs regardless of feature flag: - # fact_stale is a valid candidate (200 days old) → removed - # fact_fresh is not a candidate (5 days old) → kept assert len(result["facts"]) == 1 - assert result["facts"][0]["id"] == "fact_fresh" + assert result["facts"][0]["expected_valid_days"] == 180 + + def test_creation_time_cap_applied_to_new_fact(self): + # evd=3650 exceeds the creation-time cap (90 * 3.0 = 270); stored as 270. + updater = _make_updater( + _memory_config( + max_facts=100, + fact_confidence_threshold=0.7, + staleness_age_days=90, + staleness_max_lifetime_multiplier=3.0, + ) + ) + current_memory = _make_memory([]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [{"content": "User prefers Python", "category": "knowledge", "confidence": 0.9, "expected_valid_days": 3650}], + "factsToRemove": [], + } + + result = updater._apply_updates(current_memory, update_data) + + assert len(result["facts"]) == 1 + assert result["facts"][0]["expected_valid_days"] == 270 # clamped to 90 * 3 + + def test_not_stored_when_absent(self): + updater = _make_updater(_memory_config(max_facts=100, fact_confidence_threshold=0.7)) + current_memory = _make_memory([]) + update_data = { + "user": {}, + "history": {}, + "newFacts": [{"content": "User uses Python", "category": "knowledge", "confidence": 0.9}], + "factsToRemove": [], + } + + result = updater._apply_updates(current_memory, update_data) + + assert len(result["facts"]) == 1 + assert "expected_valid_days" not in result["facts"][0] -# ── _normalize_memory_update_data with staleFactsToRemove ───────────────── +# ── Normalization of staleFactsToRemove ──────────────────────────────────── class TestNormalizeStaleFactsToRemove: @@ -634,98 +912,182 @@ class TestNormalizeStaleFactsToRemove: assert result["staleFactsToRemove"][0]["reason"] == "" +# ── Normalization of staleFactsToExtend ──────────────────────────────────── + + +class TestNormalizeStaleFactsToExtend: + def test_valid_entries(self): + data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToExtend": [ + {"id": "fact_a", "extend_by_days": 365, "reason": "core skill"}, + ], + } + result = _normalize_memory_update_data(data) + assert len(result["staleFactsToExtend"]) == 1 + assert result["staleFactsToExtend"][0]["id"] == "fact_a" + assert result["staleFactsToExtend"][0]["extend_by_days"] == 365 + + def test_float_extend_by_days_coerced_to_int(self): + data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToExtend": [{"id": "fact_a", "extend_by_days": 90.7}], + } + result = _normalize_memory_update_data(data) + assert result["staleFactsToExtend"][0]["extend_by_days"] == 90 + + def test_fractional_below_one_dropped(self): + data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToExtend": [{"id": "fact_a", "extend_by_days": 0.9}], + } + result = _normalize_memory_update_data(data) + assert result["staleFactsToExtend"] == [] + + def test_zero_and_negative_dropped(self): + data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToExtend": [ + {"id": "fact_a", "extend_by_days": 0}, + {"id": "fact_b", "extend_by_days": -5}, + ], + } + result = _normalize_memory_update_data(data) + assert result["staleFactsToExtend"] == [] + + def test_missing_key(self): + data = {"user": {}, "history": {}, "newFacts": [], "factsToRemove": []} + result = _normalize_memory_update_data(data) + assert result["staleFactsToExtend"] == [] + + def test_non_string_id_skipped(self): + data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToExtend": [{"id": 123, "extend_by_days": 365}], + } + result = _normalize_memory_update_data(data) + assert result["staleFactsToExtend"] == [] + + def test_missing_extend_by_days_skipped(self): + data = { + "user": {}, + "history": {}, + "newFacts": [], + "factsToRemove": [], + "staleFactsToExtend": [{"id": "fact_a", "reason": "no days"}], + } + result = _normalize_memory_update_data(data) + assert result["staleFactsToExtend"] == [] + + # ── Integration: _prepare_update_prompt ──────────────────────────────────── class TestPrepareUpdatePromptStaleness: def test_staleness_section_included_when_triggered(self): - updater = MemoryUpdater() old_facts = [_make_fact(f"fact_{i}", days_ago=100) for i in range(5)] memory = _make_memory(old_facts) + updater = _make_updater( + _memory_config(staleness_review_enabled=True, staleness_age_days=90, staleness_min_candidates=3), + memory=memory, + ) msg = MagicMock() msg.type = "human" msg.content = "Hello, I'm using React now" - config = _memory_config( - enabled=True, - staleness_review_enabled=True, - staleness_age_days=90, - staleness_min_candidates=3, + result = updater._prepare_update_prompt( + messages=[msg], + agent_name=None, + correction_detected=False, + reinforcement_detected=False, ) - with ( - patch("deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", return_value=config), - patch("deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_data", return_value=memory), - ): - result = updater._prepare_update_prompt( - messages=[msg], - agent_name=None, - correction_detected=False, - reinforcement_detected=False, - ) - assert result is not None _, prompt = result assert "Staleness Review" in prompt assert "" in prompt def test_staleness_section_omitted_when_not_triggered(self): - updater = MemoryUpdater() memory = _make_memory([]) # no facts at all + updater = _make_updater( + _memory_config(staleness_review_enabled=True, staleness_age_days=90, staleness_min_candidates=3), + memory=memory, + ) msg = MagicMock() msg.type = "human" msg.content = "Hello" - config = _memory_config( - enabled=True, - staleness_review_enabled=True, - staleness_age_days=90, - staleness_min_candidates=3, + result = updater._prepare_update_prompt( + messages=[msg], + agent_name=None, + correction_detected=False, + reinforcement_detected=False, ) - with ( - patch("deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", return_value=config), - patch("deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_data", return_value=memory), - ): - result = updater._prepare_update_prompt( - messages=[msg], - agent_name=None, - correction_detected=False, - reinforcement_detected=False, - ) - assert result is not None _, prompt = result assert "Staleness Review" not in prompt assert "" not in prompt def test_staleness_section_omitted_when_disabled(self): - updater = MemoryUpdater() old_facts = [_make_fact(f"fact_{i}", days_ago=200) for i in range(10)] memory = _make_memory(old_facts) + updater = _make_updater( + _memory_config(staleness_review_enabled=False), + memory=memory, + ) msg = MagicMock() msg.type = "human" msg.content = "Hello" - config = _memory_config( - enabled=True, - staleness_review_enabled=False, + result = updater._prepare_update_prompt( + messages=[msg], + agent_name=None, + correction_detected=False, + reinforcement_detected=False, ) - with ( - patch("deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_config", return_value=config), - patch("deerflow.agents.memory.backends.deermem.deermem.core.updater.get_memory_data", return_value=memory), - ): - result = updater._prepare_update_prompt( - messages=[msg], - agent_name=None, - correction_detected=False, - reinforcement_detected=False, - ) - assert result is not None _, prompt = result assert "Staleness Review" not in prompt + + def test_staleness_section_shows_valid_annotation(self): + old_facts = [_make_fact("f1", days_ago=400, expected_valid_days=365) for _ in range(3)] + memory = _make_memory(old_facts) + updater = _make_updater( + _memory_config(staleness_review_enabled=True, staleness_age_days=90, staleness_min_candidates=3), + memory=memory, + ) + + msg = MagicMock() + msg.type = "human" + msg.content = "Hello" + + result = updater._prepare_update_prompt( + messages=[msg], + agent_name=None, + correction_detected=False, + reinforcement_detected=False, + ) + + assert result is not None + _, prompt = result + assert "valid:365d" in prompt diff --git a/config.example.yaml b/config.example.yaml index e4e28733d..572dda6cc 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -1474,6 +1474,15 @@ summarization: # staleness_min_candidates - Minimum stale facts to trigger review (default: 3) # staleness_max_removals_per_cycle - Safety cap on removals per cycle (default: 10) # staleness_protected_categories - Categories exempt from staleness review (default: ["correction"]) +# staleness_max_lifetime_multiplier - Creation-time cap multiplier for a fact LLM-assigned +# expected_valid_days; new facts clamped to +# staleness_age_days x multiplier. Default 20.0 +# (90 x 20 = 1800d ~ 5 years) supports the very-stable +# prompt tier. (default: 20.0, range: 1.0-100.0) +# staleness_max_extension_days - Absolute ceiling (in days) on expected_valid_days after a +# lifetime extension (staleFactsToExtend). Prevents timedelta +# overflow and LLM misfire from permanently deferring a fact. +# (default: 3650, range: 90-36500) memory: enabled: true injection_enabled: true # Whether to inject memory into system prompt @@ -1527,14 +1536,19 @@ memory: guaranteed_token_budget: 500 # Staleness review: periodically prune aged facts that may no longer reflect # the user current situation. When triggered, the LLM reviews facts older - # than staleness_age_days during the normal memory-update call (same LLM - # invocation - no extra API call) and decides KEEP or REMOVE for each. + # than their individual review window (expected_valid_days, or + # staleness_age_days as fallback) during the normal memory-update call (same + # LLM invocation - no extra API call) and decides KEEP, REMOVE, or EXTEND + # for each. The LLM assigns expected_valid_days when creating a fact; EXTEND + # (staleFactsToExtend) recalibrates that window at review time. staleness_review_enabled: true staleness_age_days: 90 staleness_min_candidates: 3 staleness_max_removals_per_cycle: 10 staleness_protected_categories: - correction + staleness_max_lifetime_multiplier: 20.0 # creation cap = staleness_age_days x multiplier (90 x 20 = 1800d) + staleness_max_extension_days: 3650 # absolute ceiling on extended expected_valid_days (~10 years) # Memory consolidation (opt-in, lossy: source facts are replaced by a # synthesized one; only consolidatedFrom IDs are kept). Runs in the same # memory-update LLM call as extraction/staleness - no extra API cost.