fix(memory): tolerate unbounded stored fact confidence (#7764)

* fix(memory): tolerate unbounded stored fact confidence

A stored memory fact can carry a JSON integer such as 10**400. float() on it
raises OverflowError in _coerce_source_confidence and in the dedup merge
comparison, aborting the whole update cycle even though both sites already
handle every other unusable confidence. Treat the overflow like a missing or
non-finite value: 0.5 for ranking, 0.0 for the merge comparison.

* fix(memory): harden the remaining fact-confidence coercions

The import normalizer, prompt builder and relevance scorer each coerce a
stored fact confidence with float() and caught only TypeError/ValueError, so
an unbounded JSON integer such as 10**400 raised OverflowError: import failed
before the updater fallbacks ran, and prompt/relevance ranking could abort.
Catch OverflowError and take the existing defaults; document the full set of
coercions.

README.md

@@ -3436,7 +3436,7 @@ Fact CRUD and Settings-page fact editing are not available for this backend. See
 the [Honcho backend guide](backend/packages/harness/deerflow/agents/memory/backends/honcho/README.md).
 
 Memory updates now skip duplicate fact entries at apply time, so repeated preferences and context do not accumulate endlessly across sessions.
-Legacy memory files are normalized when read or imported, including recoverable fact metadata, so older local data remains usable as the schema evolves. Frontend and backend normalization use confidence `0.5` when missing or invalid, default blank sources to `unknown`, and trim fact content.
+Legacy memory files are normalized when read or imported, including recoverable fact metadata, so older local data remains usable as the schema evolves. Frontend and backend normalization use confidence `0.5` when missing or invalid, default blank sources to `unknown`, and trim fact content. A stored confidence that cannot become a finite float, such as an unbounded JSON integer in a hand-edited file, also falls back to `0.5` instead of failing a memory read, import, prompt build or relevance ranking.
 
 In the default DeerMem `middleware` mode, automatic extraction now classifies every proposed fact by scope, durability, and authority before a deterministic write gate accepts it. Only durable, descriptive user-level facts are stored; current-thread or project constraints and one-time action permissions stay in conversation state. User-global summaries require both user scope and descriptive authority, contradiction removals are scope-gated, and a replacement-dependent removal is applied only when its replacement actually survives validation and storage. These classification labels are extraction-only metadata, add no extra LLM call, and are not written into the fact files. The explicit CRUD tools in `memory.mode: tool` remain a separate, model-directed path. Deployments that override the bundled DeerMem prompts via `memory.backend_config.prompts_dir` must add the new classification fields to their custom templates (the `memory_update` fact/summary/removal formats and the `consolidation` consolidated-fact schema): the write gate fails closed, so an un-migrated template stops every extraction-driven fact, summary, and removal write, surfacing only through the `rejected_by_scope_gate` metrics and the high-rejection-rate warning.
 

backend/packages/harness/deerflow/agents/memory/AGENTS.md

@@ -286,7 +286,7 @@ Extraction labels proposals with `scope`, `durability`, and `authority`.
 Automatic writes accept only user-scoped, durable, descriptive facts.
 Summary prose must be user-scoped and descriptive.
 Missing labels reject that item without stopping unrelated updates.
-An out-of-range `confidence` (including a model-emitted percentage) is rejected per item with a warning that names the raw value, and is never rescaled: a silent scale rewrite would change the confidence-threshold semantics. Storage's own `[0, 1]` validation must not fail the batch, because a failed batch keeps the extraction watermark and repeats the same paid call. A batch that also proposes `factsToRemove` still fails closed when any new fact is dropped, because removal `replacementFactIndex` values refer to newFacts positions.
+An out-of-range `confidence` (including a model-emitted percentage) is rejected per item with a warning that names the raw value, and is never rescaled: a silent scale rewrite would change the confidence-threshold semantics. Stored facts are editable JSON: a `confidence` such as `10**400` is a valid JSON integer, and every fact-confidence coercion on a read or import path (`_coerce_source_confidence`, the dedup merge comparison, `prompt._coerce_confidence`, `relevance._coerce_confidence`, and legacy import normalization) must treat the resulting `OverflowError` like any other unusable value. Canonical storage still rejects an out-of-range confidence as corruption at load time; these helpers cover values reaching them through editable or legacy paths. Storage's own `[0, 1]` validation must not fail the batch, because a failed batch keeps the extraction watermark and repeats the same paid call. A batch that also proposes `factsToRemove` still fails closed when any new fact is dropped, because removal `replacementFactIndex` values refer to newFacts positions.
 
 Contradiction removals include `id`, `scope`, `reason`, and optional `replacementFactIndex`.
 Task-scoped and project-scoped removals fail closed.

backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py

@@ -332,7 +332,7 @@ def _coerce_confidence(value: Any, default: float = 0.0) -> float:
     """
     try:
         confidence = float(value)
-    except (TypeError, ValueError):
+    except (TypeError, ValueError, OverflowError):
         return max(0.0, min(1.0, default))
     if not math.isfinite(confidence):
         return max(0.0, min(1.0, default))

backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/relevance.py

@@ -151,7 +151,7 @@ def _coerce_confidence(fact: dict[str, Any]) -> float:
         value = float(fact.get("confidence"))
         if not math.isfinite(value):
             raise ValueError
-    except (TypeError, ValueError):
+    except (TypeError, ValueError, OverflowError):
         return 0.0
     return min(1.0, max(0.0, value))
 

backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py

@@ -146,7 +146,7 @@ def _normalize_legacy_import_fact(value: Any) -> dict[str, Any] | None:
     else:
         try:
             numeric_confidence = float(confidence)
-        except (TypeError, ValueError):
+        except (TypeError, ValueError, OverflowError):
             numeric_confidence = 0.5
     fact["confidence"] = min(1.0, max(0.0, numeric_confidence)) if math.isfinite(numeric_confidence) else 0.5
 

backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py

@@ -76,7 +76,7 @@ def _coerce_source_confidence(fact: dict[str, Any]) -> float:
         return 0.5
     try:
         val = float(raw)
-    except (TypeError, ValueError):
+    except (TypeError, ValueError, OverflowError):
         return 0.5
     return max(0.0, min(val, 1.0)) if math.isfinite(val) else 0.5
 
@@ -2448,7 +2448,7 @@ class MemoryUpdater:
                         existing_confidence = float(merge_target.get("confidence"))
                         if not math.isfinite(existing_confidence):
                             raise ValueError
-                    except (TypeError, ValueError):
+                    except (TypeError, ValueError, OverflowError):
                         existing_confidence = 0.0
                     if confidence > existing_confidence:
                         merge_target["confidence"] = confidence

backend/tests/test_memory_confidence_overflow.py

@@ -0,0 +1,27 @@
+"""An unbounded stored fact confidence must not abort memory reads or imports.
+
+A memory document is editable JSON, so ``confidence`` can be an arbitrary
+precision integer such as ``10**400``. ``float()`` raises ``OverflowError`` on
+it; every fact-confidence coercion on a read or import path must treat that
+like any other unusable value instead of failing the whole operation.
+"""
+
+from deerflow.agents.memory.backends.deermem.deermem.core.prompt import _coerce_confidence as prompt_confidence
+from deerflow.agents.memory.backends.deermem.deermem.core.relevance import _coerce_confidence as relevance_confidence
+from deerflow.agents.memory.backends.deermem.deermem.core.storage import normalize_memory_data
+
+UNBOUNDED = 10**400
+
+
+def test_legacy_import_normalization_falls_back_to_the_default_confidence():
+    normalized = normalize_memory_data({"facts": [{"content": "User likes Rust", "confidence": UNBOUNDED}]})
+
+    assert normalized["facts"][0]["confidence"] == 0.5
+
+
+def test_prompt_confidence_falls_back_to_the_caller_default():
+    assert prompt_confidence(UNBOUNDED, default=0.0) == 0.0
+
+
+def test_relevance_confidence_falls_back_to_zero():
+    assert relevance_confidence({"confidence": UNBOUNDED}) == 0.0

backend/tests/test_memory_fact_dedup.py

@@ -124,6 +124,20 @@ class TestFactDedupGate:
         assert merged["source"] == "unknown"  # refreshed from the proposed fact's thread
         assert metrics.get("facts_merged_dedup") == 1
 
+    def test_merge_tolerates_an_unbounded_existing_confidence(self):
+        updater = _updater(fact_dedup_enabled=True)
+        current = _memory(
+            [_stored_fact("fact_old", "User prefers concise answers in chat", confidence=10**400)],
+        )
+
+        result = updater._apply_updates(
+            current,
+            _update([_fact("User prefers concise answers in chat, short form", confidence=0.95)]),
+        )
+
+        assert len(result["facts"]) == 1
+        assert result["facts"][0]["confidence"] == 0.95
+
     def test_merge_keeps_higher_existing_confidence(self):
         updater = _updater(fact_dedup_enabled=True)
         current = _memory([_stored_fact("fact_old", "User prefers concise answers in chat", confidence=0.98)])

backend/tests/test_memory_updater.py

@@ -1891,6 +1891,11 @@ class TestNullConfidenceDoesNotBlockUpdates:
         assert _coerce_source_confidence({}) == 0.5
         assert _coerce_source_confidence({"confidence": 0.83}) == 0.83
 
+    def test_coerce_source_confidence_tolerates_an_unbounded_int(self) -> None:
+        # A stored fact can carry a JSON integer such as 10**400; float() raises
+        # OverflowError, which must fall back like any other unusable value.
+        assert _coerce_source_confidence({"confidence": 10**400}) == 0.5
+
 
 class TestOutOfRangeExtractionConfidenceDoesNotBlockUpdates:
     """A model-emitted confidence outside [0, 1] must not fail the whole update.