diff --git a/README.md b/README.md index 7485b0fe4..a172988ea 100644 --- a/README.md +++ b/README.md @@ -1890,6 +1890,7 @@ Fact CRUD and Settings-page fact editing are not available for this backend. See the [Honcho backend guide](backend/packages/harness/deerflow/agents/memory/backends/honcho/README.md). Memory updates now skip duplicate fact entries at apply time, so repeated preferences and context do not accumulate endlessly across sessions. +Legacy memory files are normalized when read or imported, including recoverable fact metadata, so older local data remains usable as the schema evolves. Frontend and backend normalization use confidence `0.5` when missing or invalid, default blank sources to `unknown`, and trim fact content. In the default DeerMem `middleware` mode, automatic extraction now classifies every proposed fact by scope, durability, and authority before a deterministic write gate accepts it. Only durable, descriptive user-level facts are stored; current-thread or project constraints and one-time action permissions stay in conversation state. User-global summaries require both user scope and descriptive authority, contradiction removals are scope-gated, and a replacement-dependent removal is applied only when its replacement actually survives validation and storage. These classification labels are extraction-only metadata, add no extra LLM call, and are not written into the fact files. The explicit CRUD tools in `memory.mode: tool` remain a separate, model-directed path. Deployments that override the bundled DeerMem prompts via `memory.backend_config.prompts_dir` must add the new classification fields to their custom templates (the `memory_update` fact/summary/removal formats and the `consolidation` consolidated-fact schema): the write gate fails closed, so an un-migrated template stops every extraction-driven fact, summary, and removal write, surfacing only through the `rejected_by_scope_gate` metrics and the high-rejection-rate warning. @@ -1915,7 +1916,7 @@ Memory injection follows the configured operation mode. In `middleware` mode, De An individual Custom Agent can opt out of memory without changing the global setting. Add `memory_enabled: false` to that agent's `users/{user_id}/agents/{name}/config.yaml`. The agent still receives the current-date reminder, but DeerFlow does not inject recalled memory, queue passive or summarization-driven memory updates (including manual `/compact`), expose memory tools, or add memory-tool instructions for that agent. If an existing agent is switched off, its previously injected memory block is removed from checkpoint state before the next model call while its date reminder and conversation remain. Omitting the field (or setting it to `true`) preserves the existing global `memory` behavior. -Single-fact repository operations are genuinely incremental: an upsert/delete reads, journals, writes, and re-indexes only the addressed fact files, and returns an explicit incomplete delta rather than a cache-dependent fake full document. Summary change sets merge the supplied `user`/`history` child keys over the persisted sections so a partial update cannot erase omitted siblings; full imports normalize both sections to the complete compatibility schema before applying replacement values. Manager/API compatibility methods materialize a fresh full document only when their public response contract requires one. Fact-level point operations use separate expected user-memory and fact revisions and may explicitly rebase when every addressed fact precondition still holds. Snapshot-derived operations such as scoped clear, capped create, consolidation, and trimming never replay stale delete/trim sets: a manifest conflict reloads the complete document and recomputes the operation, with a bounded retry. Fact paths use the first two hexadecimal characters of `SHA-256(fact_id)` so generated `fact_*` IDs distribute across shards. The cache token combines the shared JSON's nanosecond mtime, size, and persisted revision; this prevents coarse-mtime same-size writes from returning stale data without scanning fact files. Direct out-of-band Markdown edits require an explicit reload. Storage-specific conflicts and corruption are translated at the MemoryManager boundary; the Gateway returns conflict as HTTP 409 and a stable, non-sensitive corruption error as HTTP 500. Full-document `save()` remains a compatibility API and computes a diff before writing; malformed or missing `facts` can no longer silently erase an agent's Markdown files. Legacy migration preserves non-empty `user`/`history` before deleting an agent `memory.json`; conflicting summaries keep the legacy file and fail loudly instead of choosing a winner. +Single-fact repository operations are genuinely incremental: an upsert/delete reads, journals, writes, and re-indexes only the addressed fact files, and returns an explicit incomplete delta rather than a cache-dependent fake full document. Summary change sets merge the supplied `user`/`history` child keys over the persisted sections so a partial update cannot erase omitted siblings; full imports normalize both sections to the complete compatibility schema before applying replacement values. Imports reject malformed fact lists or blank/non-text content with HTTP 400 before changing stored memory; recoverable legacy metadata still receives defaults. An explicit empty fact list remains an intentional clear. Manager/API compatibility methods materialize a fresh full document only when their public response contract requires one. Fact-level point operations use separate expected user-memory and fact revisions and may explicitly rebase when every addressed fact precondition still holds. Snapshot-derived operations such as scoped clear, capped create, consolidation, and trimming never replay stale delete/trim sets: a manifest conflict reloads the complete document and recomputes the operation, with a bounded retry. Fact paths use the first two hexadecimal characters of `SHA-256(fact_id)` so generated `fact_*` IDs distribute across shards. The cache token combines the shared JSON's nanosecond mtime, size, and persisted revision; this prevents coarse-mtime same-size writes from returning stale data without scanning fact files. Direct out-of-band Markdown edits require an explicit reload. Storage-specific conflicts and corruption are translated at the MemoryManager boundary; the Gateway returns conflict as HTTP 409 and a stable, non-sensitive corruption error as HTTP 500. Full-document `save()` remains a compatibility API and computes a diff before writing; malformed or missing `facts` can no longer silently erase an agent's Markdown files. Legacy migration preserves non-empty `user`/`history` before deleting an agent `memory.json`; conflicting summaries keep the legacy file and fail loudly instead of choosing a winner. Legacy facts in `memory.json` migrate automatically into the reserved `__default__` Markdown bucket on the user's first normal memory read. Operators who prefer to audit or complete the migration before serving traffic can run the optional idempotent CLI from `backend/`: diff --git a/backend/README.md b/backend/README.md index 3ae87e65d..b5f69d53f 100644 --- a/backend/README.md +++ b/backend/README.md @@ -106,7 +106,7 @@ LLM-powered persistent context retention across conversations: - **System prompt injection**: Top facts + context injected into agent prompts - **Run-level memory identity**: `GET /api/threads/{thread_id}/runs/{run_id}/events?event_types=context:memory` returns the SHA-256 identity of the effective hidden memory block without copying memory text into the event store - **Read failures**: Strict backend policies (including legacy `fail_closed`) stop the turn, including at the 5-second async injection deadline. Fail-open reads continue without new context. Timeout handling does not wait for a free worker; a timed-out read may still occupy its worker until the backend returns. -- **Storage**: JSON file with mtime-based cache invalidation +- **Storage**: JSON file with mtime-based cache invalidation and canonical normalization for legacy sections/fact metadata ### Tool Ecosystem diff --git a/backend/app/gateway/routers/memory.py b/backend/app/gateway/routers/memory.py index cf0e2ad02..85fed66dc 100644 --- a/backend/app/gateway/routers/memory.py +++ b/backend/app/gateway/routers/memory.py @@ -51,6 +51,10 @@ class UserContext(BaseModel): workContext: ContextSection = Field(default_factory=ContextSection) personalContext: ContextSection = Field(default_factory=ContextSection) topOfMind: ContextSection = Field(default_factory=ContextSection) + cognitiveStyle: ContextSection = Field( + default_factory=ContextSection, + description="Stable thinking and collaboration habits (reasoning style, depth, feedback patterns)", + ) class HistoryContext(BaseModel): @@ -435,6 +439,8 @@ async def import_memory(body: MemoryResponse, request: Request) -> MemoryRespons raise _unsupported_501(manager, "import memory") from None except (MemoryConflictError, MemoryCorruptionError) as exc: raise _map_memory_manager_error(exc) from exc + except ValueError as exc: + raise HTTPException(status_code=400, detail="Invalid memory import: facts must be a list of objects with non-empty content.") from exc except OSError as exc: raise HTTPException(status_code=500, detail="Failed to import memory data.") from exc diff --git a/backend/docs/MEMORY_COGNITIVE_STYLE.md b/backend/docs/MEMORY_COGNITIVE_STYLE.md new file mode 100644 index 000000000..3a3d7466d --- /dev/null +++ b/backend/docs/MEMORY_COGNITIVE_STYLE.md @@ -0,0 +1,140 @@ +# Memory: Cognitive Style + +Design note for contributors. Explains `user.cognitiveStyle` and related memory/prompt touchpoints. + +## One-line pitch + +**Skills teach the agent how to do tasks; memory’s `cognitiveStyle` teaches the agent how to think and collaborate with this user.** + +## Problem + +Cross-session memory already stores work context, personal preferences, and facts. In practice, two gaps show up: + +1. **Semantic mixing** — “Prefers TypeScript” and “Always wants conclusions first, then details” both land in `personalContext` or `behavior` facts. The model must infer which is *collaboration protocol* vs *project preference*. +2. **Wrong layer for collaboration prefs** — Task skills (`SKILL.md`) are procedural and shared. Stable response/collaboration preferences (structure, depth, correction style) are **user-scoped slow variables**, not one-off task steps. + +Without an explicit slot, collaboration style is under-specified in injection and easy to drop under token pressure. + +## Approach (not a new subsystem) + +Extend the existing memory pipeline: + +| Layer | Role | +|-------|------| +| `user.cognitiveStyle.summary` | 2–4 sentence paragraph: reasoning & collaboration habits | +| `facts[]` with `category: cognitive` | Atomic, confidence-ranked supplements | +| `normalize_memory_data()` | Backward-compatible fill for older sections and fact metadata | +| `core/prompts/memory_update.chat.yaml` | LLM sets `cognitiveStyle.shouldUpdate` only when new signals are clear | +| `format_memory_for_injection()` | Injects as `Thinking Style:` under User Context | + +**Non-goals (this change):** + +- Vector / embedding “personality library” +- Separate debounce or sampling schedule for `cognitiveStyle` only +- Replacing `personalContext` or skills + +## Update frequency (read vs write) + +| Event | Behavior | +|-------|----------| +| **Read (every turn)** | If `injection_enabled`, current `cognitiveStyle` is loaded into `` within `max_injection_tokens` | +| **Write (after turn)** | `MemoryMiddleware` queues filtered conversation; **debounce** (`debounce_seconds`, default 30s) batches updates | +| **Write (cognitive field)** | Same LLM pass as other user sections; field changes only when JSON has `cognitiveStyle.shouldUpdate: true` | + +So: conversations **trigger** the memory job often; **cognitiveStyle text changes** only when the updater model sees durable new evidence—not every chit-chat turn. + +## How this differs from nearby concepts + +| Concept | Scope | Lifetime | +|---------|--------|----------| +| **Thread / checkpointer** | This session’s messages & tools | Session | +| **Skill** | How to run a task type | Shared / installable | +| **workContext / topOfMind** | What the user is doing | Cross-session, changes often | +| **personalContext** | Language, interests, tone | Cross-session | +| **cognitiveStyle** | How they reason, structure answers, give feedback | Cross-session, **slow** | +| **fact (`cognitive`)** | One line habit or meta-preference | Cross-session, ranked by confidence | + +## Adding a new memory field (schema evolution) + +When extending the global summary JSON (`user.*` / `history.*`) or the per-agent Markdown fact schema, keep **read**, **import**, migration, and **API** paths aligned so older exports still work. + +| Step | Location | +|------|----------| +| 1. Backend normalize | `deerflow/agents/memory/backends/deermem/deermem/core/storage.py` — add keys to `normalize_memory_data()` / fact normalization; update `create_empty_memory()` | +| 2. Frontend normalize | `frontend/src/core/memory/import-memory.ts` — add section keys and normalize recoverable legacy fact metadata before narrowing to `UserMemory` | +| 3. Types & API models | `frontend/src/core/memory/types.ts`, `backend/app/gateway/routers/memory.py` (`UserContext` / `HistoryContext`) | +| 4. Updater prompt | `core/prompts/memory_update.chat.yaml` and `core/prompts/fact_extraction.yaml`; add injection rendering in `core/prompt.py::format_memory_for_injection()`. Fact categories must also be added to `storage.py::CORE_CATEGORIES` | +| 5. Settings UI & i18n | `memory-settings-page.tsx`, `en-US.ts` / `zh-CN.ts` | +| 6. Tests | Backend: legacy sections and facts in `tests/test_memory_storage.py` / `tests/test_memory_normalize.py` / `tests/test_deermem_self_contained.py`. Frontend: import and API-read behavior in `tests/unit/core/memory/` | +| 7. Import path | Keep the stable export envelope strict (`version`, `lastUpdated`, object `user`/`history`, array `facts`), then normalize additive fields inside that valid envelope | + +**Avoid:** normalizing only sections while leaving legacy fact metadata unchecked, or making one unrecoverable fact fail the entire background API read. User-initiated imports remain strict for facts without usable content; API reads drop only those unrecoverable entries. + +## Verification + +```bash +cd backend +PYTHONPATH=. uv run pytest -q tests/test_memory_storage.py tests/test_memory_prompt_injection.py tests/test_memory_normalize.py tests/test_deermem_self_contained.py +PYTHONPATH=. uv run pytest tests/test_memory_router.py -v + +cd ../frontend +pnpm test tests/unit/core/memory +``` + +Manual: + +1. Enable `memory` in `config.yaml`, run `make dev`. +2. In a thread, state a stable collaboration rule (e.g. “先给结论,不要长铺垫”). +3. Wait ≥ `debounce_seconds`, open **Settings → Memory** or `GET /api/memory`. +4. Confirm `user.cognitiveStyle.summary` and/or a `cognitive` fact; start a **new thread** and check behavior. + +## Issue + +**Title:** `feat(memory): add cognitiveStyle for stable reasoning & collaboration habits` + +**Summary:** + +- Adds `user.cognitiveStyle` to memory schema with backward-compatible normalization. +- Teaches the memory updater to extract thinking/collaboration habits separately from work/personal context. +- Injects as `Thinking Style:` in system prompt; supports `cognitive` fact category. +- Documents rationale in `backend/docs/MEMORY_COGNITIVE_STYLE.md` and harness memory docs. + +**Motivation:** Cross-session memory should distinguish project context from stable collaboration preferences (response structure, correction style, depth). This change extends the existing memory harness only; it does not add a new store. + +--- + +## 中文说明 + +### 背景 + +跨会话 memory 已有 `workContext`、`personalContext` 与 `behavior` 类 facts。实践中两类信息容易混在同一字段里: + +- 项目/工具偏好(例如常用 TypeScript) +- 协作偏好(例如先给结论、控制篇幅、纠错方式) + +后者更新频率低于 `topOfMind`,又不同于一次性会话信息。单独增加 `user.cognitiveStyle` 便于注入时固定展示为 `Thinking Style:`,并与任务级 Skill 区分。 + +### 实现范围 + +- 扩展全局 summary JSON 与 per-agent Markdown fact schema,`normalize_memory_data()` 兼容旧 section 和缺少元数据的旧 facts +- `core/prompts/memory_update.chat.yaml` 输出 `cognitiveStyle.shouldUpdate`;可选 `category: cognitive` 的 facts +- 复用现有 MemoryManager → DeerMem → 防抖队列 → Updater → 注入链路,不新增子系统 + +### 更新频率 + +| 事件 | 行为 | +|------|------| +| 每轮对话开始 | 在 `max_injection_tokens` 内注入已有 `cognitiveStyle`(读) | +| 每轮对话结束 | 与其它 memory 段相同,可能入队;默认 `debounce_seconds` 合并 | +| 写入 `cognitiveStyle` | 仅当 LLM 返回 `shouldUpdate: true` 时更新段落 | + +### 与 Skill 的区别 + +| 类型 | 内容 | +|------|------| +| Skill | 某类任务的步骤与模板(可共享、可安装) | +| `cognitiveStyle` | 该用户稳定的回复结构、讨论深度、反馈习惯(按用户持久化) | + +### 以后新增 memory 字段时 + +按上文 **Adding a new memory field** 清单同步改后端 `normalize_memory_data()` 与前端 `normalizeMemoryPayload()`;导入走 normalize,不要只对完整新 schema 做严校验。后台 API 读取应丢弃无法恢复的单条 fact,而不是让整个 Memory 页面失败。 diff --git a/backend/packages/harness/deerflow/agents/memory/AGENTS.md b/backend/packages/harness/deerflow/agents/memory/AGENTS.md index 964de5292..c02e06436 100644 --- a/backend/packages/harness/deerflow/agents/memory/AGENTS.md +++ b/backend/packages/harness/deerflow/agents/memory/AGENTS.md @@ -129,6 +129,10 @@ Every destructive migration first writes a verified `{manifest_filename}.v1.bak` Missing or mismatched backups abort migration without changing v1 data. Delete legacy agent JSON only after safe summary adoption or equality checks. Summary conflicts keep the source file and return an error. +Compare both summary operands after additive normalization; preserve extension +fields and keep fact migration validation strict. Replacement imports reject +invalid fact containers or unusable content before normalization or storage +access; recoverable metadata may default, but malformed facts never mean clear. Run the proactive migration from `backend/`: @@ -353,3 +357,5 @@ runs by default. Attachment-only messages with an empty preserved request stay query-less. - Ranking must be deterministic, network-free, and mutation-free: caller-owned fact dicts are read-only inputs. + +Legacy fact normalization in DeerMem and `frontend/src/core/memory/import-memory.ts` uses neutral confidence `0.5` for missing or invalid values, clamps finite confidence to `[0, 1]`, trims content, and defaults blank or missing sources to `unknown`. Keep these compatibility defaults aligned. diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py index 6380b60da..e6e84f200 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompt.py @@ -547,6 +547,10 @@ def format_memory_for_injection( if top_of_mind.get("summary"): user_sections.append(f"Current Focus: {_escape_summary(top_of_mind['summary'])}") + cognitive_style = user_data.get("cognitiveStyle", {}) + if cognitive_style.get("summary"): + user_sections.append(f"Thinking Style: {_escape_summary(cognitive_style['summary'])}") + if user_sections: sections.append("User Context:\n" + "\n".join(f"- {s}" for s in user_sections)) diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/fact_extraction.yaml b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/fact_extraction.yaml index 6828e0934..22fb26ba0 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/fact_extraction.yaml +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/fact_extraction.yaml @@ -9,7 +9,7 @@ template: |- Extract facts in this JSON format: {{ "facts": [ - {{ "content": "...", "category": "preference|knowledge|context|behavior|goal|correction", "confidence": 0.0-1.0 }} + {{ "content": "...", "category": "preference|knowledge|context|behavior|cognitive|goal|correction", "confidence": 0.0-1.0 }} ] }} @@ -18,6 +18,7 @@ template: |- - knowledge: User's expertise or knowledge areas - context: Background context (location, job, projects) - behavior: Behavioral patterns + - cognitive: Stable reasoning, response-structure, and collaboration habits - goal: User's goals or objectives - correction: Explicit corrections or mistakes to avoid repeating diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/memory_update.chat.yaml b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/memory_update.chat.yaml index 82e491079..20eb81fc9 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/memory_update.chat.yaml +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/prompts/memory_update.chat.yaml @@ -39,6 +39,9 @@ messages: Example: Primary project work, parallel technical investigations, ongoing learning/tracking Include: Active implementation work, troubleshooting issues, market/research interests Note: This captures SEVERAL concurrent focus areas, not just one task + - cognitiveStyle: Stable reasoning and collaboration habits (2-4 sentences) + Include: Preferred answer structure, desired depth, feedback/correction style, decision-making patterns + Update conservatively only when durable cross-session evidence is clear **History** (Temporal context - rich paragraphs): - recentMonths: Detailed summary of recent activities (4-6 sentences or 1-2 paragraphs) @@ -63,6 +66,7 @@ messages: * knowledge: Specific expertise, technologies mastered, domain knowledge * context: Background facts (job title, projects, locations, languages) * behavior: Working patterns, communication habits, problem-solving approaches + * cognitive: Stable reasoning, response-structure, and collaboration habits * goal: Stated objectives, learning targets, project ambitions * correction: Explicit agent mistakes or user corrections, including the correct approach - Fact lifetime (``expected_valid_days``, optional integer): @@ -85,6 +89,7 @@ messages: - personalContext: Languages, personality, interests outside direct work tasks - topOfMind: Multiple ongoing priorities and focus areas user cares about recently (gets updated most frequently) Should capture 3-5 concurrent themes: main work, side explorations, learning/tracking interests + - cognitiveStyle: Stable preferences for how the assistant should reason, structure answers, and collaborate - recentMonths: Detailed account of recent technical explorations and work - earlierContext: Patterns from slightly older interactions still relevant - longTermBackground: Unchanging foundational facts about the user @@ -99,7 +104,8 @@ messages: "user": {{ "workContext": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }}, "personalContext": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }}, - "topOfMind": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }} + "topOfMind": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }}, + "cognitiveStyle": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }} }}, "history": {{ "recentMonths": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }}, @@ -107,7 +113,7 @@ messages: "longTermBackground": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }} }}, "newFacts": [ - {{ "content": "...", "category": "preference|knowledge|context|behavior|goal|correction", "confidence": 0.0-1.0, "expected_valid_days": 90, "scope": "user|thread|project", "durability": "durable|temporary", "authority": "descriptive|transactional" }} + {{ "content": "...", "category": "preference|knowledge|context|behavior|cognitive|goal|correction", "confidence": 0.0-1.0, "expected_valid_days": 90, "scope": "user|thread|project", "durability": "durable|temporary", "authority": "descriptive|transactional" }} ], "factsToReinforce": [ {{ "id": "fact_id_1", "scope": "user|thread|project", "reason": "explicit user confirmation of this existing fact" }} diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py index cdda9e161..f79233fbb 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/storage.py @@ -48,7 +48,7 @@ from .paths import ( logger = logging.getLogger(__name__) DOCUMENT_VERSION = "2.0" -CORE_CATEGORIES = frozenset({"preference", "correction", "context", "goal", "behavior", "identity", "constraint", "decision", "other"}) +CORE_CATEGORIES = frozenset({"preference", "correction", "context", "goal", "behavior", "cognitive", "identity", "constraint", "decision", "other"}) class MemoryStorageError(RuntimeError): @@ -103,6 +103,7 @@ def create_empty_memory() -> dict[str, Any]: "workContext": {"summary": "", "updatedAt": ""}, "personalContext": {"summary": "", "updatedAt": ""}, "topOfMind": {"summary": "", "updatedAt": ""}, + "cognitiveStyle": {"summary": "", "updatedAt": ""}, }, "history": { "recentMonths": {"summary": "", "updatedAt": ""}, @@ -113,6 +114,84 @@ def create_empty_memory() -> dict[str, Any]: } +def _normalize_context_section(value: Any) -> dict[str, Any]: + """Return a canonical summary section while preserving extension fields.""" + if not isinstance(value, dict): + return {"summary": "", "updatedAt": ""} + section = copy.deepcopy(value) + section["summary"] = value.get("summary") if isinstance(value.get("summary"), str) else "" + section["updatedAt"] = value.get("updatedAt") if isinstance(value.get("updatedAt"), str) else "" + return section + + +def _normalize_legacy_import_fact(value: Any) -> dict[str, Any] | None: + """Canonicalize recoverable public/legacy fact fields before repository writes.""" + if not isinstance(value, dict): + return None + content = value.get("content") + if not isinstance(content, str) or not content.strip(): + return None + + fact = copy.deepcopy(value) + fact["content"] = content.strip() + fact_id = fact.get("id") + fact["id"] = fact_id.strip() if isinstance(fact_id, str) and fact_id.strip() else f"fact_{uuid.uuid4().hex[:8]}" + category = fact.get("category") + fact["category"] = category.strip() if isinstance(category, str) and category.strip() else "context" + + confidence = fact.get("confidence", 0.5) + if isinstance(confidence, bool): + numeric_confidence = 0.5 + else: + try: + numeric_confidence = float(confidence) + except (TypeError, ValueError): + numeric_confidence = 0.5 + fact["confidence"] = min(1.0, max(0.0, numeric_confidence)) if math.isfinite(numeric_confidence) else 0.5 + + created_at = fact.get("createdAt") + fact["createdAt"] = created_at.strip() if isinstance(created_at, str) else "" + source = fact.get("source") + fact["source"] = source.strip() if isinstance(source, str) and source.strip() else "unknown" + if "sourceError" in fact and fact["sourceError"] is not None and not isinstance(fact["sourceError"], str): + fact.pop("sourceError") + return fact + + +def _normalize_memory_summaries(data: dict[str, Any]) -> dict[str, Any]: + """Normalize additive summary fields without relaxing fact validation.""" + empty = create_empty_memory() + summaries: dict[str, Any] = {} + for section_name, section_keys in ( + ("user", ("workContext", "personalContext", "topOfMind", "cognitiveStyle")), + ("history", ("recentMonths", "earlierContext", "longTermBackground")), + ): + incoming = data.get(section_name) + incoming = incoming if isinstance(incoming, dict) else {} + complete = copy.deepcopy(empty[section_name]) + for key, value in incoming.items(): + complete[key] = _normalize_context_section(value) if key in section_keys else copy.deepcopy(value) + for key in section_keys: + complete[key] = _normalize_context_section(complete.get(key)) + summaries[section_name] = complete + + return summaries + + +def normalize_memory_data(data: dict[str, Any]) -> dict[str, Any]: + """Return a canonical compatibility document without mutating the caller.""" + normalized = copy.deepcopy(data) if isinstance(data, dict) else {} + normalized.update(_normalize_memory_summaries(normalized)) + + facts = normalized.get("facts") + normalized["facts"] = [fact for value in facts if (fact := _normalize_legacy_import_fact(value)) is not None] if isinstance(facts, list) else [] + if not isinstance(normalized.get("version"), str): + normalized["version"] = "1.0" + if not isinstance(normalized.get("lastUpdated"), str): + normalized["lastUpdated"] = "" + return normalized + + def _has_meaningful_data(value: Any) -> bool: """Return whether a legacy summary value contains anything worth preserving.""" if isinstance(value, dict): @@ -1157,16 +1236,13 @@ class FileMemoryStorage(MemoryStorage): if not sources: return False, from_version, [] - base = global_memory or create_empty_memory() - migrated_summaries = { - "user": copy.deepcopy(base.get("user", {})), - "history": copy.deepcopy(base.get("history", {})), - } + migrated_summaries = _normalize_memory_summaries(global_memory or {}) if legacy_memory is not None and adopt_legacy_summaries: + legacy_summaries = _normalize_memory_summaries(legacy_memory) for section in ("user", "history"): migrated_summaries[section] = _merge_legacy_summary_section( canonical=migrated_summaries[section], - legacy=legacy_memory.get(section, {}), + legacy=legacy_summaries[section], section=section, legacy_path=legacy_path, ) @@ -1253,6 +1329,17 @@ class FileMemoryStorage(MemoryStorage): raise MemoryStorageCorruption(f"Legacy facts in {path} must be a list or mapping") result = {key: copy.deepcopy(value) for key, value in memory_file.items() if key != "facts"} result.setdefault("revision", 0) + summary_view = normalize_memory_data( + { + "version": result.get("version"), + "lastUpdated": result.get("lastUpdated"), + "user": result.get("user"), + "history": result.get("history"), + "facts": [], + } + ) + result["user"] = summary_view["user"] + result["history"] = summary_view["history"] result["facts"] = facts return result diff --git a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py index 3303c0d7b..e413ae545 100644 --- a/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py +++ b/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py @@ -32,6 +32,7 @@ from .storage import ( MemoryManifestRevisionConflict, MemoryStorage, create_empty_memory, + normalize_memory_data, utc_now_iso_z, ) @@ -963,19 +964,12 @@ class MemoryUpdater: """Persist imported memory data via the injected storage.""" if not isinstance(memory_data, dict): raise ValueError("memory_data") - memory_data = copy.deepcopy(memory_data) - empty = create_empty_memory() - for section in ("user", "history"): - incoming_section = memory_data.get(section, {}) - if not isinstance(incoming_section, dict): - raise ValueError(f"memory_data.{section}") - complete_section = copy.deepcopy(empty[section]) - for key, value in incoming_section.items(): - if key in complete_section and isinstance(complete_section[key], dict) and isinstance(value, dict): - complete_section[key].update(copy.deepcopy(value)) - else: - complete_section[key] = copy.deepcopy(value) - memory_data[section] = complete_section + # Replacement imports must not turn malformed facts into deletions. + # Validate before lenient compatibility normalization or any storage read. + raw_facts = memory_data.get("facts") + if not isinstance(raw_facts, list) or any(not isinstance(fact, dict) or not isinstance(fact.get("content"), str) or not fact["content"].strip() for fact in raw_facts): + raise ValueError("memory_data.facts must be a list of facts with non-empty content") + memory_data = normalize_memory_data(memory_data) if agent_name is not None and getattr(type(self._storage), "apply_changes", None) is not MemoryStorage.apply_changes: current = self.get_memory_data(agent_name, user_id=user_id) incoming_facts = copy.deepcopy(memory_data.get("facts", [])) @@ -1973,7 +1967,7 @@ class MemoryUpdater: # Update user sections user_updates = update_data.get("user", {}) - for section in ["workContext", "personalContext", "topOfMind"]: + for section in ["workContext", "personalContext", "topOfMind", "cognitiveStyle"]: section_data = user_updates.get(section, {}) if not isinstance(section_data, dict) or not section_data.get("shouldUpdate") or not section_data.get("summary"): continue diff --git a/backend/tests/test_deermem_self_contained.py b/backend/tests/test_deermem_self_contained.py index 5db21ff7d..6c5d14ae1 100644 --- a/backend/tests/test_deermem_self_contained.py +++ b/backend/tests/test_deermem_self_contained.py @@ -210,6 +210,7 @@ def test_import_empty_summary_sections_replace_existing_summaries_with_complete_ "workContext": {"summary": "", "updatedAt": ""}, "personalContext": {"summary": "", "updatedAt": ""}, "topOfMind": {"summary": "", "updatedAt": ""}, + "cognitiveStyle": {"summary": "", "updatedAt": ""}, } assert imported["history"] == { "recentMonths": {"summary": "", "updatedAt": ""}, @@ -930,3 +931,76 @@ def test_from_backend_config_null_values_do_not_warn_as_unknown(caplog): with caplog.at_level("WARNING", logger=cfg_logger): DeerMemConfig.from_backend_config({"model": None}) assert not any("Unknown backend_config keys" in r.message for r in caplog.records) + + +def test_apply_updates_cognitive_style_and_fact_category(deermem_data_dir) -> None: + dm = DeerMem(backend_config=None) + current_memory = dm.get_memory(user_id="cognitive-user") + + result = dm._updater._apply_updates( + current_memory, + { + "user": { + "cognitiveStyle": { + "summary": "Prefers conclusions first, then details.", + "shouldUpdate": True, + "scope": "user", + "authority": "descriptive", + } + }, + "newFacts": [ + { + "content": "User prefers conclusions before implementation details.", + "category": "cognitive", + "confidence": 0.92, + "scope": "user", + "durability": "durable", + "authority": "descriptive", + } + ], + }, + thread_id="thread-cognitive", + ) + + assert result["user"]["cognitiveStyle"]["summary"] == "Prefers conclusions first, then details." + assert result["user"]["cognitiveStyle"]["updatedAt"] + assert result["facts"][0]["category"] == "cognitive" + assert result["facts"][0]["source"] == "thread-cognitive" + + +def test_import_memory_persists_normalized_legacy_payload(deermem_data_dir) -> None: + dm = DeerMem(backend_config=None) + legacy = { + "version": "1.0", + "lastUpdated": "", + "user": {}, + "history": {}, + "facts": [{"content": "User prefers conclusions first.", "category": "cognitive"}], + } + + result = dm.import_memory(legacy, user_id="legacy-user") + + assert result["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""} + assert result["facts"][0]["id"].startswith("fact_") + assert result["facts"][0]["category"] == "cognitive" + assert result["facts"][0]["confidence"] == 0.5 + assert result["facts"][0]["createdAt"] == "" + assert result["facts"][0]["source"] == "unknown" + + +@pytest.mark.parametrize("facts", [[{"id": "keep", "content": " "}], [None], {}, None, "missing", [{"id": "keep", "content": 42}], [{"id": "new", "content": "valid"}, {}]]) +def test_replacement_import_rejects_unrecoverable_facts_without_writes(deermem_data_dir, facts): + dm = DeerMem(backend_config=None) + before = dm.import_memory({"user": {}, "history": {}, "facts": [{"id": "keep", "content": "Saved preference"}]}, user_id="alice") + snapshot = {str(path.relative_to(deermem_data_dir)): path.read_bytes() for path in deermem_data_dir.rglob("*") if path.is_file()} + + payload = {"user": {}, "history": {}, "facts": facts} + if facts == "missing": + payload.pop("facts") + with pytest.raises(ValueError, match="facts"): + dm.import_memory(payload, user_id="alice") + + assert {str(path.relative_to(deermem_data_dir)): path.read_bytes() for path in deermem_data_dir.rglob("*") if path.is_file()} == snapshot + reloaded = DeerMem(backend_config=None).get_memory(user_id="alice") + assert reloaded["facts"] == before["facts"] + assert reloaded["revision"] == before["revision"] diff --git a/backend/tests/test_memory_normalize.py b/backend/tests/test_memory_normalize.py new file mode 100644 index 000000000..ecaa9e0da --- /dev/null +++ b/backend/tests/test_memory_normalize.py @@ -0,0 +1,106 @@ +"""Tests for memory schema normalization.""" + +import copy + +import pytest + +from deerflow.agents.memory.backends.deermem.deermem.core.storage import create_empty_memory, normalize_memory_data + + +def test_normalize_memory_data_adds_cognitive_style() -> None: + legacy = { + "version": "1.0", + "lastUpdated": "", + "user": { + "workContext": {"summary": "work", "updatedAt": "2026-01-01T00:00:00Z"}, + "personalContext": {"summary": "", "updatedAt": ""}, + "topOfMind": {"summary": "", "updatedAt": ""}, + }, + "history": { + "recentMonths": {"summary": "", "updatedAt": ""}, + "earlierContext": {"summary": "", "updatedAt": ""}, + "longTermBackground": {"summary": "", "updatedAt": ""}, + }, + "facts": [], + } + + result = normalize_memory_data(legacy) + + assert "cognitiveStyle" in result["user"] + assert result["user"]["cognitiveStyle"]["summary"] == "" + assert result["user"]["cognitiveStyle"]["updatedAt"] == "" + + +def test_create_empty_memory_includes_cognitive_style() -> None: + empty = create_empty_memory() + assert empty["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""} + + +def test_normalize_memory_data_preserves_unknown_fields() -> None: + payload = { + "version": "1.0", + "revision": 7, + "lastUpdated": "2026-01-01T00:00:00Z", + "display": {"title": "Memory export"}, + "data": {"future": True}, + "user": { + "workContext": { + "summary": "work", + "updatedAt": "2026-01-01T00:00:00Z", + "confidence": 0.8, + }, + "providerState": {"loaded": True}, + }, + "history": { + "timeline": {"entries": ["2026-01"]}, + }, + "facts": [ + { + "content": "User prefers conclusions first.", + "category": "cognitive", + "topics": ["communication"], + } + ], + } + + result = normalize_memory_data(payload) + + assert result["revision"] == 7 + assert result["display"] == {"title": "Memory export"} + assert result["data"] == {"future": True} + assert result["user"]["providerState"] == {"loaded": True} + assert result["user"]["workContext"]["confidence"] == 0.8 + assert result["user"]["workContext"]["summary"] == "work" + assert result["history"]["timeline"] == {"entries": ["2026-01"]} + assert result["facts"][0]["topics"] == ["communication"] + assert result["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""} + + +def test_normalize_memory_data_does_not_mutate_caller() -> None: + payload = { + "version": "1.0", + "lastUpdated": "", + "user": {"workContext": {"summary": "work"}}, + "history": {}, + "facts": [{"content": "kept"}], + } + snapshot = copy.deepcopy(payload) + + result = normalize_memory_data(payload) + + assert result is not payload + assert payload == snapshot + + +@pytest.mark.parametrize("confidence,expected", [(None, 0.5), (True, 0.5), ("invalid", 0.5), (float("nan"), 0.5), (float("inf"), 0.5), (0, 0), (-1, 0), (2, 1), ("0.8", 0.8)]) +def test_normalize_legacy_fact_metadata(confidence, expected): + fact = {"id": "legacy", "content": " Keep conclusions first. ", "confidence": confidence, "source": " "} + result = normalize_memory_data({"facts": [fact]})["facts"][0] + assert result["confidence"] == expected + assert result["content"] == "Keep conclusions first." + assert result["source"] == "unknown" + + +def test_normalize_missing_fact_confidence_uses_neutral_default(): + result = normalize_memory_data({"facts": [{"content": "Legacy preference"}]}) + assert result["facts"][0]["confidence"] == 0.5 diff --git a/backend/tests/test_memory_prompt_injection.py b/backend/tests/test_memory_prompt_injection.py index 5bc9086bc..a0611c180 100644 --- a/backend/tests/test_memory_prompt_injection.py +++ b/backend/tests/test_memory_prompt_injection.py @@ -4,7 +4,12 @@ import math import pytest -from deerflow.agents.memory.backends.deermem.deermem.core.prompt import _coerce_confidence, format_memory_for_injection +from deerflow.agents.memory.backends.deermem.deermem.core.prompt import ( + FACT_EXTRACTION_PROMPT, + _coerce_confidence, + format_memory_for_injection, + load_prompt_messages, +) def test_format_memory_includes_facts_section() -> None: @@ -820,3 +825,56 @@ def test_format_memory_tolerates_non_string_summary() -> None: result = format_memory_for_injection(memory_data, max_tokens=2000) assert "Current Focus: 12345" in result + + +def test_format_memory_includes_cognitive_style() -> None: + memory_data = { + "user": { + "cognitiveStyle": { + "summary": "Prefers conclusions first, then details.", + "updatedAt": "2026-01-01T00:00:00Z", + } + }, + "history": {}, + "facts": [], + } + + result = format_memory_for_injection(memory_data, max_tokens=2000) + + assert "Thinking Style:" in result + assert "Prefers conclusions first, then details." in result + + +def test_cognitive_fact_category_is_documented_and_rendered() -> None: + messages = load_prompt_messages( + "memory_update", + { + "current_memory": "{}", + "conversation": "", + "correction_hint": "", + "staleness_review_section": "", + "consolidation_section": "", + }, + ) + memory_update_prompt = messages[0].content + assert isinstance(memory_update_prompt, str) + assert "cognitive|goal|correction" in memory_update_prompt + assert "cognitive|goal|correction" in FACT_EXTRACTION_PROMPT + assert "- cognitive:" in FACT_EXTRACTION_PROMPT + + result = format_memory_for_injection( + { + "user": {}, + "history": {}, + "facts": [ + { + "content": "User prefers conclusions first.", + "category": "cognitive", + "confidence": 0.9, + } + ], + }, + max_tokens=2000, + ) + + assert "[cognitive | 0.90] User prefers conclusions first." in result diff --git a/backend/tests/test_memory_router.py b/backend/tests/test_memory_router.py index 517744b7b..6afc05a08 100644 --- a/backend/tests/test_memory_router.py +++ b/backend/tests/test_memory_router.py @@ -587,3 +587,19 @@ def test_reload_memory_route_returns_501_when_read_also_unsupported() -> None: with TestClient(app) as client: response = client.post("/api/memory/reload") assert response.status_code == 501 + + +def test_import_blank_fact_returns_400_without_replacing_saved_memory(tmp_path): + manager = DeerMem(backend_config={"storage_path": str(tmp_path)}) + before = manager.import_memory(_sample_memory(facts=[{"id": "keep", "content": "Saved preference"}]), user_id="alice") + payload = _sample_memory(facts=[{**before["facts"][0], "content": " "}]) + app = make_authed_test_app() + app.include_router(memory.router) + with ( + patch("app.gateway.routers.memory.get_memory_manager", return_value=manager), + patch("app.gateway.routers.memory.get_effective_user_id", return_value="alice"), + TestClient(app) as client, + ): + response = client.post("/api/memory/import", json=payload) + assert response.status_code == 400 + assert manager.get_memory(user_id="alice") == before diff --git a/backend/tests/test_memory_storage.py b/backend/tests/test_memory_storage.py index fd8dbcbd5..00c3a4c29 100644 --- a/backend/tests/test_memory_storage.py +++ b/backend/tests/test_memory_storage.py @@ -15,13 +15,14 @@ from deerflow.agents.memory.backends.deermem.deermem.core.storage import ( MemoryStorage, create_empty_memory, create_storage, + normalize_memory_data, ) def _storage_at(memory_file) -> FileMemoryStorage: - """A FileMemoryStorage whose absolute storage_path is a single shared file.""" - resolved = str(memory_file.resolve()) - return FileMemoryStorage(DeerMemConfig(storage_path=resolved)) + """A FileMemoryStorage rooted at the directory containing ``memory_file``.""" + root = str(memory_file.parent.resolve()) + return FileMemoryStorage(DeerMemConfig(storage_path=root)) class TestCreateEmptyMemory: @@ -37,6 +38,45 @@ class TestCreateEmptyMemory: assert isinstance(memory["facts"], list) +class TestNormalizeMemoryData: + """Test backward-compatible memory schema normalization.""" + + def test_normalizes_legacy_facts_without_mutating_input(self): + legacy = { + "version": "1.0", + "lastUpdated": "", + "user": {}, + "history": {}, + "facts": [ + {"content": "User prefers conclusions first", "category": "cognitive"}, + None, + {"category": "context"}, + ], + } + + normalized = normalize_memory_data(legacy) + + assert legacy == { + "version": "1.0", + "lastUpdated": "", + "user": {}, + "history": {}, + "facts": [ + {"content": "User prefers conclusions first", "category": "cognitive"}, + None, + {"category": "context"}, + ], + } + assert len(normalized["facts"]) == 1 + fact = normalized["facts"][0] + assert fact["id"].startswith("fact_") + assert fact["content"] == "User prefers conclusions first" + assert fact["category"] == "cognitive" + assert fact["confidence"] == 0.5 + assert fact["createdAt"] == "" + assert fact["source"] == "unknown" + + class TestMemoryStorageInterface: """Test MemoryStorage abstract base class.""" @@ -183,6 +223,36 @@ class TestCreateStorage: assert isinstance(storage, FileMemoryStorage) +def test_load_normalizes_legacy_json_without_cognitive_style(tmp_path) -> None: + memory_file = tmp_path / "memory.json" + memory_file.write_text( + '{"version":"1.0","lastUpdated":"","user":{"workContext":{"summary":"work","updatedAt":""}},"history":{},"facts":[]}', + encoding="utf-8", + ) + storage = _storage_at(memory_file) + + loaded = storage.load() + + assert loaded["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""} + assert loaded["user"]["workContext"]["summary"] == "work" + + +def test_cache_hit_returns_an_equivalent_normalized_copy(tmp_path) -> None: + memory_file = tmp_path / "memory.json" + memory_file.write_text( + '{"version":"1.0","lastUpdated":"","user":{},"history":{},"facts":[{"content":"Legacy cached fact"}]}', + encoding="utf-8", + ) + + storage = _storage_at(memory_file) + first = storage.load() + second = storage.load() + + assert second == first + assert second is not first + assert second["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""} + + class TestMarkdownMemoryStorage: """Opt-in ``storage_class="markdown"``: tolerant load path (issue #3124).""" diff --git a/backend/tests/test_memory_storage_markdown.py b/backend/tests/test_memory_storage_markdown.py index 35589cfff..dd4c80af5 100644 --- a/backend/tests/test_memory_storage_markdown.py +++ b/backend/tests/test_memory_storage_markdown.py @@ -1187,3 +1187,44 @@ def test_windows_lock_file_does_not_grow_per_acquisition(storage: FileMemoryStor assert storage.save(create_empty_memory(), user_id="alice") lock_path = storage._get_memory_file_path(user_id="alice").parent / ".memory.lock" assert lock_path.stat().st_size == 1 + + +@pytest.mark.parametrize("global_version", ["1.0", "2.0"]) +def test_load_migrates_identical_pre_cognitive_summaries(storage, global_version): + memory_path = storage._get_memory_file_path("__default__", user_id="alice") + legacy_path = memory_path.parent / "agents" / "__default__" / "memory.json" + legacy_path.parent.mkdir(parents=True) + summaries = { + "user": {"workContext": {"summary": "historical profile", "updatedAt": "then", "confidence": 0.8}, "providerState": {"key": "kept"}}, + "history": {"recentMonths": {"summary": "historical context", "updatedAt": "then"}, "timeline": ["kept"]}, + } + global_data = {"version": global_version, "revision": 0, "lastUpdated": "", **copy.deepcopy(summaries)} + if global_version == "1.0": + global_data["facts"] = [] + memory_path.write_text(json.dumps(global_data), encoding="utf-8") + fact = _memory_with_fact()["facts"][0] + legacy_path.write_text(json.dumps({"version": "1.0", **summaries, "facts": [fact]}), encoding="utf-8") + + loaded = storage.load("__default__", user_id="alice") + + assert [item["id"] for item in loaded["facts"]] == [fact["id"]] + assert loaded["user"]["workContext"] == summaries["user"]["workContext"] + assert loaded["user"]["providerState"] == summaries["user"]["providerState"] + assert loaded["history"]["timeline"] == ["kept"] + assert loaded["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""} + assert not legacy_path.exists() + + +@pytest.mark.parametrize("facts,error_type", [([None], MemoryStorageCorruption), ([{"id": "invalid", "content": " "}], ValueError)]) +def test_summary_normalization_does_not_relax_legacy_fact_migration(storage, facts, error_type): + memory_path = storage._get_memory_file_path("__default__", user_id="alice") + legacy_path = memory_path.parent / "agents" / "__default__" / "memory.json" + legacy_path.parent.mkdir(parents=True) + original = json.dumps({"version": "1.0", "user": {}, "history": {}, "facts": facts}) + legacy_path.write_text(original, encoding="utf-8") + + with pytest.raises(error_type): + storage.load("__default__", user_id="alice") + + assert legacy_path.read_text(encoding="utf-8") == original + assert not memory_path.exists() diff --git a/backend/tests/test_memory_updater.py b/backend/tests/test_memory_updater.py index 42a0c4464..2f856f7a9 100644 --- a/backend/tests/test_memory_updater.py +++ b/backend/tests/test_memory_updater.py @@ -31,6 +31,7 @@ def _make_memory(facts: list[dict[str, object]] | None = None) -> dict[str, obje "workContext": {"summary": "", "updatedAt": ""}, "personalContext": {"summary": "", "updatedAt": ""}, "topOfMind": {"summary": "", "updatedAt": ""}, + "cognitiveStyle": {"summary": "", "updatedAt": ""}, }, "history": { "recentMonths": {"summary": "", "updatedAt": ""}, diff --git a/frontend/src/components/workspace/settings/memory-settings-page.tsx b/frontend/src/components/workspace/settings/memory-settings-page.tsx index 982090f8b..501c39240 100644 --- a/frontend/src/components/workspace/settings/memory-settings-page.tsx +++ b/frontend/src/components/workspace/settings/memory-settings-page.tsx @@ -34,6 +34,7 @@ import { useMemory, useUpdateMemoryFact, } from "@/core/memory/hooks"; +import { normalizeMemoryPayload } from "@/core/memory/import-memory"; import type { MemoryFactInput, MemoryFactPatchInput, @@ -68,60 +69,6 @@ type PendingImport = { memory: UserMemory; }; -function isRecord(value: unknown): value is Record { - return typeof value === "object" && value !== null; -} - -function isMemorySection(value: unknown): value is { - summary: string; - updatedAt: string; -} { - return ( - isRecord(value) && - typeof value.summary === "string" && - typeof value.updatedAt === "string" - ); -} - -function isMemoryFact(value: unknown): value is UserMemory["facts"][number] { - return ( - isRecord(value) && - typeof value.id === "string" && - typeof value.content === "string" && - typeof value.category === "string" && - typeof value.confidence === "number" && - Number.isFinite(value.confidence) && - typeof value.createdAt === "string" && - typeof value.source === "string" - ); -} - -function isImportedMemory(value: unknown): value is UserMemory { - if (!isRecord(value)) { - return false; - } - - if ( - typeof value.version !== "string" || - typeof value.lastUpdated !== "string" || - !isRecord(value.user) || - !isRecord(value.history) || - !Array.isArray(value.facts) - ) { - return false; - } - - return ( - isMemorySection(value.user.workContext) && - isMemorySection(value.user.personalContext) && - isMemorySection(value.user.topOfMind) && - isMemorySection(value.history.recentMonths) && - isMemorySection(value.history.earlierContext) && - isMemorySection(value.history.longTermBackground) && - value.facts.every(isMemoryFact) - ); -} - type FactFormState = { content: string; category: string; @@ -189,6 +136,11 @@ function buildMemorySectionGroups( summary: memory.user.topOfMind.summary, updatedAt: memory.user.topOfMind.updatedAt, }, + { + title: t.settings.memory.markdown.cognitiveStyle, + summary: memory.user.cognitiveStyle.summary, + updatedAt: memory.user.cognitiveStyle.updatedAt, + }, ], }, { @@ -255,6 +207,7 @@ function isMemorySummaryEmpty(memory: UserMemory) { memory.user.workContext.summary.trim() === "" && memory.user.personalContext.summary.trim() === "" && memory.user.topOfMind.summary.trim() === "" && + memory.user.cognitiveStyle.summary.trim() === "" && memory.history.recentMonths.summary.trim() === "" && memory.history.earlierContext.summary.trim() === "" && memory.history.longTermBackground.summary.trim() === "" @@ -277,6 +230,13 @@ function upperFirst(str: string) { return str.charAt(0).toUpperCase() + str.slice(1); } +function formatFactCreatedAt(createdAt: string, unknownLabel: string) { + if (!createdAt || Number.isNaN(Date.parse(createdAt))) { + return unknownLabel; + } + return formatTimeAgo(createdAt); +} + export function MemorySettingsPage() { const { t } = useI18n(); const { memory, isLoading, error } = useMemory(); @@ -424,13 +384,14 @@ export function MemorySettingsPage() { try { const parsed: unknown = JSON.parse(await file.text()); - if (!isImportedMemory(parsed)) { + const memory = normalizeMemoryPayload(parsed); + if (!memory) { toast.error(t.settings.memory.importInvalidFile); return; } setPendingImport({ fileName: file.name, - memory: parsed, + memory, }); } catch { toast.error(t.settings.memory.importInvalidFile); @@ -703,7 +664,10 @@ export function MemorySettingsPage() { {t.settings.memory.markdown.table.createdAt}: {" "} - {formatTimeAgo(fact.createdAt)} + {formatFactCreatedAt( + fact.createdAt, + t.settings.memory.markdown.table.unknown, + )} @@ -711,13 +675,15 @@ export function MemorySettingsPage() { {" "} {fact.source === "manual" ? ( t.settings.memory.manualFactSource - ) : ( + ) : fact.source && fact.source !== "unknown" ? ( {t.settings.memory.markdown.table.view} + ) : ( + t.settings.memory.markdown.table.unknown )} diff --git a/frontend/src/core/api/static-response.ts b/frontend/src/core/api/static-response.ts index 1ab8430f7..38a2a70eb 100644 --- a/frontend/src/core/api/static-response.ts +++ b/frontend/src/core/api/static-response.ts @@ -99,7 +99,12 @@ export async function staticApiResponse( data = { version: "1.0", lastUpdated: "", - user: { workContext: empty, personalContext: empty, topOfMind: empty }, + user: { + workContext: empty, + personalContext: empty, + topOfMind: empty, + cognitiveStyle: empty, + }, history: { recentMonths: empty, earlierContext: empty, diff --git a/frontend/src/core/i18n/locales/en-US.ts b/frontend/src/core/i18n/locales/en-US.ts index 1834bbacd..f30976cf3 100644 --- a/frontend/src/core/i18n/locales/en-US.ts +++ b/frontend/src/core/i18n/locales/en-US.ts @@ -1416,6 +1416,7 @@ export const enUS: Translations = { work: "Work", personal: "Personal", topOfMind: "Top of mind", + cognitiveStyle: "Thinking style", historyBackground: "History", recentMonths: "Recent months", earlierContext: "Earlier context", @@ -1434,6 +1435,7 @@ export const enUS: Translations = { }, content: "Content", source: "Source", + unknown: "Unknown", createdAt: "CreatedAt", view: "View", }, diff --git a/frontend/src/core/i18n/locales/types.ts b/frontend/src/core/i18n/locales/types.ts index ca7fa52f4..dfbbd5d4f 100644 --- a/frontend/src/core/i18n/locales/types.ts +++ b/frontend/src/core/i18n/locales/types.ts @@ -1203,6 +1203,7 @@ export interface Translations { work: string; personal: string; topOfMind: string; + cognitiveStyle: string; historyBackground: string; recentMonths: string; earlierContext: string; @@ -1221,6 +1222,7 @@ export interface Translations { }; content: string; source: string; + unknown: string; createdAt: string; view: string; }; diff --git a/frontend/src/core/i18n/locales/zh-CN.ts b/frontend/src/core/i18n/locales/zh-CN.ts index 0763b697a..2729fcd5b 100644 --- a/frontend/src/core/i18n/locales/zh-CN.ts +++ b/frontend/src/core/i18n/locales/zh-CN.ts @@ -1332,6 +1332,7 @@ export const zhCN: Translations = { work: "工作", personal: "个人", topOfMind: "近期关注(Top of mind)", + cognitiveStyle: "思维习惯", historyBackground: "历史背景", recentMonths: "近几个月", earlierContext: "更早上下文", @@ -1350,6 +1351,7 @@ export const zhCN: Translations = { }, content: "内容", source: "来源", + unknown: "未知", createdAt: "创建时间", view: "查看", }, diff --git a/frontend/src/core/memory/api.ts b/frontend/src/core/memory/api.ts index a68a0bab2..b2c957cd1 100644 --- a/frontend/src/core/memory/api.ts +++ b/frontend/src/core/memory/api.ts @@ -1,6 +1,7 @@ import { fetch } from "../api/fetcher"; import { getBackendBaseURL } from "../config"; +import { normalizeUserMemory } from "./normalize"; import type { MemoryFactInput, MemoryFactPatchInput, @@ -77,7 +78,7 @@ async function readMemoryResponse( ); } - return response.json() as Promise; + return normalizeUserMemory(await response.json()); } export async function loadMemory(): Promise { diff --git a/frontend/src/core/memory/import-memory.ts b/frontend/src/core/memory/import-memory.ts new file mode 100644 index 000000000..305e730e0 --- /dev/null +++ b/frontend/src/core/memory/import-memory.ts @@ -0,0 +1,170 @@ +import type { UserMemory } from "./types"; + +type ContextSection = UserMemory["user"]["workContext"]; +type MemoryFact = UserMemory["facts"][number]; +type InvalidFactStrategy = "reject" | "drop"; + +const USER_SECTION_KEYS = [ + "workContext", + "personalContext", + "topOfMind", + "cognitiveStyle", +] as const satisfies ReadonlyArray; + +const HISTORY_SECTION_KEYS = [ + "recentMonths", + "earlierContext", + "longTermBackground", +] as const satisfies ReadonlyArray; + +function emptySection(): ContextSection { + return { summary: "", updatedAt: "" }; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function normalizeContextSection(value: unknown): ContextSection { + if (!isRecord(value)) { + return emptySection(); + } + + return { + ...value, + summary: typeof value.summary === "string" ? value.summary : "", + updatedAt: typeof value.updatedAt === "string" ? value.updatedAt : "", + } as ContextSection; +} + +function generateLegacyFactId(index: number): string { + const randomUUID = globalThis.crypto?.randomUUID?.(); + return randomUUID + ? `fact_${randomUUID.replaceAll("-", "").slice(0, 8)}` + : `fact_legacy_${index}`; +} + +function normalizeMemoryFact(value: unknown, index: number): MemoryFact | null { + if (!isRecord(value)) { + return null; + } + + const content = typeof value.content === "string" ? value.content.trim() : ""; + if (!content) { + return null; + } + + const category = + typeof value.category === "string" && value.category.trim() + ? value.category.trim() + : "context"; + const rawConfidence = value.confidence; + const numericConfidence = + typeof rawConfidence === "number" + ? rawConfidence + : typeof rawConfidence === "string" && + /^[+-]?(?:\d+\.?\d*|\.\d+)(?:e[+-]?\d+)?$/i.test(rawConfidence.trim()) + ? Number(rawConfidence) + : NaN; + const confidence = Number.isFinite(numericConfidence) + ? Math.min(1, Math.max(0, numericConfidence)) + : 0.5; + + const fact = { + ...value, + id: + typeof value.id === "string" && value.id.trim() + ? value.id.trim() + : generateLegacyFactId(index), + content, + category, + confidence, + createdAt: + typeof value.createdAt === "string" ? value.createdAt.trim() : "", + source: + typeof value.source === "string" && value.source.trim() + ? value.source.trim() + : "unknown", + } as MemoryFact & Record; + + if ( + "sourceError" in fact && + fact.sourceError !== null && + typeof fact.sourceError !== "string" + ) { + delete fact.sourceError; + } + + return fact; +} + +/** + * Normalize and validate memory JSON (unknown → UserMemory | null). + * + * Normalization is additive: only contract-owned fields are validated and + * defaulted, while every unrecognized field (top-level, section, and per-fact) + * passes through untouched. The frontend must never be narrower than the + * Gateway contract — `MemoryResponse` declares fields such as the top-level + * `revision`, and rebuilding from a whitelist here would silently drop them + * from every response passing through `readMemoryResponse()`. + * + * The envelope (string `version`/`lastUpdated`, record `user`/`history`, array + * `facts`) is strict on both call paths. Unrecoverable facts can either reject + * a user-initiated import or be dropped on the background API read path. + */ +export function normalizeMemoryPayload( + value: unknown, + options: { invalidFactStrategy?: InvalidFactStrategy } = {}, +): UserMemory | null { + if ( + !isRecord(value) || + typeof value.version !== "string" || + typeof value.lastUpdated !== "string" || + !isRecord(value.user) || + !isRecord(value.history) || + !Array.isArray(value.facts) + ) { + return null; + } + + const user = value.user; + const history = value.history; + const invalidFactStrategy = options.invalidFactStrategy ?? "reject"; + const facts: MemoryFact[] = []; + + for (const [index, factValue] of value.facts.entries()) { + const fact = normalizeMemoryFact(factValue, index); + if (!fact) { + if (invalidFactStrategy === "reject") { + return null; + } + continue; + } + facts.push(fact); + } + + const normalizedUser = { + ...user, + ...Object.fromEntries( + USER_SECTION_KEYS.map((key) => [key, normalizeContextSection(user[key])]), + ), + } as unknown as UserMemory["user"]; + const normalizedHistory = { + ...history, + ...Object.fromEntries( + HISTORY_SECTION_KEYS.map((key) => [ + key, + normalizeContextSection(history[key]), + ]), + ), + } as unknown as UserMemory["history"]; + + return { + ...value, + version: value.version, + lastUpdated: value.lastUpdated, + user: normalizedUser, + history: normalizedHistory, + facts, + } as unknown as UserMemory; +} diff --git a/frontend/src/core/memory/normalize.ts b/frontend/src/core/memory/normalize.ts new file mode 100644 index 000000000..37a748ee7 --- /dev/null +++ b/frontend/src/core/memory/normalize.ts @@ -0,0 +1,13 @@ +import { normalizeMemoryPayload } from "./import-memory"; +import type { UserMemory } from "./types"; + +/** Normalize API/import payloads (unknown → UserMemory). Throws if invalid. */ +export function normalizeUserMemory(value: unknown): UserMemory { + const normalized = normalizeMemoryPayload(value, { + invalidFactStrategy: "drop", + }); + if (!normalized) { + throw new Error("Invalid memory payload"); + } + return normalized; +} diff --git a/frontend/src/core/memory/types.ts b/frontend/src/core/memory/types.ts index b69f29f20..71d7f074a 100644 --- a/frontend/src/core/memory/types.ts +++ b/frontend/src/core/memory/types.ts @@ -5,6 +5,7 @@ export interface MemoryFact { confidence: number; createdAt: string; source: string; + [key: string]: unknown; } export interface MemoryFactInput { @@ -21,6 +22,7 @@ export interface MemoryFactPatchInput { export interface UserMemory { version: string; + revision?: number; lastUpdated: string; user: { workContext: { @@ -35,6 +37,10 @@ export interface UserMemory { summary: string; updatedAt: string; }; + cognitiveStyle: { + summary: string; + updatedAt: string; + }; }; history: { recentMonths: { diff --git a/frontend/tests/e2e/settings-memory-import.spec.ts b/frontend/tests/e2e/settings-memory-import.spec.ts new file mode 100644 index 000000000..0cd897d9a --- /dev/null +++ b/frontend/tests/e2e/settings-memory-import.spec.ts @@ -0,0 +1,194 @@ +import { expect, test, type Page } from "@playwright/test"; + +import { mockLangGraphAPI } from "./utils/mock-api"; + +const EMPTY_MEMORY = { + version: "1.0", + lastUpdated: "", + user: { + workContext: { summary: "", updatedAt: "" }, + personalContext: { summary: "", updatedAt: "" }, + topOfMind: { summary: "", updatedAt: "" }, + cognitiveStyle: { summary: "", updatedAt: "" }, + }, + history: { + recentMonths: { summary: "", updatedAt: "" }, + earlierContext: { summary: "", updatedAt: "" }, + longTermBackground: { summary: "", updatedAt: "" }, + }, + facts: [], +}; + +const LEGACY_MEMORY_WITHOUT_COGNITIVE_STYLE = { + version: "1.0", + lastUpdated: "2026-01-01T00:00:00Z", + user: { + workContext: { summary: "Works on DeerFlow", updatedAt: "" }, + personalContext: { summary: "", updatedAt: "" }, + topOfMind: { summary: "Memory import compatibility", updatedAt: "" }, + }, + history: { + recentMonths: { summary: "", updatedAt: "" }, + earlierContext: { summary: "", updatedAt: "" }, + longTermBackground: { summary: "", updatedAt: "" }, + }, + facts: [ + { + content: "User prefers conclusions first.", + category: "cognitive", + }, + ], +}; + +async function openMemorySettings(page: Page) { + mockLangGraphAPI(page); + await page.route(/\/api\/memory$/, async (route) => { + if (route.request().method() === "GET") { + await route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify(EMPTY_MEMORY), + }); + return; + } + await route.fallback(); + }); + + await page.goto("/workspace/chats/new"); + const sidebar = page.locator("[data-sidebar='sidebar']"); + await sidebar.getByRole("button", { name: /Settings and more/ }).click(); + await page.getByRole("menuitem", { name: "Settings" }).click(); + + const settingsDialog = page.getByRole("dialog", { name: "Settings" }); + await expect(settingsDialog).toBeVisible(); + await settingsDialog.getByRole("button", { name: "Memory" }).click(); + await expect( + settingsDialog.getByRole("button", { name: "Import memory" }), + ).toBeVisible(); + return settingsDialog; +} + +async function selectMemoryFile( + settingsDialog: ReturnType, + fileName: string, + payload: unknown, +) { + await settingsDialog.locator('input[type="file"]').setInputFiles({ + name: fileName, + mimeType: "application/json", + buffer: Buffer.from(JSON.stringify(payload)), + }); +} + +const invalidImports = [ + { + name: "facts-only JSON", + fileName: "facts-only.json", + payload: { facts: [] }, + }, + { + name: "missing metadata", + fileName: "missing-metadata.json", + payload: { user: {}, history: {}, facts: [] }, + }, + { + name: "non-object user/history", + fileName: "invalid-sections.json", + payload: { + version: "1.0", + lastUpdated: "2026-07-17T00:00:00Z", + user: "not-an-object", + history: 123, + facts: [], + }, + }, +]; + +test.describe("Memory settings import validation", () => { + for (const invalidImport of invalidImports) { + test(`does not enable confirmation for ${invalidImport.name}`, async ({ + page, + }) => { + const settingsDialog = await openMemorySettings(page); + + await selectMemoryFile( + settingsDialog, + invalidImport.fileName, + invalidImport.payload, + ); + + await expect( + page.getByText( + "Failed to read the selected memory file. Please choose a valid JSON export.", + ), + ).toBeVisible(); + await expect( + page.getByRole("dialog", { name: "Import memory?" }), + ).toHaveCount(0); + }); + } + + test("keeps confirmation available for a legacy export missing cognitiveStyle", async ({ + page, + }) => { + const settingsDialog = await openMemorySettings(page); + + await selectMemoryFile( + settingsDialog, + "legacy-without-cognitive-style.json", + LEGACY_MEMORY_WITHOUT_COGNITIVE_STYLE, + ); + + const confirmDialog = page.getByRole("dialog", { name: "Import memory?" }); + await expect(confirmDialog).toBeVisible(); + await expect(confirmDialog).toContainText( + "legacy-without-cognitive-style.json", + ); + await expect( + confirmDialog.getByRole("button", { name: "Import" }), + ).toBeEnabled(); + }); + + test("round-trips unknown fields through the import request", async ({ + page, + }) => { + const memoryWithExtensions = { + ...LEGACY_MEMORY_WITHOUT_COGNITIVE_STYLE, + revision: 4, + display: { title: "Memory export" }, + data: { future: true }, + }; + const importedPayloads: unknown[] = []; + await page.route(/\/api\/memory\/import$/, async (route) => { + if (route.request().method() === "POST") { + importedPayloads.push(route.request().postDataJSON()); + await route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify(memoryWithExtensions), + }); + return; + } + await route.fallback(); + }); + + const settingsDialog = await openMemorySettings(page); + await selectMemoryFile( + settingsDialog, + "with-extensions.json", + memoryWithExtensions, + ); + + const confirmDialog = page.getByRole("dialog", { name: "Import memory?" }); + await expect(confirmDialog).toBeVisible(); + await confirmDialog.getByRole("button", { name: "Import" }).click(); + + await expect.poll(() => importedPayloads.length).toBe(1); + expect(importedPayloads[0]).toMatchObject({ + revision: 4, + display: { title: "Memory export" }, + data: { future: true }, + user: { cognitiveStyle: { summary: "", updatedAt: "" } }, + }); + }); +}); diff --git a/frontend/tests/unit/core/memory/fixtures.ts b/frontend/tests/unit/core/memory/fixtures.ts new file mode 100644 index 000000000..d4c5b3575 --- /dev/null +++ b/frontend/tests/unit/core/memory/fixtures.ts @@ -0,0 +1,45 @@ +import type { UserMemory } from "@/core/memory/types"; + +/** Legacy export shape before user.cognitiveStyle existed (PR #3182). */ +export function legacyMemoryWithoutCognitiveStyle(): Omit< + UserMemory, + "user" +> & { + user: Omit & { + workContext: UserMemory["user"]["workContext"]; + personalContext: UserMemory["user"]["personalContext"]; + topOfMind: UserMemory["user"]["topOfMind"]; + }; +} { + return { + version: "1.0", + lastUpdated: "2026-01-01T00:00:00Z", + user: { + workContext: { + summary: "Works on DeerFlow", + updatedAt: "2026-01-01T00:00:00Z", + }, + personalContext: { summary: "", updatedAt: "" }, + topOfMind: { summary: "Memory import compatibility", updatedAt: "" }, + }, + history: { + recentMonths: { summary: "", updatedAt: "" }, + earlierContext: { summary: "", updatedAt: "" }, + longTermBackground: { summary: "", updatedAt: "" }, + }, + facts: [], + }; +} + +/** Legacy payload whose facts predate generated ids and source/timestamp metadata. */ +export function legacyMemoryWithIncompleteFacts() { + return { + ...legacyMemoryWithoutCognitiveStyle(), + facts: [ + { + content: "User prefers conclusions first.", + category: "cognitive", + }, + ], + }; +} diff --git a/frontend/tests/unit/core/memory/import-memory.test.ts b/frontend/tests/unit/core/memory/import-memory.test.ts new file mode 100644 index 000000000..ad607db8c --- /dev/null +++ b/frontend/tests/unit/core/memory/import-memory.test.ts @@ -0,0 +1,262 @@ +import { describe, expect, it } from "@rstest/core"; + +import { normalizeMemoryPayload } from "@/core/memory/import-memory"; + +import { + legacyMemoryWithIncompleteFacts, + legacyMemoryWithoutCognitiveStyle, +} from "./fixtures"; + +/** Legacy strict guard (pre–normalizeMemoryPayload): required every section to exist. */ +function isImportedMemoryStrict(value: unknown): boolean { + if (typeof value !== "object" || value === null) { + return false; + } + + const record = value as Record; + if ( + typeof record.version !== "string" || + typeof record.lastUpdated !== "string" || + typeof record.user !== "object" || + record.user === null || + typeof record.history !== "object" || + record.history === null || + !Array.isArray(record.facts) + ) { + return false; + } + + const user = record.user as Record; + const history = record.history as Record; + + function isSection(section: unknown): boolean { + if (typeof section !== "object" || section === null) { + return false; + } + const s = section as Record; + return typeof s.summary === "string" && typeof s.updatedAt === "string"; + } + + return ( + isSection(user.workContext) && + isSection(user.personalContext) && + isSection(user.topOfMind) && + isSection(user.cognitiveStyle) && + isSection(history.recentMonths) && + isSection(history.earlierContext) && + isSection(history.longTermBackground) + ); +} + +describe("legacy memory import compatibility (TDD)", () => { + const legacy = legacyMemoryWithoutCognitiveStyle(); + + it("legacy strict guard would reject exports missing cognitiveStyle", () => { + expect(isImportedMemoryStrict(legacy)).toBe(false); + }); + + it("normalizeMemoryPayload accepts legacy export without cognitiveStyle", () => { + const result = normalizeMemoryPayload(legacy); + + expect(result).not.toBeNull(); + expect(result!.user.cognitiveStyle).toEqual({ + summary: "", + updatedAt: "", + }); + expect(result!.user.workContext.summary).toBe("Works on DeerFlow"); + }); + + it("normalizeMemoryPayload preserves existing cognitiveStyle summary", () => { + const withStyle = { + ...legacy, + user: { + ...legacy.user, + cognitiveStyle: { + summary: "Conclusions first, then details.", + updatedAt: "2026-02-01T00:00:00Z", + }, + }, + }; + + const result = normalizeMemoryPayload(withStyle); + + expect(result).not.toBeNull(); + expect(result!.user.cognitiveStyle.summary).toBe( + "Conclusions first, then details.", + ); + }); + + it("normalizeMemoryPayload returns null for non-object payloads", () => { + expect(normalizeMemoryPayload(null)).toBeNull(); + expect(normalizeMemoryPayload("not-json")).toBeNull(); + expect(normalizeMemoryPayload({})).toBeNull(); + }); + + it('rejects the truncated import envelope {"facts": []}', () => { + expect(normalizeMemoryPayload({ facts: [] })).toBeNull(); + }); + + it("rejects imports with missing or malformed metadata", () => { + const valid = legacyMemoryWithoutCognitiveStyle(); + + expect( + normalizeMemoryPayload({ + user: valid.user, + history: valid.history, + facts: valid.facts, + }), + ).toBeNull(); + expect(normalizeMemoryPayload({ ...valid, version: 1 })).toBeNull(); + expect(normalizeMemoryPayload({ ...valid, lastUpdated: null })).toBeNull(); + }); + + it("rejects imports with missing or non-record user/history envelopes", () => { + const valid = legacyMemoryWithoutCognitiveStyle(); + + expect(normalizeMemoryPayload({ ...valid, user: undefined })).toBeNull(); + expect(normalizeMemoryPayload({ ...valid, history: undefined })).toBeNull(); + expect( + normalizeMemoryPayload({ ...valid, user: "not-an-object" }), + ).toBeNull(); + expect(normalizeMemoryPayload({ ...valid, history: 123 })).toBeNull(); + expect(normalizeMemoryPayload({ ...valid, user: [] })).toBeNull(); + expect(normalizeMemoryPayload({ ...valid, history: [] })).toBeNull(); + }); + + it("normalizeMemoryPayload fills missing user sections like backend normalize", () => { + const partial = { + version: "1.0", + lastUpdated: "", + user: { + workContext: { summary: "only work", updatedAt: "" }, + }, + history: {}, + facts: [], + }; + + const result = normalizeMemoryPayload(partial); + + expect(result).not.toBeNull(); + expect(result!.user.workContext.summary).toBe("only work"); + expect(result!.user.personalContext).toEqual({ + summary: "", + updatedAt: "", + }); + expect(result!.user.topOfMind).toEqual({ summary: "", updatedAt: "" }); + expect(result!.user.cognitiveStyle).toEqual({ + summary: "", + updatedAt: "", + }); + expect(result!.history.recentMonths).toEqual({ + summary: "", + updatedAt: "", + }); + }); + + it("normalizes legacy facts that are missing generated metadata", () => { + const result = normalizeMemoryPayload(legacyMemoryWithIncompleteFacts()); + + expect(result).not.toBeNull(); + expect(result!.facts).toHaveLength(1); + expect(result!.facts[0]!.id).toMatch(/^fact_/); + expect(result!.facts[0]).toMatchObject({ + content: "User prefers conclusions first.", + category: "cognitive", + confidence: 0.5, + createdAt: "", + source: "unknown", + }); + }); + + it("rejects imports containing facts without usable content", () => { + const invalid = { + ...legacyMemoryWithoutCognitiveStyle(), + facts: [{ category: "context" }], + }; + + expect(normalizeMemoryPayload(invalid)).toBeNull(); + }); + + it("preserves unknown fields on the strict import path", () => { + const futureExport = { + version: "1.0", + revision: 4, + lastUpdated: "2026-07-01T00:00:00Z", + display: { title: "Memory export" }, + data: { future: true }, + user: { + workContext: { + summary: "Works on DeerFlow", + updatedAt: "2026-06-01T00:00:00Z", + confidence: 0.8, + }, + providerState: { loaded: true }, + }, + history: { + timeline: { entries: ["2026-06"] }, + }, + facts: [ + { + content: "User prefers conclusions first.", + category: "cognitive", + topics: ["communication"], + }, + ], + }; + + const result = normalizeMemoryPayload(futureExport); + + expect(result).not.toBeNull(); + expect(result!.revision).toBe(4); + expect(result).toMatchObject({ + display: { title: "Memory export" }, + data: { future: true }, + }); + expect(result!.user).toMatchObject({ + providerState: { loaded: true }, + workContext: { confidence: 0.8 }, + }); + expect(result!.history).toMatchObject({ + timeline: { entries: ["2026-06"] }, + }); + expect(result!.facts[0]).toMatchObject({ + topics: ["communication"], + }); + expect(result!.user.cognitiveStyle).toEqual({ + summary: "", + updatedAt: "", + }); + }); +}); + +describe("legacy fact metadata parity", () => { + it.each([ + [undefined, 0.5], + [null, 0.5], + [true, 0.5], + ["invalid", 0.5], + [NaN, 0.5], + [Infinity, 0.5], + [0, 0], + [-1, 0], + [2, 1], + ["0.8", 0.8], + ])("normalizes confidence %s to %s", (confidence, expected) => { + const result = normalizeMemoryPayload({ + ...legacyMemoryWithoutCognitiveStyle(), + facts: [ + { + id: "legacy", + content: " Keep conclusions first. ", + confidence, + source: " ", + }, + ], + }); + expect(result!.facts[0]).toMatchObject({ + confidence: expected, + content: "Keep conclusions first.", + source: "unknown", + }); + }); +}); diff --git a/frontend/tests/unit/core/memory/normalize.test.ts b/frontend/tests/unit/core/memory/normalize.test.ts new file mode 100644 index 000000000..f455f35db --- /dev/null +++ b/frontend/tests/unit/core/memory/normalize.test.ts @@ -0,0 +1,41 @@ +import { describe, expect, it } from "@rstest/core"; + +import { normalizeUserMemory } from "@/core/memory/normalize"; + +import { + legacyMemoryWithIncompleteFacts, + legacyMemoryWithoutCognitiveStyle, +} from "./fixtures"; + +describe("normalizeUserMemory (API read path)", () => { + it("fills cognitiveStyle when legacy payload omits it", () => { + const result = normalizeUserMemory(legacyMemoryWithoutCognitiveStyle()); + + expect(result.user.cognitiveStyle).toEqual({ + summary: "", + updatedAt: "", + }); + expect(result.user.workContext.summary).toBe("Works on DeerFlow"); + }); + + it("throws when payload is not a valid memory object", () => { + expect(() => normalizeUserMemory({})).toThrow("Invalid memory payload"); + }); + + it("keeps the API read path available when one fact is unrecoverable", () => { + const legacy = legacyMemoryWithIncompleteFacts(); + const result = normalizeUserMemory({ + ...legacy, + facts: [...legacy.facts, { category: "context" }], + }); + + expect(result.facts).toHaveLength(1); + expect(result.facts[0]).toMatchObject({ + content: "User prefers conclusions first.", + category: "cognitive", + confidence: 0.5, + createdAt: "", + source: "unknown", + }); + }); +});