mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-25 05:56:18 +00:00
feat(memory): add cognitiveStyle for collaboration preferences (#3182)
* feat(memory): add cognitiveStyle for collaboration preferences Extend cross-session memory with user.cognitiveStyle and cognitive facts, backward-compatible normalization, prompt injection as Thinking Style, and settings UI. Add design notes and unit tests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(memory): normalize legacy memory imports on frontend Add normalizeMemoryPayload aligned with backend normalize_memory_data, use it for Settings import and API reads, and add unit/regression tests. Document schema-evolution steps and sync FACT_EXTRACTION_PROMPT cognitive category. * fix(memory): harden legacy memory normalization * fix(memory): keep import envelope validation strict * fix(memory): keep unknown fields through frontend import normalization normalizeMemoryPayload rebuilt the memory document from a section whitelist, so every key it did not name was dropped. Because readMemoryResponse routes all memory API responses through it, the loss showed up on read, export, and import alike: revision vanished, and any future backend field would silently break lossless export/import. The Gateway models are extra="allow" and deermem normalizes additively, which left the frontend as the only lossy hop in the chain. Spread before overriding at each nesting level so normalization only validates and defaults contract-owned fields, leaving the envelope check as strict as before. Covered by a unit test on the strict import path, two backend tests for extension preservation and caller immutability, and an E2E round-trip that asserts on the captured import request body. * docs(memory): state the normalization invariant that actually holds The docstring cited `display` as a backend-driven field alongside `revision`, but #4546 was closed unmerged and no such field exists in the contract. It also promised lossless export/import, which the Gateway's `extra="ignore"` models preclude for any undeclared field. Replace both with the invariant the code is really defending: the frontend must not be narrower than the Gateway contract, using the declared top-level `revision` as the concrete example. * fix(memory): preserve legacy migration and reject lossy imports * fix(memory): align legacy fact normalization defaults * test(memory): use authenticated app for import regression --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
parent
1ffe82eb65
commit
45411cee69
@ -1890,6 +1890,7 @@ Fact CRUD and Settings-page fact editing are not available for this backend. See
|
|||||||
the [Honcho backend guide](backend/packages/harness/deerflow/agents/memory/backends/honcho/README.md).
|
the [Honcho backend guide](backend/packages/harness/deerflow/agents/memory/backends/honcho/README.md).
|
||||||
|
|
||||||
Memory updates now skip duplicate fact entries at apply time, so repeated preferences and context do not accumulate endlessly across sessions.
|
Memory updates now skip duplicate fact entries at apply time, so repeated preferences and context do not accumulate endlessly across sessions.
|
||||||
|
Legacy memory files are normalized when read or imported, including recoverable fact metadata, so older local data remains usable as the schema evolves. Frontend and backend normalization use confidence `0.5` when missing or invalid, default blank sources to `unknown`, and trim fact content.
|
||||||
|
|
||||||
In the default DeerMem `middleware` mode, automatic extraction now classifies every proposed fact by scope, durability, and authority before a deterministic write gate accepts it. Only durable, descriptive user-level facts are stored; current-thread or project constraints and one-time action permissions stay in conversation state. User-global summaries require both user scope and descriptive authority, contradiction removals are scope-gated, and a replacement-dependent removal is applied only when its replacement actually survives validation and storage. These classification labels are extraction-only metadata, add no extra LLM call, and are not written into the fact files. The explicit CRUD tools in `memory.mode: tool` remain a separate, model-directed path. Deployments that override the bundled DeerMem prompts via `memory.backend_config.prompts_dir` must add the new classification fields to their custom templates (the `memory_update` fact/summary/removal formats and the `consolidation` consolidated-fact schema): the write gate fails closed, so an un-migrated template stops every extraction-driven fact, summary, and removal write, surfacing only through the `rejected_by_scope_gate` metrics and the high-rejection-rate warning.
|
In the default DeerMem `middleware` mode, automatic extraction now classifies every proposed fact by scope, durability, and authority before a deterministic write gate accepts it. Only durable, descriptive user-level facts are stored; current-thread or project constraints and one-time action permissions stay in conversation state. User-global summaries require both user scope and descriptive authority, contradiction removals are scope-gated, and a replacement-dependent removal is applied only when its replacement actually survives validation and storage. These classification labels are extraction-only metadata, add no extra LLM call, and are not written into the fact files. The explicit CRUD tools in `memory.mode: tool` remain a separate, model-directed path. Deployments that override the bundled DeerMem prompts via `memory.backend_config.prompts_dir` must add the new classification fields to their custom templates (the `memory_update` fact/summary/removal formats and the `consolidation` consolidated-fact schema): the write gate fails closed, so an un-migrated template stops every extraction-driven fact, summary, and removal write, surfacing only through the `rejected_by_scope_gate` metrics and the high-rejection-rate warning.
|
||||||
|
|
||||||
@ -1915,7 +1916,7 @@ Memory injection follows the configured operation mode. In `middleware` mode, De
|
|||||||
|
|
||||||
An individual Custom Agent can opt out of memory without changing the global setting. Add `memory_enabled: false` to that agent's `users/{user_id}/agents/{name}/config.yaml`. The agent still receives the current-date reminder, but DeerFlow does not inject recalled memory, queue passive or summarization-driven memory updates (including manual `/compact`), expose memory tools, or add memory-tool instructions for that agent. If an existing agent is switched off, its previously injected memory block is removed from checkpoint state before the next model call while its date reminder and conversation remain. Omitting the field (or setting it to `true`) preserves the existing global `memory` behavior.
|
An individual Custom Agent can opt out of memory without changing the global setting. Add `memory_enabled: false` to that agent's `users/{user_id}/agents/{name}/config.yaml`. The agent still receives the current-date reminder, but DeerFlow does not inject recalled memory, queue passive or summarization-driven memory updates (including manual `/compact`), expose memory tools, or add memory-tool instructions for that agent. If an existing agent is switched off, its previously injected memory block is removed from checkpoint state before the next model call while its date reminder and conversation remain. Omitting the field (or setting it to `true`) preserves the existing global `memory` behavior.
|
||||||
|
|
||||||
Single-fact repository operations are genuinely incremental: an upsert/delete reads, journals, writes, and re-indexes only the addressed fact files, and returns an explicit incomplete delta rather than a cache-dependent fake full document. Summary change sets merge the supplied `user`/`history` child keys over the persisted sections so a partial update cannot erase omitted siblings; full imports normalize both sections to the complete compatibility schema before applying replacement values. Manager/API compatibility methods materialize a fresh full document only when their public response contract requires one. Fact-level point operations use separate expected user-memory and fact revisions and may explicitly rebase when every addressed fact precondition still holds. Snapshot-derived operations such as scoped clear, capped create, consolidation, and trimming never replay stale delete/trim sets: a manifest conflict reloads the complete document and recomputes the operation, with a bounded retry. Fact paths use the first two hexadecimal characters of `SHA-256(fact_id)` so generated `fact_*` IDs distribute across shards. The cache token combines the shared JSON's nanosecond mtime, size, and persisted revision; this prevents coarse-mtime same-size writes from returning stale data without scanning fact files. Direct out-of-band Markdown edits require an explicit reload. Storage-specific conflicts and corruption are translated at the MemoryManager boundary; the Gateway returns conflict as HTTP 409 and a stable, non-sensitive corruption error as HTTP 500. Full-document `save()` remains a compatibility API and computes a diff before writing; malformed or missing `facts` can no longer silently erase an agent's Markdown files. Legacy migration preserves non-empty `user`/`history` before deleting an agent `memory.json`; conflicting summaries keep the legacy file and fail loudly instead of choosing a winner.
|
Single-fact repository operations are genuinely incremental: an upsert/delete reads, journals, writes, and re-indexes only the addressed fact files, and returns an explicit incomplete delta rather than a cache-dependent fake full document. Summary change sets merge the supplied `user`/`history` child keys over the persisted sections so a partial update cannot erase omitted siblings; full imports normalize both sections to the complete compatibility schema before applying replacement values. Imports reject malformed fact lists or blank/non-text content with HTTP 400 before changing stored memory; recoverable legacy metadata still receives defaults. An explicit empty fact list remains an intentional clear. Manager/API compatibility methods materialize a fresh full document only when their public response contract requires one. Fact-level point operations use separate expected user-memory and fact revisions and may explicitly rebase when every addressed fact precondition still holds. Snapshot-derived operations such as scoped clear, capped create, consolidation, and trimming never replay stale delete/trim sets: a manifest conflict reloads the complete document and recomputes the operation, with a bounded retry. Fact paths use the first two hexadecimal characters of `SHA-256(fact_id)` so generated `fact_*` IDs distribute across shards. The cache token combines the shared JSON's nanosecond mtime, size, and persisted revision; this prevents coarse-mtime same-size writes from returning stale data without scanning fact files. Direct out-of-band Markdown edits require an explicit reload. Storage-specific conflicts and corruption are translated at the MemoryManager boundary; the Gateway returns conflict as HTTP 409 and a stable, non-sensitive corruption error as HTTP 500. Full-document `save()` remains a compatibility API and computes a diff before writing; malformed or missing `facts` can no longer silently erase an agent's Markdown files. Legacy migration preserves non-empty `user`/`history` before deleting an agent `memory.json`; conflicting summaries keep the legacy file and fail loudly instead of choosing a winner.
|
||||||
|
|
||||||
Legacy facts in `memory.json` migrate automatically into the reserved `__default__` Markdown bucket on the user's first normal memory read. Operators who prefer to audit or complete the migration before serving traffic can run the optional idempotent CLI from `backend/`:
|
Legacy facts in `memory.json` migrate automatically into the reserved `__default__` Markdown bucket on the user's first normal memory read. Operators who prefer to audit or complete the migration before serving traffic can run the optional idempotent CLI from `backend/`:
|
||||||
|
|
||||||
|
|||||||
@ -106,7 +106,7 @@ LLM-powered persistent context retention across conversations:
|
|||||||
- **System prompt injection**: Top facts + context injected into agent prompts
|
- **System prompt injection**: Top facts + context injected into agent prompts
|
||||||
- **Run-level memory identity**: `GET /api/threads/{thread_id}/runs/{run_id}/events?event_types=context:memory` returns the SHA-256 identity of the effective hidden memory block without copying memory text into the event store
|
- **Run-level memory identity**: `GET /api/threads/{thread_id}/runs/{run_id}/events?event_types=context:memory` returns the SHA-256 identity of the effective hidden memory block without copying memory text into the event store
|
||||||
- **Read failures**: Strict backend policies (including legacy `fail_closed`) stop the turn, including at the 5-second async injection deadline. Fail-open reads continue without new context. Timeout handling does not wait for a free worker; a timed-out read may still occupy its worker until the backend returns.
|
- **Read failures**: Strict backend policies (including legacy `fail_closed`) stop the turn, including at the 5-second async injection deadline. Fail-open reads continue without new context. Timeout handling does not wait for a free worker; a timed-out read may still occupy its worker until the backend returns.
|
||||||
- **Storage**: JSON file with mtime-based cache invalidation
|
- **Storage**: JSON file with mtime-based cache invalidation and canonical normalization for legacy sections/fact metadata
|
||||||
|
|
||||||
### Tool Ecosystem
|
### Tool Ecosystem
|
||||||
|
|
||||||
|
|||||||
@ -51,6 +51,10 @@ class UserContext(BaseModel):
|
|||||||
workContext: ContextSection = Field(default_factory=ContextSection)
|
workContext: ContextSection = Field(default_factory=ContextSection)
|
||||||
personalContext: ContextSection = Field(default_factory=ContextSection)
|
personalContext: ContextSection = Field(default_factory=ContextSection)
|
||||||
topOfMind: ContextSection = Field(default_factory=ContextSection)
|
topOfMind: ContextSection = Field(default_factory=ContextSection)
|
||||||
|
cognitiveStyle: ContextSection = Field(
|
||||||
|
default_factory=ContextSection,
|
||||||
|
description="Stable thinking and collaboration habits (reasoning style, depth, feedback patterns)",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class HistoryContext(BaseModel):
|
class HistoryContext(BaseModel):
|
||||||
@ -435,6 +439,8 @@ async def import_memory(body: MemoryResponse, request: Request) -> MemoryRespons
|
|||||||
raise _unsupported_501(manager, "import memory") from None
|
raise _unsupported_501(manager, "import memory") from None
|
||||||
except (MemoryConflictError, MemoryCorruptionError) as exc:
|
except (MemoryConflictError, MemoryCorruptionError) as exc:
|
||||||
raise _map_memory_manager_error(exc) from exc
|
raise _map_memory_manager_error(exc) from exc
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=400, detail="Invalid memory import: facts must be a list of objects with non-empty content.") from exc
|
||||||
except OSError as exc:
|
except OSError as exc:
|
||||||
raise HTTPException(status_code=500, detail="Failed to import memory data.") from exc
|
raise HTTPException(status_code=500, detail="Failed to import memory data.") from exc
|
||||||
|
|
||||||
|
|||||||
140
backend/docs/MEMORY_COGNITIVE_STYLE.md
Normal file
140
backend/docs/MEMORY_COGNITIVE_STYLE.md
Normal file
@ -0,0 +1,140 @@
|
|||||||
|
# Memory: Cognitive Style
|
||||||
|
|
||||||
|
Design note for contributors. Explains `user.cognitiveStyle` and related memory/prompt touchpoints.
|
||||||
|
|
||||||
|
## One-line pitch
|
||||||
|
|
||||||
|
**Skills teach the agent how to do tasks; memory’s `cognitiveStyle` teaches the agent how to think and collaborate with this user.**
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Cross-session memory already stores work context, personal preferences, and facts. In practice, two gaps show up:
|
||||||
|
|
||||||
|
1. **Semantic mixing** — “Prefers TypeScript” and “Always wants conclusions first, then details” both land in `personalContext` or `behavior` facts. The model must infer which is *collaboration protocol* vs *project preference*.
|
||||||
|
2. **Wrong layer for collaboration prefs** — Task skills (`SKILL.md`) are procedural and shared. Stable response/collaboration preferences (structure, depth, correction style) are **user-scoped slow variables**, not one-off task steps.
|
||||||
|
|
||||||
|
Without an explicit slot, collaboration style is under-specified in injection and easy to drop under token pressure.
|
||||||
|
|
||||||
|
## Approach (not a new subsystem)
|
||||||
|
|
||||||
|
Extend the existing memory pipeline:
|
||||||
|
|
||||||
|
| Layer | Role |
|
||||||
|
|-------|------|
|
||||||
|
| `user.cognitiveStyle.summary` | 2–4 sentence paragraph: reasoning & collaboration habits |
|
||||||
|
| `facts[]` with `category: cognitive` | Atomic, confidence-ranked supplements |
|
||||||
|
| `normalize_memory_data()` | Backward-compatible fill for older sections and fact metadata |
|
||||||
|
| `core/prompts/memory_update.chat.yaml` | LLM sets `cognitiveStyle.shouldUpdate` only when new signals are clear |
|
||||||
|
| `format_memory_for_injection()` | Injects as `Thinking Style:` under User Context |
|
||||||
|
|
||||||
|
**Non-goals (this change):**
|
||||||
|
|
||||||
|
- Vector / embedding “personality library”
|
||||||
|
- Separate debounce or sampling schedule for `cognitiveStyle` only
|
||||||
|
- Replacing `personalContext` or skills
|
||||||
|
|
||||||
|
## Update frequency (read vs write)
|
||||||
|
|
||||||
|
| Event | Behavior |
|
||||||
|
|-------|----------|
|
||||||
|
| **Read (every turn)** | If `injection_enabled`, current `cognitiveStyle` is loaded into `<memory>` within `max_injection_tokens` |
|
||||||
|
| **Write (after turn)** | `MemoryMiddleware` queues filtered conversation; **debounce** (`debounce_seconds`, default 30s) batches updates |
|
||||||
|
| **Write (cognitive field)** | Same LLM pass as other user sections; field changes only when JSON has `cognitiveStyle.shouldUpdate: true` |
|
||||||
|
|
||||||
|
So: conversations **trigger** the memory job often; **cognitiveStyle text changes** only when the updater model sees durable new evidence—not every chit-chat turn.
|
||||||
|
|
||||||
|
## How this differs from nearby concepts
|
||||||
|
|
||||||
|
| Concept | Scope | Lifetime |
|
||||||
|
|---------|--------|----------|
|
||||||
|
| **Thread / checkpointer** | This session’s messages & tools | Session |
|
||||||
|
| **Skill** | How to run a task type | Shared / installable |
|
||||||
|
| **workContext / topOfMind** | What the user is doing | Cross-session, changes often |
|
||||||
|
| **personalContext** | Language, interests, tone | Cross-session |
|
||||||
|
| **cognitiveStyle** | How they reason, structure answers, give feedback | Cross-session, **slow** |
|
||||||
|
| **fact (`cognitive`)** | One line habit or meta-preference | Cross-session, ranked by confidence |
|
||||||
|
|
||||||
|
## Adding a new memory field (schema evolution)
|
||||||
|
|
||||||
|
When extending the global summary JSON (`user.*` / `history.*`) or the per-agent Markdown fact schema, keep **read**, **import**, migration, and **API** paths aligned so older exports still work.
|
||||||
|
|
||||||
|
| Step | Location |
|
||||||
|
|------|----------|
|
||||||
|
| 1. Backend normalize | `deerflow/agents/memory/backends/deermem/deermem/core/storage.py` — add keys to `normalize_memory_data()` / fact normalization; update `create_empty_memory()` |
|
||||||
|
| 2. Frontend normalize | `frontend/src/core/memory/import-memory.ts` — add section keys and normalize recoverable legacy fact metadata before narrowing to `UserMemory` |
|
||||||
|
| 3. Types & API models | `frontend/src/core/memory/types.ts`, `backend/app/gateway/routers/memory.py` (`UserContext` / `HistoryContext`) |
|
||||||
|
| 4. Updater prompt | `core/prompts/memory_update.chat.yaml` and `core/prompts/fact_extraction.yaml`; add injection rendering in `core/prompt.py::format_memory_for_injection()`. Fact categories must also be added to `storage.py::CORE_CATEGORIES` |
|
||||||
|
| 5. Settings UI & i18n | `memory-settings-page.tsx`, `en-US.ts` / `zh-CN.ts` |
|
||||||
|
| 6. Tests | Backend: legacy sections and facts in `tests/test_memory_storage.py` / `tests/test_memory_normalize.py` / `tests/test_deermem_self_contained.py`. Frontend: import and API-read behavior in `tests/unit/core/memory/` |
|
||||||
|
| 7. Import path | Keep the stable export envelope strict (`version`, `lastUpdated`, object `user`/`history`, array `facts`), then normalize additive fields inside that valid envelope |
|
||||||
|
|
||||||
|
**Avoid:** normalizing only sections while leaving legacy fact metadata unchecked, or making one unrecoverable fact fail the entire background API read. User-initiated imports remain strict for facts without usable content; API reads drop only those unrecoverable entries.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
PYTHONPATH=. uv run pytest -q tests/test_memory_storage.py tests/test_memory_prompt_injection.py tests/test_memory_normalize.py tests/test_deermem_self_contained.py
|
||||||
|
PYTHONPATH=. uv run pytest tests/test_memory_router.py -v
|
||||||
|
|
||||||
|
cd ../frontend
|
||||||
|
pnpm test tests/unit/core/memory
|
||||||
|
```
|
||||||
|
|
||||||
|
Manual:
|
||||||
|
|
||||||
|
1. Enable `memory` in `config.yaml`, run `make dev`.
|
||||||
|
2. In a thread, state a stable collaboration rule (e.g. “先给结论,不要长铺垫”).
|
||||||
|
3. Wait ≥ `debounce_seconds`, open **Settings → Memory** or `GET /api/memory`.
|
||||||
|
4. Confirm `user.cognitiveStyle.summary` and/or a `cognitive` fact; start a **new thread** and check behavior.
|
||||||
|
|
||||||
|
## Issue
|
||||||
|
|
||||||
|
**Title:** `feat(memory): add cognitiveStyle for stable reasoning & collaboration habits`
|
||||||
|
|
||||||
|
**Summary:**
|
||||||
|
|
||||||
|
- Adds `user.cognitiveStyle` to memory schema with backward-compatible normalization.
|
||||||
|
- Teaches the memory updater to extract thinking/collaboration habits separately from work/personal context.
|
||||||
|
- Injects as `Thinking Style:` in system prompt; supports `cognitive` fact category.
|
||||||
|
- Documents rationale in `backend/docs/MEMORY_COGNITIVE_STYLE.md` and harness memory docs.
|
||||||
|
|
||||||
|
**Motivation:** Cross-session memory should distinguish project context from stable collaboration preferences (response structure, correction style, depth). This change extends the existing memory harness only; it does not add a new store.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 中文说明
|
||||||
|
|
||||||
|
### 背景
|
||||||
|
|
||||||
|
跨会话 memory 已有 `workContext`、`personalContext` 与 `behavior` 类 facts。实践中两类信息容易混在同一字段里:
|
||||||
|
|
||||||
|
- 项目/工具偏好(例如常用 TypeScript)
|
||||||
|
- 协作偏好(例如先给结论、控制篇幅、纠错方式)
|
||||||
|
|
||||||
|
后者更新频率低于 `topOfMind`,又不同于一次性会话信息。单独增加 `user.cognitiveStyle` 便于注入时固定展示为 `Thinking Style:`,并与任务级 Skill 区分。
|
||||||
|
|
||||||
|
### 实现范围
|
||||||
|
|
||||||
|
- 扩展全局 summary JSON 与 per-agent Markdown fact schema,`normalize_memory_data()` 兼容旧 section 和缺少元数据的旧 facts
|
||||||
|
- `core/prompts/memory_update.chat.yaml` 输出 `cognitiveStyle.shouldUpdate`;可选 `category: cognitive` 的 facts
|
||||||
|
- 复用现有 MemoryManager → DeerMem → 防抖队列 → Updater → 注入链路,不新增子系统
|
||||||
|
|
||||||
|
### 更新频率
|
||||||
|
|
||||||
|
| 事件 | 行为 |
|
||||||
|
|------|------|
|
||||||
|
| 每轮对话开始 | 在 `max_injection_tokens` 内注入已有 `cognitiveStyle`(读) |
|
||||||
|
| 每轮对话结束 | 与其它 memory 段相同,可能入队;默认 `debounce_seconds` 合并 |
|
||||||
|
| 写入 `cognitiveStyle` | 仅当 LLM 返回 `shouldUpdate: true` 时更新段落 |
|
||||||
|
|
||||||
|
### 与 Skill 的区别
|
||||||
|
|
||||||
|
| 类型 | 内容 |
|
||||||
|
|------|------|
|
||||||
|
| Skill | 某类任务的步骤与模板(可共享、可安装) |
|
||||||
|
| `cognitiveStyle` | 该用户稳定的回复结构、讨论深度、反馈习惯(按用户持久化) |
|
||||||
|
|
||||||
|
### 以后新增 memory 字段时
|
||||||
|
|
||||||
|
按上文 **Adding a new memory field** 清单同步改后端 `normalize_memory_data()` 与前端 `normalizeMemoryPayload()`;导入走 normalize,不要只对完整新 schema 做严校验。后台 API 读取应丢弃无法恢复的单条 fact,而不是让整个 Memory 页面失败。
|
||||||
@ -129,6 +129,10 @@ Every destructive migration first writes a verified `{manifest_filename}.v1.bak`
|
|||||||
Missing or mismatched backups abort migration without changing v1 data.
|
Missing or mismatched backups abort migration without changing v1 data.
|
||||||
Delete legacy agent JSON only after safe summary adoption or equality checks.
|
Delete legacy agent JSON only after safe summary adoption or equality checks.
|
||||||
Summary conflicts keep the source file and return an error.
|
Summary conflicts keep the source file and return an error.
|
||||||
|
Compare both summary operands after additive normalization; preserve extension
|
||||||
|
fields and keep fact migration validation strict. Replacement imports reject
|
||||||
|
invalid fact containers or unusable content before normalization or storage
|
||||||
|
access; recoverable metadata may default, but malformed facts never mean clear.
|
||||||
|
|
||||||
Run the proactive migration from `backend/`:
|
Run the proactive migration from `backend/`:
|
||||||
|
|
||||||
@ -353,3 +357,5 @@ runs by default.
|
|||||||
Attachment-only messages with an empty preserved request stay query-less.
|
Attachment-only messages with an empty preserved request stay query-less.
|
||||||
- Ranking must be deterministic, network-free, and mutation-free: caller-owned
|
- Ranking must be deterministic, network-free, and mutation-free: caller-owned
|
||||||
fact dicts are read-only inputs.
|
fact dicts are read-only inputs.
|
||||||
|
|
||||||
|
Legacy fact normalization in DeerMem and `frontend/src/core/memory/import-memory.ts` uses neutral confidence `0.5` for missing or invalid values, clamps finite confidence to `[0, 1]`, trims content, and defaults blank or missing sources to `unknown`. Keep these compatibility defaults aligned.
|
||||||
|
|||||||
@ -547,6 +547,10 @@ def format_memory_for_injection(
|
|||||||
if top_of_mind.get("summary"):
|
if top_of_mind.get("summary"):
|
||||||
user_sections.append(f"Current Focus: {_escape_summary(top_of_mind['summary'])}")
|
user_sections.append(f"Current Focus: {_escape_summary(top_of_mind['summary'])}")
|
||||||
|
|
||||||
|
cognitive_style = user_data.get("cognitiveStyle", {})
|
||||||
|
if cognitive_style.get("summary"):
|
||||||
|
user_sections.append(f"Thinking Style: {_escape_summary(cognitive_style['summary'])}")
|
||||||
|
|
||||||
if user_sections:
|
if user_sections:
|
||||||
sections.append("User Context:\n" + "\n".join(f"- {s}" for s in user_sections))
|
sections.append("User Context:\n" + "\n".join(f"- {s}" for s in user_sections))
|
||||||
|
|
||||||
|
|||||||
@ -9,7 +9,7 @@ template: |-
|
|||||||
Extract facts in this JSON format:
|
Extract facts in this JSON format:
|
||||||
{{
|
{{
|
||||||
"facts": [
|
"facts": [
|
||||||
{{ "content": "...", "category": "preference|knowledge|context|behavior|goal|correction", "confidence": 0.0-1.0 }}
|
{{ "content": "...", "category": "preference|knowledge|context|behavior|cognitive|goal|correction", "confidence": 0.0-1.0 }}
|
||||||
]
|
]
|
||||||
}}
|
}}
|
||||||
|
|
||||||
@ -18,6 +18,7 @@ template: |-
|
|||||||
- knowledge: User's expertise or knowledge areas
|
- knowledge: User's expertise or knowledge areas
|
||||||
- context: Background context (location, job, projects)
|
- context: Background context (location, job, projects)
|
||||||
- behavior: Behavioral patterns
|
- behavior: Behavioral patterns
|
||||||
|
- cognitive: Stable reasoning, response-structure, and collaboration habits
|
||||||
- goal: User's goals or objectives
|
- goal: User's goals or objectives
|
||||||
- correction: Explicit corrections or mistakes to avoid repeating
|
- correction: Explicit corrections or mistakes to avoid repeating
|
||||||
|
|
||||||
|
|||||||
@ -39,6 +39,9 @@ messages:
|
|||||||
Example: Primary project work, parallel technical investigations, ongoing learning/tracking
|
Example: Primary project work, parallel technical investigations, ongoing learning/tracking
|
||||||
Include: Active implementation work, troubleshooting issues, market/research interests
|
Include: Active implementation work, troubleshooting issues, market/research interests
|
||||||
Note: This captures SEVERAL concurrent focus areas, not just one task
|
Note: This captures SEVERAL concurrent focus areas, not just one task
|
||||||
|
- cognitiveStyle: Stable reasoning and collaboration habits (2-4 sentences)
|
||||||
|
Include: Preferred answer structure, desired depth, feedback/correction style, decision-making patterns
|
||||||
|
Update conservatively only when durable cross-session evidence is clear
|
||||||
|
|
||||||
**History** (Temporal context - rich paragraphs):
|
**History** (Temporal context - rich paragraphs):
|
||||||
- recentMonths: Detailed summary of recent activities (4-6 sentences or 1-2 paragraphs)
|
- recentMonths: Detailed summary of recent activities (4-6 sentences or 1-2 paragraphs)
|
||||||
@ -63,6 +66,7 @@ messages:
|
|||||||
* knowledge: Specific expertise, technologies mastered, domain knowledge
|
* knowledge: Specific expertise, technologies mastered, domain knowledge
|
||||||
* context: Background facts (job title, projects, locations, languages)
|
* context: Background facts (job title, projects, locations, languages)
|
||||||
* behavior: Working patterns, communication habits, problem-solving approaches
|
* behavior: Working patterns, communication habits, problem-solving approaches
|
||||||
|
* cognitive: Stable reasoning, response-structure, and collaboration habits
|
||||||
* goal: Stated objectives, learning targets, project ambitions
|
* goal: Stated objectives, learning targets, project ambitions
|
||||||
* correction: Explicit agent mistakes or user corrections, including the correct approach
|
* correction: Explicit agent mistakes or user corrections, including the correct approach
|
||||||
- Fact lifetime (``expected_valid_days``, optional integer):
|
- Fact lifetime (``expected_valid_days``, optional integer):
|
||||||
@ -85,6 +89,7 @@ messages:
|
|||||||
- personalContext: Languages, personality, interests outside direct work tasks
|
- personalContext: Languages, personality, interests outside direct work tasks
|
||||||
- topOfMind: Multiple ongoing priorities and focus areas user cares about recently (gets updated most frequently)
|
- topOfMind: Multiple ongoing priorities and focus areas user cares about recently (gets updated most frequently)
|
||||||
Should capture 3-5 concurrent themes: main work, side explorations, learning/tracking interests
|
Should capture 3-5 concurrent themes: main work, side explorations, learning/tracking interests
|
||||||
|
- cognitiveStyle: Stable preferences for how the assistant should reason, structure answers, and collaborate
|
||||||
- recentMonths: Detailed account of recent technical explorations and work
|
- recentMonths: Detailed account of recent technical explorations and work
|
||||||
- earlierContext: Patterns from slightly older interactions still relevant
|
- earlierContext: Patterns from slightly older interactions still relevant
|
||||||
- longTermBackground: Unchanging foundational facts about the user
|
- longTermBackground: Unchanging foundational facts about the user
|
||||||
@ -99,7 +104,8 @@ messages:
|
|||||||
"user": {{
|
"user": {{
|
||||||
"workContext": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }},
|
"workContext": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }},
|
||||||
"personalContext": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }},
|
"personalContext": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }},
|
||||||
"topOfMind": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }}
|
"topOfMind": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }},
|
||||||
|
"cognitiveStyle": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }}
|
||||||
}},
|
}},
|
||||||
"history": {{
|
"history": {{
|
||||||
"recentMonths": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }},
|
"recentMonths": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }},
|
||||||
@ -107,7 +113,7 @@ messages:
|
|||||||
"longTermBackground": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }}
|
"longTermBackground": {{ "summary": "...", "shouldUpdate": true/false, "scope": "user|thread|project", "authority": "descriptive|transactional" }}
|
||||||
}},
|
}},
|
||||||
"newFacts": [
|
"newFacts": [
|
||||||
{{ "content": "...", "category": "preference|knowledge|context|behavior|goal|correction", "confidence": 0.0-1.0, "expected_valid_days": 90, "scope": "user|thread|project", "durability": "durable|temporary", "authority": "descriptive|transactional" }}
|
{{ "content": "...", "category": "preference|knowledge|context|behavior|cognitive|goal|correction", "confidence": 0.0-1.0, "expected_valid_days": 90, "scope": "user|thread|project", "durability": "durable|temporary", "authority": "descriptive|transactional" }}
|
||||||
],
|
],
|
||||||
"factsToReinforce": [
|
"factsToReinforce": [
|
||||||
{{ "id": "fact_id_1", "scope": "user|thread|project", "reason": "explicit user confirmation of this existing fact" }}
|
{{ "id": "fact_id_1", "scope": "user|thread|project", "reason": "explicit user confirmation of this existing fact" }}
|
||||||
|
|||||||
@ -48,7 +48,7 @@ from .paths import (
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
DOCUMENT_VERSION = "2.0"
|
DOCUMENT_VERSION = "2.0"
|
||||||
CORE_CATEGORIES = frozenset({"preference", "correction", "context", "goal", "behavior", "identity", "constraint", "decision", "other"})
|
CORE_CATEGORIES = frozenset({"preference", "correction", "context", "goal", "behavior", "cognitive", "identity", "constraint", "decision", "other"})
|
||||||
|
|
||||||
|
|
||||||
class MemoryStorageError(RuntimeError):
|
class MemoryStorageError(RuntimeError):
|
||||||
@ -103,6 +103,7 @@ def create_empty_memory() -> dict[str, Any]:
|
|||||||
"workContext": {"summary": "", "updatedAt": ""},
|
"workContext": {"summary": "", "updatedAt": ""},
|
||||||
"personalContext": {"summary": "", "updatedAt": ""},
|
"personalContext": {"summary": "", "updatedAt": ""},
|
||||||
"topOfMind": {"summary": "", "updatedAt": ""},
|
"topOfMind": {"summary": "", "updatedAt": ""},
|
||||||
|
"cognitiveStyle": {"summary": "", "updatedAt": ""},
|
||||||
},
|
},
|
||||||
"history": {
|
"history": {
|
||||||
"recentMonths": {"summary": "", "updatedAt": ""},
|
"recentMonths": {"summary": "", "updatedAt": ""},
|
||||||
@ -113,6 +114,84 @@ def create_empty_memory() -> dict[str, Any]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_context_section(value: Any) -> dict[str, Any]:
|
||||||
|
"""Return a canonical summary section while preserving extension fields."""
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
return {"summary": "", "updatedAt": ""}
|
||||||
|
section = copy.deepcopy(value)
|
||||||
|
section["summary"] = value.get("summary") if isinstance(value.get("summary"), str) else ""
|
||||||
|
section["updatedAt"] = value.get("updatedAt") if isinstance(value.get("updatedAt"), str) else ""
|
||||||
|
return section
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_legacy_import_fact(value: Any) -> dict[str, Any] | None:
|
||||||
|
"""Canonicalize recoverable public/legacy fact fields before repository writes."""
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
return None
|
||||||
|
content = value.get("content")
|
||||||
|
if not isinstance(content, str) or not content.strip():
|
||||||
|
return None
|
||||||
|
|
||||||
|
fact = copy.deepcopy(value)
|
||||||
|
fact["content"] = content.strip()
|
||||||
|
fact_id = fact.get("id")
|
||||||
|
fact["id"] = fact_id.strip() if isinstance(fact_id, str) and fact_id.strip() else f"fact_{uuid.uuid4().hex[:8]}"
|
||||||
|
category = fact.get("category")
|
||||||
|
fact["category"] = category.strip() if isinstance(category, str) and category.strip() else "context"
|
||||||
|
|
||||||
|
confidence = fact.get("confidence", 0.5)
|
||||||
|
if isinstance(confidence, bool):
|
||||||
|
numeric_confidence = 0.5
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
numeric_confidence = float(confidence)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
numeric_confidence = 0.5
|
||||||
|
fact["confidence"] = min(1.0, max(0.0, numeric_confidence)) if math.isfinite(numeric_confidence) else 0.5
|
||||||
|
|
||||||
|
created_at = fact.get("createdAt")
|
||||||
|
fact["createdAt"] = created_at.strip() if isinstance(created_at, str) else ""
|
||||||
|
source = fact.get("source")
|
||||||
|
fact["source"] = source.strip() if isinstance(source, str) and source.strip() else "unknown"
|
||||||
|
if "sourceError" in fact and fact["sourceError"] is not None and not isinstance(fact["sourceError"], str):
|
||||||
|
fact.pop("sourceError")
|
||||||
|
return fact
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_memory_summaries(data: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""Normalize additive summary fields without relaxing fact validation."""
|
||||||
|
empty = create_empty_memory()
|
||||||
|
summaries: dict[str, Any] = {}
|
||||||
|
for section_name, section_keys in (
|
||||||
|
("user", ("workContext", "personalContext", "topOfMind", "cognitiveStyle")),
|
||||||
|
("history", ("recentMonths", "earlierContext", "longTermBackground")),
|
||||||
|
):
|
||||||
|
incoming = data.get(section_name)
|
||||||
|
incoming = incoming if isinstance(incoming, dict) else {}
|
||||||
|
complete = copy.deepcopy(empty[section_name])
|
||||||
|
for key, value in incoming.items():
|
||||||
|
complete[key] = _normalize_context_section(value) if key in section_keys else copy.deepcopy(value)
|
||||||
|
for key in section_keys:
|
||||||
|
complete[key] = _normalize_context_section(complete.get(key))
|
||||||
|
summaries[section_name] = complete
|
||||||
|
|
||||||
|
return summaries
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_memory_data(data: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""Return a canonical compatibility document without mutating the caller."""
|
||||||
|
normalized = copy.deepcopy(data) if isinstance(data, dict) else {}
|
||||||
|
normalized.update(_normalize_memory_summaries(normalized))
|
||||||
|
|
||||||
|
facts = normalized.get("facts")
|
||||||
|
normalized["facts"] = [fact for value in facts if (fact := _normalize_legacy_import_fact(value)) is not None] if isinstance(facts, list) else []
|
||||||
|
if not isinstance(normalized.get("version"), str):
|
||||||
|
normalized["version"] = "1.0"
|
||||||
|
if not isinstance(normalized.get("lastUpdated"), str):
|
||||||
|
normalized["lastUpdated"] = ""
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
def _has_meaningful_data(value: Any) -> bool:
|
def _has_meaningful_data(value: Any) -> bool:
|
||||||
"""Return whether a legacy summary value contains anything worth preserving."""
|
"""Return whether a legacy summary value contains anything worth preserving."""
|
||||||
if isinstance(value, dict):
|
if isinstance(value, dict):
|
||||||
@ -1157,16 +1236,13 @@ class FileMemoryStorage(MemoryStorage):
|
|||||||
if not sources:
|
if not sources:
|
||||||
return False, from_version, []
|
return False, from_version, []
|
||||||
|
|
||||||
base = global_memory or create_empty_memory()
|
migrated_summaries = _normalize_memory_summaries(global_memory or {})
|
||||||
migrated_summaries = {
|
|
||||||
"user": copy.deepcopy(base.get("user", {})),
|
|
||||||
"history": copy.deepcopy(base.get("history", {})),
|
|
||||||
}
|
|
||||||
if legacy_memory is not None and adopt_legacy_summaries:
|
if legacy_memory is not None and adopt_legacy_summaries:
|
||||||
|
legacy_summaries = _normalize_memory_summaries(legacy_memory)
|
||||||
for section in ("user", "history"):
|
for section in ("user", "history"):
|
||||||
migrated_summaries[section] = _merge_legacy_summary_section(
|
migrated_summaries[section] = _merge_legacy_summary_section(
|
||||||
canonical=migrated_summaries[section],
|
canonical=migrated_summaries[section],
|
||||||
legacy=legacy_memory.get(section, {}),
|
legacy=legacy_summaries[section],
|
||||||
section=section,
|
section=section,
|
||||||
legacy_path=legacy_path,
|
legacy_path=legacy_path,
|
||||||
)
|
)
|
||||||
@ -1253,6 +1329,17 @@ class FileMemoryStorage(MemoryStorage):
|
|||||||
raise MemoryStorageCorruption(f"Legacy facts in {path} must be a list or mapping")
|
raise MemoryStorageCorruption(f"Legacy facts in {path} must be a list or mapping")
|
||||||
result = {key: copy.deepcopy(value) for key, value in memory_file.items() if key != "facts"}
|
result = {key: copy.deepcopy(value) for key, value in memory_file.items() if key != "facts"}
|
||||||
result.setdefault("revision", 0)
|
result.setdefault("revision", 0)
|
||||||
|
summary_view = normalize_memory_data(
|
||||||
|
{
|
||||||
|
"version": result.get("version"),
|
||||||
|
"lastUpdated": result.get("lastUpdated"),
|
||||||
|
"user": result.get("user"),
|
||||||
|
"history": result.get("history"),
|
||||||
|
"facts": [],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
result["user"] = summary_view["user"]
|
||||||
|
result["history"] = summary_view["history"]
|
||||||
result["facts"] = facts
|
result["facts"] = facts
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
|||||||
@ -32,6 +32,7 @@ from .storage import (
|
|||||||
MemoryManifestRevisionConflict,
|
MemoryManifestRevisionConflict,
|
||||||
MemoryStorage,
|
MemoryStorage,
|
||||||
create_empty_memory,
|
create_empty_memory,
|
||||||
|
normalize_memory_data,
|
||||||
utc_now_iso_z,
|
utc_now_iso_z,
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -963,19 +964,12 @@ class MemoryUpdater:
|
|||||||
"""Persist imported memory data via the injected storage."""
|
"""Persist imported memory data via the injected storage."""
|
||||||
if not isinstance(memory_data, dict):
|
if not isinstance(memory_data, dict):
|
||||||
raise ValueError("memory_data")
|
raise ValueError("memory_data")
|
||||||
memory_data = copy.deepcopy(memory_data)
|
# Replacement imports must not turn malformed facts into deletions.
|
||||||
empty = create_empty_memory()
|
# Validate before lenient compatibility normalization or any storage read.
|
||||||
for section in ("user", "history"):
|
raw_facts = memory_data.get("facts")
|
||||||
incoming_section = memory_data.get(section, {})
|
if not isinstance(raw_facts, list) or any(not isinstance(fact, dict) or not isinstance(fact.get("content"), str) or not fact["content"].strip() for fact in raw_facts):
|
||||||
if not isinstance(incoming_section, dict):
|
raise ValueError("memory_data.facts must be a list of facts with non-empty content")
|
||||||
raise ValueError(f"memory_data.{section}")
|
memory_data = normalize_memory_data(memory_data)
|
||||||
complete_section = copy.deepcopy(empty[section])
|
|
||||||
for key, value in incoming_section.items():
|
|
||||||
if key in complete_section and isinstance(complete_section[key], dict) and isinstance(value, dict):
|
|
||||||
complete_section[key].update(copy.deepcopy(value))
|
|
||||||
else:
|
|
||||||
complete_section[key] = copy.deepcopy(value)
|
|
||||||
memory_data[section] = complete_section
|
|
||||||
if agent_name is not None and getattr(type(self._storage), "apply_changes", None) is not MemoryStorage.apply_changes:
|
if agent_name is not None and getattr(type(self._storage), "apply_changes", None) is not MemoryStorage.apply_changes:
|
||||||
current = self.get_memory_data(agent_name, user_id=user_id)
|
current = self.get_memory_data(agent_name, user_id=user_id)
|
||||||
incoming_facts = copy.deepcopy(memory_data.get("facts", []))
|
incoming_facts = copy.deepcopy(memory_data.get("facts", []))
|
||||||
@ -1973,7 +1967,7 @@ class MemoryUpdater:
|
|||||||
|
|
||||||
# Update user sections
|
# Update user sections
|
||||||
user_updates = update_data.get("user", {})
|
user_updates = update_data.get("user", {})
|
||||||
for section in ["workContext", "personalContext", "topOfMind"]:
|
for section in ["workContext", "personalContext", "topOfMind", "cognitiveStyle"]:
|
||||||
section_data = user_updates.get(section, {})
|
section_data = user_updates.get(section, {})
|
||||||
if not isinstance(section_data, dict) or not section_data.get("shouldUpdate") or not section_data.get("summary"):
|
if not isinstance(section_data, dict) or not section_data.get("shouldUpdate") or not section_data.get("summary"):
|
||||||
continue
|
continue
|
||||||
|
|||||||
@ -210,6 +210,7 @@ def test_import_empty_summary_sections_replace_existing_summaries_with_complete_
|
|||||||
"workContext": {"summary": "", "updatedAt": ""},
|
"workContext": {"summary": "", "updatedAt": ""},
|
||||||
"personalContext": {"summary": "", "updatedAt": ""},
|
"personalContext": {"summary": "", "updatedAt": ""},
|
||||||
"topOfMind": {"summary": "", "updatedAt": ""},
|
"topOfMind": {"summary": "", "updatedAt": ""},
|
||||||
|
"cognitiveStyle": {"summary": "", "updatedAt": ""},
|
||||||
}
|
}
|
||||||
assert imported["history"] == {
|
assert imported["history"] == {
|
||||||
"recentMonths": {"summary": "", "updatedAt": ""},
|
"recentMonths": {"summary": "", "updatedAt": ""},
|
||||||
@ -930,3 +931,76 @@ def test_from_backend_config_null_values_do_not_warn_as_unknown(caplog):
|
|||||||
with caplog.at_level("WARNING", logger=cfg_logger):
|
with caplog.at_level("WARNING", logger=cfg_logger):
|
||||||
DeerMemConfig.from_backend_config({"model": None})
|
DeerMemConfig.from_backend_config({"model": None})
|
||||||
assert not any("Unknown backend_config keys" in r.message for r in caplog.records)
|
assert not any("Unknown backend_config keys" in r.message for r in caplog.records)
|
||||||
|
|
||||||
|
|
||||||
|
def test_apply_updates_cognitive_style_and_fact_category(deermem_data_dir) -> None:
|
||||||
|
dm = DeerMem(backend_config=None)
|
||||||
|
current_memory = dm.get_memory(user_id="cognitive-user")
|
||||||
|
|
||||||
|
result = dm._updater._apply_updates(
|
||||||
|
current_memory,
|
||||||
|
{
|
||||||
|
"user": {
|
||||||
|
"cognitiveStyle": {
|
||||||
|
"summary": "Prefers conclusions first, then details.",
|
||||||
|
"shouldUpdate": True,
|
||||||
|
"scope": "user",
|
||||||
|
"authority": "descriptive",
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"newFacts": [
|
||||||
|
{
|
||||||
|
"content": "User prefers conclusions before implementation details.",
|
||||||
|
"category": "cognitive",
|
||||||
|
"confidence": 0.92,
|
||||||
|
"scope": "user",
|
||||||
|
"durability": "durable",
|
||||||
|
"authority": "descriptive",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
thread_id="thread-cognitive",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["user"]["cognitiveStyle"]["summary"] == "Prefers conclusions first, then details."
|
||||||
|
assert result["user"]["cognitiveStyle"]["updatedAt"]
|
||||||
|
assert result["facts"][0]["category"] == "cognitive"
|
||||||
|
assert result["facts"][0]["source"] == "thread-cognitive"
|
||||||
|
|
||||||
|
|
||||||
|
def test_import_memory_persists_normalized_legacy_payload(deermem_data_dir) -> None:
|
||||||
|
dm = DeerMem(backend_config=None)
|
||||||
|
legacy = {
|
||||||
|
"version": "1.0",
|
||||||
|
"lastUpdated": "",
|
||||||
|
"user": {},
|
||||||
|
"history": {},
|
||||||
|
"facts": [{"content": "User prefers conclusions first.", "category": "cognitive"}],
|
||||||
|
}
|
||||||
|
|
||||||
|
result = dm.import_memory(legacy, user_id="legacy-user")
|
||||||
|
|
||||||
|
assert result["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""}
|
||||||
|
assert result["facts"][0]["id"].startswith("fact_")
|
||||||
|
assert result["facts"][0]["category"] == "cognitive"
|
||||||
|
assert result["facts"][0]["confidence"] == 0.5
|
||||||
|
assert result["facts"][0]["createdAt"] == ""
|
||||||
|
assert result["facts"][0]["source"] == "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("facts", [[{"id": "keep", "content": " "}], [None], {}, None, "missing", [{"id": "keep", "content": 42}], [{"id": "new", "content": "valid"}, {}]])
|
||||||
|
def test_replacement_import_rejects_unrecoverable_facts_without_writes(deermem_data_dir, facts):
|
||||||
|
dm = DeerMem(backend_config=None)
|
||||||
|
before = dm.import_memory({"user": {}, "history": {}, "facts": [{"id": "keep", "content": "Saved preference"}]}, user_id="alice")
|
||||||
|
snapshot = {str(path.relative_to(deermem_data_dir)): path.read_bytes() for path in deermem_data_dir.rglob("*") if path.is_file()}
|
||||||
|
|
||||||
|
payload = {"user": {}, "history": {}, "facts": facts}
|
||||||
|
if facts == "missing":
|
||||||
|
payload.pop("facts")
|
||||||
|
with pytest.raises(ValueError, match="facts"):
|
||||||
|
dm.import_memory(payload, user_id="alice")
|
||||||
|
|
||||||
|
assert {str(path.relative_to(deermem_data_dir)): path.read_bytes() for path in deermem_data_dir.rglob("*") if path.is_file()} == snapshot
|
||||||
|
reloaded = DeerMem(backend_config=None).get_memory(user_id="alice")
|
||||||
|
assert reloaded["facts"] == before["facts"]
|
||||||
|
assert reloaded["revision"] == before["revision"]
|
||||||
|
|||||||
106
backend/tests/test_memory_normalize.py
Normal file
106
backend/tests/test_memory_normalize.py
Normal file
@ -0,0 +1,106 @@
|
|||||||
|
"""Tests for memory schema normalization."""
|
||||||
|
|
||||||
|
import copy
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from deerflow.agents.memory.backends.deermem.deermem.core.storage import create_empty_memory, normalize_memory_data
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_memory_data_adds_cognitive_style() -> None:
|
||||||
|
legacy = {
|
||||||
|
"version": "1.0",
|
||||||
|
"lastUpdated": "",
|
||||||
|
"user": {
|
||||||
|
"workContext": {"summary": "work", "updatedAt": "2026-01-01T00:00:00Z"},
|
||||||
|
"personalContext": {"summary": "", "updatedAt": ""},
|
||||||
|
"topOfMind": {"summary": "", "updatedAt": ""},
|
||||||
|
},
|
||||||
|
"history": {
|
||||||
|
"recentMonths": {"summary": "", "updatedAt": ""},
|
||||||
|
"earlierContext": {"summary": "", "updatedAt": ""},
|
||||||
|
"longTermBackground": {"summary": "", "updatedAt": ""},
|
||||||
|
},
|
||||||
|
"facts": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
result = normalize_memory_data(legacy)
|
||||||
|
|
||||||
|
assert "cognitiveStyle" in result["user"]
|
||||||
|
assert result["user"]["cognitiveStyle"]["summary"] == ""
|
||||||
|
assert result["user"]["cognitiveStyle"]["updatedAt"] == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_create_empty_memory_includes_cognitive_style() -> None:
|
||||||
|
empty = create_empty_memory()
|
||||||
|
assert empty["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""}
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_memory_data_preserves_unknown_fields() -> None:
|
||||||
|
payload = {
|
||||||
|
"version": "1.0",
|
||||||
|
"revision": 7,
|
||||||
|
"lastUpdated": "2026-01-01T00:00:00Z",
|
||||||
|
"display": {"title": "Memory export"},
|
||||||
|
"data": {"future": True},
|
||||||
|
"user": {
|
||||||
|
"workContext": {
|
||||||
|
"summary": "work",
|
||||||
|
"updatedAt": "2026-01-01T00:00:00Z",
|
||||||
|
"confidence": 0.8,
|
||||||
|
},
|
||||||
|
"providerState": {"loaded": True},
|
||||||
|
},
|
||||||
|
"history": {
|
||||||
|
"timeline": {"entries": ["2026-01"]},
|
||||||
|
},
|
||||||
|
"facts": [
|
||||||
|
{
|
||||||
|
"content": "User prefers conclusions first.",
|
||||||
|
"category": "cognitive",
|
||||||
|
"topics": ["communication"],
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
result = normalize_memory_data(payload)
|
||||||
|
|
||||||
|
assert result["revision"] == 7
|
||||||
|
assert result["display"] == {"title": "Memory export"}
|
||||||
|
assert result["data"] == {"future": True}
|
||||||
|
assert result["user"]["providerState"] == {"loaded": True}
|
||||||
|
assert result["user"]["workContext"]["confidence"] == 0.8
|
||||||
|
assert result["user"]["workContext"]["summary"] == "work"
|
||||||
|
assert result["history"]["timeline"] == {"entries": ["2026-01"]}
|
||||||
|
assert result["facts"][0]["topics"] == ["communication"]
|
||||||
|
assert result["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""}
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_memory_data_does_not_mutate_caller() -> None:
|
||||||
|
payload = {
|
||||||
|
"version": "1.0",
|
||||||
|
"lastUpdated": "",
|
||||||
|
"user": {"workContext": {"summary": "work"}},
|
||||||
|
"history": {},
|
||||||
|
"facts": [{"content": "kept"}],
|
||||||
|
}
|
||||||
|
snapshot = copy.deepcopy(payload)
|
||||||
|
|
||||||
|
result = normalize_memory_data(payload)
|
||||||
|
|
||||||
|
assert result is not payload
|
||||||
|
assert payload == snapshot
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("confidence,expected", [(None, 0.5), (True, 0.5), ("invalid", 0.5), (float("nan"), 0.5), (float("inf"), 0.5), (0, 0), (-1, 0), (2, 1), ("0.8", 0.8)])
|
||||||
|
def test_normalize_legacy_fact_metadata(confidence, expected):
|
||||||
|
fact = {"id": "legacy", "content": " Keep conclusions first. ", "confidence": confidence, "source": " "}
|
||||||
|
result = normalize_memory_data({"facts": [fact]})["facts"][0]
|
||||||
|
assert result["confidence"] == expected
|
||||||
|
assert result["content"] == "Keep conclusions first."
|
||||||
|
assert result["source"] == "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_missing_fact_confidence_uses_neutral_default():
|
||||||
|
result = normalize_memory_data({"facts": [{"content": "Legacy preference"}]})
|
||||||
|
assert result["facts"][0]["confidence"] == 0.5
|
||||||
@ -4,7 +4,12 @@ import math
|
|||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from deerflow.agents.memory.backends.deermem.deermem.core.prompt import _coerce_confidence, format_memory_for_injection
|
from deerflow.agents.memory.backends.deermem.deermem.core.prompt import (
|
||||||
|
FACT_EXTRACTION_PROMPT,
|
||||||
|
_coerce_confidence,
|
||||||
|
format_memory_for_injection,
|
||||||
|
load_prompt_messages,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_format_memory_includes_facts_section() -> None:
|
def test_format_memory_includes_facts_section() -> None:
|
||||||
@ -820,3 +825,56 @@ def test_format_memory_tolerates_non_string_summary() -> None:
|
|||||||
result = format_memory_for_injection(memory_data, max_tokens=2000)
|
result = format_memory_for_injection(memory_data, max_tokens=2000)
|
||||||
|
|
||||||
assert "Current Focus: 12345" in result
|
assert "Current Focus: 12345" in result
|
||||||
|
|
||||||
|
|
||||||
|
def test_format_memory_includes_cognitive_style() -> None:
|
||||||
|
memory_data = {
|
||||||
|
"user": {
|
||||||
|
"cognitiveStyle": {
|
||||||
|
"summary": "Prefers conclusions first, then details.",
|
||||||
|
"updatedAt": "2026-01-01T00:00:00Z",
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"history": {},
|
||||||
|
"facts": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
result = format_memory_for_injection(memory_data, max_tokens=2000)
|
||||||
|
|
||||||
|
assert "Thinking Style:" in result
|
||||||
|
assert "Prefers conclusions first, then details." in result
|
||||||
|
|
||||||
|
|
||||||
|
def test_cognitive_fact_category_is_documented_and_rendered() -> None:
|
||||||
|
messages = load_prompt_messages(
|
||||||
|
"memory_update",
|
||||||
|
{
|
||||||
|
"current_memory": "{}",
|
||||||
|
"conversation": "",
|
||||||
|
"correction_hint": "",
|
||||||
|
"staleness_review_section": "",
|
||||||
|
"consolidation_section": "",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
memory_update_prompt = messages[0].content
|
||||||
|
assert isinstance(memory_update_prompt, str)
|
||||||
|
assert "cognitive|goal|correction" in memory_update_prompt
|
||||||
|
assert "cognitive|goal|correction" in FACT_EXTRACTION_PROMPT
|
||||||
|
assert "- cognitive:" in FACT_EXTRACTION_PROMPT
|
||||||
|
|
||||||
|
result = format_memory_for_injection(
|
||||||
|
{
|
||||||
|
"user": {},
|
||||||
|
"history": {},
|
||||||
|
"facts": [
|
||||||
|
{
|
||||||
|
"content": "User prefers conclusions first.",
|
||||||
|
"category": "cognitive",
|
||||||
|
"confidence": 0.9,
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
max_tokens=2000,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "[cognitive | 0.90] User prefers conclusions first." in result
|
||||||
|
|||||||
@ -587,3 +587,19 @@ def test_reload_memory_route_returns_501_when_read_also_unsupported() -> None:
|
|||||||
with TestClient(app) as client:
|
with TestClient(app) as client:
|
||||||
response = client.post("/api/memory/reload")
|
response = client.post("/api/memory/reload")
|
||||||
assert response.status_code == 501
|
assert response.status_code == 501
|
||||||
|
|
||||||
|
|
||||||
|
def test_import_blank_fact_returns_400_without_replacing_saved_memory(tmp_path):
|
||||||
|
manager = DeerMem(backend_config={"storage_path": str(tmp_path)})
|
||||||
|
before = manager.import_memory(_sample_memory(facts=[{"id": "keep", "content": "Saved preference"}]), user_id="alice")
|
||||||
|
payload = _sample_memory(facts=[{**before["facts"][0], "content": " "}])
|
||||||
|
app = make_authed_test_app()
|
||||||
|
app.include_router(memory.router)
|
||||||
|
with (
|
||||||
|
patch("app.gateway.routers.memory.get_memory_manager", return_value=manager),
|
||||||
|
patch("app.gateway.routers.memory.get_effective_user_id", return_value="alice"),
|
||||||
|
TestClient(app) as client,
|
||||||
|
):
|
||||||
|
response = client.post("/api/memory/import", json=payload)
|
||||||
|
assert response.status_code == 400
|
||||||
|
assert manager.get_memory(user_id="alice") == before
|
||||||
|
|||||||
@ -15,13 +15,14 @@ from deerflow.agents.memory.backends.deermem.deermem.core.storage import (
|
|||||||
MemoryStorage,
|
MemoryStorage,
|
||||||
create_empty_memory,
|
create_empty_memory,
|
||||||
create_storage,
|
create_storage,
|
||||||
|
normalize_memory_data,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _storage_at(memory_file) -> FileMemoryStorage:
|
def _storage_at(memory_file) -> FileMemoryStorage:
|
||||||
"""A FileMemoryStorage whose absolute storage_path is a single shared file."""
|
"""A FileMemoryStorage rooted at the directory containing ``memory_file``."""
|
||||||
resolved = str(memory_file.resolve())
|
root = str(memory_file.parent.resolve())
|
||||||
return FileMemoryStorage(DeerMemConfig(storage_path=resolved))
|
return FileMemoryStorage(DeerMemConfig(storage_path=root))
|
||||||
|
|
||||||
|
|
||||||
class TestCreateEmptyMemory:
|
class TestCreateEmptyMemory:
|
||||||
@ -37,6 +38,45 @@ class TestCreateEmptyMemory:
|
|||||||
assert isinstance(memory["facts"], list)
|
assert isinstance(memory["facts"], list)
|
||||||
|
|
||||||
|
|
||||||
|
class TestNormalizeMemoryData:
|
||||||
|
"""Test backward-compatible memory schema normalization."""
|
||||||
|
|
||||||
|
def test_normalizes_legacy_facts_without_mutating_input(self):
|
||||||
|
legacy = {
|
||||||
|
"version": "1.0",
|
||||||
|
"lastUpdated": "",
|
||||||
|
"user": {},
|
||||||
|
"history": {},
|
||||||
|
"facts": [
|
||||||
|
{"content": "User prefers conclusions first", "category": "cognitive"},
|
||||||
|
None,
|
||||||
|
{"category": "context"},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
normalized = normalize_memory_data(legacy)
|
||||||
|
|
||||||
|
assert legacy == {
|
||||||
|
"version": "1.0",
|
||||||
|
"lastUpdated": "",
|
||||||
|
"user": {},
|
||||||
|
"history": {},
|
||||||
|
"facts": [
|
||||||
|
{"content": "User prefers conclusions first", "category": "cognitive"},
|
||||||
|
None,
|
||||||
|
{"category": "context"},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
assert len(normalized["facts"]) == 1
|
||||||
|
fact = normalized["facts"][0]
|
||||||
|
assert fact["id"].startswith("fact_")
|
||||||
|
assert fact["content"] == "User prefers conclusions first"
|
||||||
|
assert fact["category"] == "cognitive"
|
||||||
|
assert fact["confidence"] == 0.5
|
||||||
|
assert fact["createdAt"] == ""
|
||||||
|
assert fact["source"] == "unknown"
|
||||||
|
|
||||||
|
|
||||||
class TestMemoryStorageInterface:
|
class TestMemoryStorageInterface:
|
||||||
"""Test MemoryStorage abstract base class."""
|
"""Test MemoryStorage abstract base class."""
|
||||||
|
|
||||||
@ -183,6 +223,36 @@ class TestCreateStorage:
|
|||||||
assert isinstance(storage, FileMemoryStorage)
|
assert isinstance(storage, FileMemoryStorage)
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_normalizes_legacy_json_without_cognitive_style(tmp_path) -> None:
|
||||||
|
memory_file = tmp_path / "memory.json"
|
||||||
|
memory_file.write_text(
|
||||||
|
'{"version":"1.0","lastUpdated":"","user":{"workContext":{"summary":"work","updatedAt":""}},"history":{},"facts":[]}',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
storage = _storage_at(memory_file)
|
||||||
|
|
||||||
|
loaded = storage.load()
|
||||||
|
|
||||||
|
assert loaded["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""}
|
||||||
|
assert loaded["user"]["workContext"]["summary"] == "work"
|
||||||
|
|
||||||
|
|
||||||
|
def test_cache_hit_returns_an_equivalent_normalized_copy(tmp_path) -> None:
|
||||||
|
memory_file = tmp_path / "memory.json"
|
||||||
|
memory_file.write_text(
|
||||||
|
'{"version":"1.0","lastUpdated":"","user":{},"history":{},"facts":[{"content":"Legacy cached fact"}]}',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
storage = _storage_at(memory_file)
|
||||||
|
first = storage.load()
|
||||||
|
second = storage.load()
|
||||||
|
|
||||||
|
assert second == first
|
||||||
|
assert second is not first
|
||||||
|
assert second["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""}
|
||||||
|
|
||||||
|
|
||||||
class TestMarkdownMemoryStorage:
|
class TestMarkdownMemoryStorage:
|
||||||
"""Opt-in ``storage_class="markdown"``: tolerant load path (issue #3124)."""
|
"""Opt-in ``storage_class="markdown"``: tolerant load path (issue #3124)."""
|
||||||
|
|
||||||
|
|||||||
@ -1187,3 +1187,44 @@ def test_windows_lock_file_does_not_grow_per_acquisition(storage: FileMemoryStor
|
|||||||
assert storage.save(create_empty_memory(), user_id="alice")
|
assert storage.save(create_empty_memory(), user_id="alice")
|
||||||
lock_path = storage._get_memory_file_path(user_id="alice").parent / ".memory.lock"
|
lock_path = storage._get_memory_file_path(user_id="alice").parent / ".memory.lock"
|
||||||
assert lock_path.stat().st_size == 1
|
assert lock_path.stat().st_size == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("global_version", ["1.0", "2.0"])
|
||||||
|
def test_load_migrates_identical_pre_cognitive_summaries(storage, global_version):
|
||||||
|
memory_path = storage._get_memory_file_path("__default__", user_id="alice")
|
||||||
|
legacy_path = memory_path.parent / "agents" / "__default__" / "memory.json"
|
||||||
|
legacy_path.parent.mkdir(parents=True)
|
||||||
|
summaries = {
|
||||||
|
"user": {"workContext": {"summary": "historical profile", "updatedAt": "then", "confidence": 0.8}, "providerState": {"key": "kept"}},
|
||||||
|
"history": {"recentMonths": {"summary": "historical context", "updatedAt": "then"}, "timeline": ["kept"]},
|
||||||
|
}
|
||||||
|
global_data = {"version": global_version, "revision": 0, "lastUpdated": "", **copy.deepcopy(summaries)}
|
||||||
|
if global_version == "1.0":
|
||||||
|
global_data["facts"] = []
|
||||||
|
memory_path.write_text(json.dumps(global_data), encoding="utf-8")
|
||||||
|
fact = _memory_with_fact()["facts"][0]
|
||||||
|
legacy_path.write_text(json.dumps({"version": "1.0", **summaries, "facts": [fact]}), encoding="utf-8")
|
||||||
|
|
||||||
|
loaded = storage.load("__default__", user_id="alice")
|
||||||
|
|
||||||
|
assert [item["id"] for item in loaded["facts"]] == [fact["id"]]
|
||||||
|
assert loaded["user"]["workContext"] == summaries["user"]["workContext"]
|
||||||
|
assert loaded["user"]["providerState"] == summaries["user"]["providerState"]
|
||||||
|
assert loaded["history"]["timeline"] == ["kept"]
|
||||||
|
assert loaded["user"]["cognitiveStyle"] == {"summary": "", "updatedAt": ""}
|
||||||
|
assert not legacy_path.exists()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("facts,error_type", [([None], MemoryStorageCorruption), ([{"id": "invalid", "content": " "}], ValueError)])
|
||||||
|
def test_summary_normalization_does_not_relax_legacy_fact_migration(storage, facts, error_type):
|
||||||
|
memory_path = storage._get_memory_file_path("__default__", user_id="alice")
|
||||||
|
legacy_path = memory_path.parent / "agents" / "__default__" / "memory.json"
|
||||||
|
legacy_path.parent.mkdir(parents=True)
|
||||||
|
original = json.dumps({"version": "1.0", "user": {}, "history": {}, "facts": facts})
|
||||||
|
legacy_path.write_text(original, encoding="utf-8")
|
||||||
|
|
||||||
|
with pytest.raises(error_type):
|
||||||
|
storage.load("__default__", user_id="alice")
|
||||||
|
|
||||||
|
assert legacy_path.read_text(encoding="utf-8") == original
|
||||||
|
assert not memory_path.exists()
|
||||||
|
|||||||
@ -31,6 +31,7 @@ def _make_memory(facts: list[dict[str, object]] | None = None) -> dict[str, obje
|
|||||||
"workContext": {"summary": "", "updatedAt": ""},
|
"workContext": {"summary": "", "updatedAt": ""},
|
||||||
"personalContext": {"summary": "", "updatedAt": ""},
|
"personalContext": {"summary": "", "updatedAt": ""},
|
||||||
"topOfMind": {"summary": "", "updatedAt": ""},
|
"topOfMind": {"summary": "", "updatedAt": ""},
|
||||||
|
"cognitiveStyle": {"summary": "", "updatedAt": ""},
|
||||||
},
|
},
|
||||||
"history": {
|
"history": {
|
||||||
"recentMonths": {"summary": "", "updatedAt": ""},
|
"recentMonths": {"summary": "", "updatedAt": ""},
|
||||||
|
|||||||
@ -34,6 +34,7 @@ import {
|
|||||||
useMemory,
|
useMemory,
|
||||||
useUpdateMemoryFact,
|
useUpdateMemoryFact,
|
||||||
} from "@/core/memory/hooks";
|
} from "@/core/memory/hooks";
|
||||||
|
import { normalizeMemoryPayload } from "@/core/memory/import-memory";
|
||||||
import type {
|
import type {
|
||||||
MemoryFactInput,
|
MemoryFactInput,
|
||||||
MemoryFactPatchInput,
|
MemoryFactPatchInput,
|
||||||
@ -68,60 +69,6 @@ type PendingImport = {
|
|||||||
memory: UserMemory;
|
memory: UserMemory;
|
||||||
};
|
};
|
||||||
|
|
||||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
|
||||||
return typeof value === "object" && value !== null;
|
|
||||||
}
|
|
||||||
|
|
||||||
function isMemorySection(value: unknown): value is {
|
|
||||||
summary: string;
|
|
||||||
updatedAt: string;
|
|
||||||
} {
|
|
||||||
return (
|
|
||||||
isRecord(value) &&
|
|
||||||
typeof value.summary === "string" &&
|
|
||||||
typeof value.updatedAt === "string"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function isMemoryFact(value: unknown): value is UserMemory["facts"][number] {
|
|
||||||
return (
|
|
||||||
isRecord(value) &&
|
|
||||||
typeof value.id === "string" &&
|
|
||||||
typeof value.content === "string" &&
|
|
||||||
typeof value.category === "string" &&
|
|
||||||
typeof value.confidence === "number" &&
|
|
||||||
Number.isFinite(value.confidence) &&
|
|
||||||
typeof value.createdAt === "string" &&
|
|
||||||
typeof value.source === "string"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function isImportedMemory(value: unknown): value is UserMemory {
|
|
||||||
if (!isRecord(value)) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (
|
|
||||||
typeof value.version !== "string" ||
|
|
||||||
typeof value.lastUpdated !== "string" ||
|
|
||||||
!isRecord(value.user) ||
|
|
||||||
!isRecord(value.history) ||
|
|
||||||
!Array.isArray(value.facts)
|
|
||||||
) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
return (
|
|
||||||
isMemorySection(value.user.workContext) &&
|
|
||||||
isMemorySection(value.user.personalContext) &&
|
|
||||||
isMemorySection(value.user.topOfMind) &&
|
|
||||||
isMemorySection(value.history.recentMonths) &&
|
|
||||||
isMemorySection(value.history.earlierContext) &&
|
|
||||||
isMemorySection(value.history.longTermBackground) &&
|
|
||||||
value.facts.every(isMemoryFact)
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
type FactFormState = {
|
type FactFormState = {
|
||||||
content: string;
|
content: string;
|
||||||
category: string;
|
category: string;
|
||||||
@ -189,6 +136,11 @@ function buildMemorySectionGroups(
|
|||||||
summary: memory.user.topOfMind.summary,
|
summary: memory.user.topOfMind.summary,
|
||||||
updatedAt: memory.user.topOfMind.updatedAt,
|
updatedAt: memory.user.topOfMind.updatedAt,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
title: t.settings.memory.markdown.cognitiveStyle,
|
||||||
|
summary: memory.user.cognitiveStyle.summary,
|
||||||
|
updatedAt: memory.user.cognitiveStyle.updatedAt,
|
||||||
|
},
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -255,6 +207,7 @@ function isMemorySummaryEmpty(memory: UserMemory) {
|
|||||||
memory.user.workContext.summary.trim() === "" &&
|
memory.user.workContext.summary.trim() === "" &&
|
||||||
memory.user.personalContext.summary.trim() === "" &&
|
memory.user.personalContext.summary.trim() === "" &&
|
||||||
memory.user.topOfMind.summary.trim() === "" &&
|
memory.user.topOfMind.summary.trim() === "" &&
|
||||||
|
memory.user.cognitiveStyle.summary.trim() === "" &&
|
||||||
memory.history.recentMonths.summary.trim() === "" &&
|
memory.history.recentMonths.summary.trim() === "" &&
|
||||||
memory.history.earlierContext.summary.trim() === "" &&
|
memory.history.earlierContext.summary.trim() === "" &&
|
||||||
memory.history.longTermBackground.summary.trim() === ""
|
memory.history.longTermBackground.summary.trim() === ""
|
||||||
@ -277,6 +230,13 @@ function upperFirst(str: string) {
|
|||||||
return str.charAt(0).toUpperCase() + str.slice(1);
|
return str.charAt(0).toUpperCase() + str.slice(1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function formatFactCreatedAt(createdAt: string, unknownLabel: string) {
|
||||||
|
if (!createdAt || Number.isNaN(Date.parse(createdAt))) {
|
||||||
|
return unknownLabel;
|
||||||
|
}
|
||||||
|
return formatTimeAgo(createdAt);
|
||||||
|
}
|
||||||
|
|
||||||
export function MemorySettingsPage() {
|
export function MemorySettingsPage() {
|
||||||
const { t } = useI18n();
|
const { t } = useI18n();
|
||||||
const { memory, isLoading, error } = useMemory();
|
const { memory, isLoading, error } = useMemory();
|
||||||
@ -424,13 +384,14 @@ export function MemorySettingsPage() {
|
|||||||
|
|
||||||
try {
|
try {
|
||||||
const parsed: unknown = JSON.parse(await file.text());
|
const parsed: unknown = JSON.parse(await file.text());
|
||||||
if (!isImportedMemory(parsed)) {
|
const memory = normalizeMemoryPayload(parsed);
|
||||||
|
if (!memory) {
|
||||||
toast.error(t.settings.memory.importInvalidFile);
|
toast.error(t.settings.memory.importInvalidFile);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
setPendingImport({
|
setPendingImport({
|
||||||
fileName: file.name,
|
fileName: file.name,
|
||||||
memory: parsed,
|
memory,
|
||||||
});
|
});
|
||||||
} catch {
|
} catch {
|
||||||
toast.error(t.settings.memory.importInvalidFile);
|
toast.error(t.settings.memory.importInvalidFile);
|
||||||
@ -703,7 +664,10 @@ export function MemorySettingsPage() {
|
|||||||
<span className="text-muted-foreground">
|
<span className="text-muted-foreground">
|
||||||
{t.settings.memory.markdown.table.createdAt}:
|
{t.settings.memory.markdown.table.createdAt}:
|
||||||
</span>{" "}
|
</span>{" "}
|
||||||
{formatTimeAgo(fact.createdAt)}
|
{formatFactCreatedAt(
|
||||||
|
fact.createdAt,
|
||||||
|
t.settings.memory.markdown.table.unknown,
|
||||||
|
)}
|
||||||
</span>
|
</span>
|
||||||
<span>
|
<span>
|
||||||
<span className="text-muted-foreground">
|
<span className="text-muted-foreground">
|
||||||
@ -711,13 +675,15 @@ export function MemorySettingsPage() {
|
|||||||
</span>{" "}
|
</span>{" "}
|
||||||
{fact.source === "manual" ? (
|
{fact.source === "manual" ? (
|
||||||
t.settings.memory.manualFactSource
|
t.settings.memory.manualFactSource
|
||||||
) : (
|
) : fact.source && fact.source !== "unknown" ? (
|
||||||
<Link
|
<Link
|
||||||
href={pathOfThread(fact.source)}
|
href={pathOfThread(fact.source)}
|
||||||
className="text-primary underline-offset-4 hover:underline"
|
className="text-primary underline-offset-4 hover:underline"
|
||||||
>
|
>
|
||||||
{t.settings.memory.markdown.table.view}
|
{t.settings.memory.markdown.table.view}
|
||||||
</Link>
|
</Link>
|
||||||
|
) : (
|
||||||
|
t.settings.memory.markdown.table.unknown
|
||||||
)}
|
)}
|
||||||
</span>
|
</span>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@ -99,7 +99,12 @@ export async function staticApiResponse(
|
|||||||
data = {
|
data = {
|
||||||
version: "1.0",
|
version: "1.0",
|
||||||
lastUpdated: "",
|
lastUpdated: "",
|
||||||
user: { workContext: empty, personalContext: empty, topOfMind: empty },
|
user: {
|
||||||
|
workContext: empty,
|
||||||
|
personalContext: empty,
|
||||||
|
topOfMind: empty,
|
||||||
|
cognitiveStyle: empty,
|
||||||
|
},
|
||||||
history: {
|
history: {
|
||||||
recentMonths: empty,
|
recentMonths: empty,
|
||||||
earlierContext: empty,
|
earlierContext: empty,
|
||||||
|
|||||||
@ -1416,6 +1416,7 @@ export const enUS: Translations = {
|
|||||||
work: "Work",
|
work: "Work",
|
||||||
personal: "Personal",
|
personal: "Personal",
|
||||||
topOfMind: "Top of mind",
|
topOfMind: "Top of mind",
|
||||||
|
cognitiveStyle: "Thinking style",
|
||||||
historyBackground: "History",
|
historyBackground: "History",
|
||||||
recentMonths: "Recent months",
|
recentMonths: "Recent months",
|
||||||
earlierContext: "Earlier context",
|
earlierContext: "Earlier context",
|
||||||
@ -1434,6 +1435,7 @@ export const enUS: Translations = {
|
|||||||
},
|
},
|
||||||
content: "Content",
|
content: "Content",
|
||||||
source: "Source",
|
source: "Source",
|
||||||
|
unknown: "Unknown",
|
||||||
createdAt: "CreatedAt",
|
createdAt: "CreatedAt",
|
||||||
view: "View",
|
view: "View",
|
||||||
},
|
},
|
||||||
|
|||||||
@ -1203,6 +1203,7 @@ export interface Translations {
|
|||||||
work: string;
|
work: string;
|
||||||
personal: string;
|
personal: string;
|
||||||
topOfMind: string;
|
topOfMind: string;
|
||||||
|
cognitiveStyle: string;
|
||||||
historyBackground: string;
|
historyBackground: string;
|
||||||
recentMonths: string;
|
recentMonths: string;
|
||||||
earlierContext: string;
|
earlierContext: string;
|
||||||
@ -1221,6 +1222,7 @@ export interface Translations {
|
|||||||
};
|
};
|
||||||
content: string;
|
content: string;
|
||||||
source: string;
|
source: string;
|
||||||
|
unknown: string;
|
||||||
createdAt: string;
|
createdAt: string;
|
||||||
view: string;
|
view: string;
|
||||||
};
|
};
|
||||||
|
|||||||
@ -1332,6 +1332,7 @@ export const zhCN: Translations = {
|
|||||||
work: "工作",
|
work: "工作",
|
||||||
personal: "个人",
|
personal: "个人",
|
||||||
topOfMind: "近期关注(Top of mind)",
|
topOfMind: "近期关注(Top of mind)",
|
||||||
|
cognitiveStyle: "思维习惯",
|
||||||
historyBackground: "历史背景",
|
historyBackground: "历史背景",
|
||||||
recentMonths: "近几个月",
|
recentMonths: "近几个月",
|
||||||
earlierContext: "更早上下文",
|
earlierContext: "更早上下文",
|
||||||
@ -1350,6 +1351,7 @@ export const zhCN: Translations = {
|
|||||||
},
|
},
|
||||||
content: "内容",
|
content: "内容",
|
||||||
source: "来源",
|
source: "来源",
|
||||||
|
unknown: "未知",
|
||||||
createdAt: "创建时间",
|
createdAt: "创建时间",
|
||||||
view: "查看",
|
view: "查看",
|
||||||
},
|
},
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
import { fetch } from "../api/fetcher";
|
import { fetch } from "../api/fetcher";
|
||||||
import { getBackendBaseURL } from "../config";
|
import { getBackendBaseURL } from "../config";
|
||||||
|
|
||||||
|
import { normalizeUserMemory } from "./normalize";
|
||||||
import type {
|
import type {
|
||||||
MemoryFactInput,
|
MemoryFactInput,
|
||||||
MemoryFactPatchInput,
|
MemoryFactPatchInput,
|
||||||
@ -77,7 +78,7 @@ async function readMemoryResponse(
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.json() as Promise<UserMemory>;
|
return normalizeUserMemory(await response.json());
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function loadMemory(): Promise<UserMemory> {
|
export async function loadMemory(): Promise<UserMemory> {
|
||||||
|
|||||||
170
frontend/src/core/memory/import-memory.ts
Normal file
170
frontend/src/core/memory/import-memory.ts
Normal file
@ -0,0 +1,170 @@
|
|||||||
|
import type { UserMemory } from "./types";
|
||||||
|
|
||||||
|
type ContextSection = UserMemory["user"]["workContext"];
|
||||||
|
type MemoryFact = UserMemory["facts"][number];
|
||||||
|
type InvalidFactStrategy = "reject" | "drop";
|
||||||
|
|
||||||
|
const USER_SECTION_KEYS = [
|
||||||
|
"workContext",
|
||||||
|
"personalContext",
|
||||||
|
"topOfMind",
|
||||||
|
"cognitiveStyle",
|
||||||
|
] as const satisfies ReadonlyArray<keyof UserMemory["user"]>;
|
||||||
|
|
||||||
|
const HISTORY_SECTION_KEYS = [
|
||||||
|
"recentMonths",
|
||||||
|
"earlierContext",
|
||||||
|
"longTermBackground",
|
||||||
|
] as const satisfies ReadonlyArray<keyof UserMemory["history"]>;
|
||||||
|
|
||||||
|
function emptySection(): ContextSection {
|
||||||
|
return { summary: "", updatedAt: "" };
|
||||||
|
}
|
||||||
|
|
||||||
|
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||||
|
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeContextSection(value: unknown): ContextSection {
|
||||||
|
if (!isRecord(value)) {
|
||||||
|
return emptySection();
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
...value,
|
||||||
|
summary: typeof value.summary === "string" ? value.summary : "",
|
||||||
|
updatedAt: typeof value.updatedAt === "string" ? value.updatedAt : "",
|
||||||
|
} as ContextSection;
|
||||||
|
}
|
||||||
|
|
||||||
|
function generateLegacyFactId(index: number): string {
|
||||||
|
const randomUUID = globalThis.crypto?.randomUUID?.();
|
||||||
|
return randomUUID
|
||||||
|
? `fact_${randomUUID.replaceAll("-", "").slice(0, 8)}`
|
||||||
|
: `fact_legacy_${index}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeMemoryFact(value: unknown, index: number): MemoryFact | null {
|
||||||
|
if (!isRecord(value)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const content = typeof value.content === "string" ? value.content.trim() : "";
|
||||||
|
if (!content) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const category =
|
||||||
|
typeof value.category === "string" && value.category.trim()
|
||||||
|
? value.category.trim()
|
||||||
|
: "context";
|
||||||
|
const rawConfidence = value.confidence;
|
||||||
|
const numericConfidence =
|
||||||
|
typeof rawConfidence === "number"
|
||||||
|
? rawConfidence
|
||||||
|
: typeof rawConfidence === "string" &&
|
||||||
|
/^[+-]?(?:\d+\.?\d*|\.\d+)(?:e[+-]?\d+)?$/i.test(rawConfidence.trim())
|
||||||
|
? Number(rawConfidence)
|
||||||
|
: NaN;
|
||||||
|
const confidence = Number.isFinite(numericConfidence)
|
||||||
|
? Math.min(1, Math.max(0, numericConfidence))
|
||||||
|
: 0.5;
|
||||||
|
|
||||||
|
const fact = {
|
||||||
|
...value,
|
||||||
|
id:
|
||||||
|
typeof value.id === "string" && value.id.trim()
|
||||||
|
? value.id.trim()
|
||||||
|
: generateLegacyFactId(index),
|
||||||
|
content,
|
||||||
|
category,
|
||||||
|
confidence,
|
||||||
|
createdAt:
|
||||||
|
typeof value.createdAt === "string" ? value.createdAt.trim() : "",
|
||||||
|
source:
|
||||||
|
typeof value.source === "string" && value.source.trim()
|
||||||
|
? value.source.trim()
|
||||||
|
: "unknown",
|
||||||
|
} as MemoryFact & Record<string, unknown>;
|
||||||
|
|
||||||
|
if (
|
||||||
|
"sourceError" in fact &&
|
||||||
|
fact.sourceError !== null &&
|
||||||
|
typeof fact.sourceError !== "string"
|
||||||
|
) {
|
||||||
|
delete fact.sourceError;
|
||||||
|
}
|
||||||
|
|
||||||
|
return fact;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Normalize and validate memory JSON (unknown → UserMemory | null).
|
||||||
|
*
|
||||||
|
* Normalization is additive: only contract-owned fields are validated and
|
||||||
|
* defaulted, while every unrecognized field (top-level, section, and per-fact)
|
||||||
|
* passes through untouched. The frontend must never be narrower than the
|
||||||
|
* Gateway contract — `MemoryResponse` declares fields such as the top-level
|
||||||
|
* `revision`, and rebuilding from a whitelist here would silently drop them
|
||||||
|
* from every response passing through `readMemoryResponse()`.
|
||||||
|
*
|
||||||
|
* The envelope (string `version`/`lastUpdated`, record `user`/`history`, array
|
||||||
|
* `facts`) is strict on both call paths. Unrecoverable facts can either reject
|
||||||
|
* a user-initiated import or be dropped on the background API read path.
|
||||||
|
*/
|
||||||
|
export function normalizeMemoryPayload(
|
||||||
|
value: unknown,
|
||||||
|
options: { invalidFactStrategy?: InvalidFactStrategy } = {},
|
||||||
|
): UserMemory | null {
|
||||||
|
if (
|
||||||
|
!isRecord(value) ||
|
||||||
|
typeof value.version !== "string" ||
|
||||||
|
typeof value.lastUpdated !== "string" ||
|
||||||
|
!isRecord(value.user) ||
|
||||||
|
!isRecord(value.history) ||
|
||||||
|
!Array.isArray(value.facts)
|
||||||
|
) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const user = value.user;
|
||||||
|
const history = value.history;
|
||||||
|
const invalidFactStrategy = options.invalidFactStrategy ?? "reject";
|
||||||
|
const facts: MemoryFact[] = [];
|
||||||
|
|
||||||
|
for (const [index, factValue] of value.facts.entries()) {
|
||||||
|
const fact = normalizeMemoryFact(factValue, index);
|
||||||
|
if (!fact) {
|
||||||
|
if (invalidFactStrategy === "reject") {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
facts.push(fact);
|
||||||
|
}
|
||||||
|
|
||||||
|
const normalizedUser = {
|
||||||
|
...user,
|
||||||
|
...Object.fromEntries(
|
||||||
|
USER_SECTION_KEYS.map((key) => [key, normalizeContextSection(user[key])]),
|
||||||
|
),
|
||||||
|
} as unknown as UserMemory["user"];
|
||||||
|
const normalizedHistory = {
|
||||||
|
...history,
|
||||||
|
...Object.fromEntries(
|
||||||
|
HISTORY_SECTION_KEYS.map((key) => [
|
||||||
|
key,
|
||||||
|
normalizeContextSection(history[key]),
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
} as unknown as UserMemory["history"];
|
||||||
|
|
||||||
|
return {
|
||||||
|
...value,
|
||||||
|
version: value.version,
|
||||||
|
lastUpdated: value.lastUpdated,
|
||||||
|
user: normalizedUser,
|
||||||
|
history: normalizedHistory,
|
||||||
|
facts,
|
||||||
|
} as unknown as UserMemory;
|
||||||
|
}
|
||||||
13
frontend/src/core/memory/normalize.ts
Normal file
13
frontend/src/core/memory/normalize.ts
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
import { normalizeMemoryPayload } from "./import-memory";
|
||||||
|
import type { UserMemory } from "./types";
|
||||||
|
|
||||||
|
/** Normalize API/import payloads (unknown → UserMemory). Throws if invalid. */
|
||||||
|
export function normalizeUserMemory(value: unknown): UserMemory {
|
||||||
|
const normalized = normalizeMemoryPayload(value, {
|
||||||
|
invalidFactStrategy: "drop",
|
||||||
|
});
|
||||||
|
if (!normalized) {
|
||||||
|
throw new Error("Invalid memory payload");
|
||||||
|
}
|
||||||
|
return normalized;
|
||||||
|
}
|
||||||
@ -5,6 +5,7 @@ export interface MemoryFact {
|
|||||||
confidence: number;
|
confidence: number;
|
||||||
createdAt: string;
|
createdAt: string;
|
||||||
source: string;
|
source: string;
|
||||||
|
[key: string]: unknown;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface MemoryFactInput {
|
export interface MemoryFactInput {
|
||||||
@ -21,6 +22,7 @@ export interface MemoryFactPatchInput {
|
|||||||
|
|
||||||
export interface UserMemory {
|
export interface UserMemory {
|
||||||
version: string;
|
version: string;
|
||||||
|
revision?: number;
|
||||||
lastUpdated: string;
|
lastUpdated: string;
|
||||||
user: {
|
user: {
|
||||||
workContext: {
|
workContext: {
|
||||||
@ -35,6 +37,10 @@ export interface UserMemory {
|
|||||||
summary: string;
|
summary: string;
|
||||||
updatedAt: string;
|
updatedAt: string;
|
||||||
};
|
};
|
||||||
|
cognitiveStyle: {
|
||||||
|
summary: string;
|
||||||
|
updatedAt: string;
|
||||||
|
};
|
||||||
};
|
};
|
||||||
history: {
|
history: {
|
||||||
recentMonths: {
|
recentMonths: {
|
||||||
|
|||||||
194
frontend/tests/e2e/settings-memory-import.spec.ts
Normal file
194
frontend/tests/e2e/settings-memory-import.spec.ts
Normal file
@ -0,0 +1,194 @@
|
|||||||
|
import { expect, test, type Page } from "@playwright/test";
|
||||||
|
|
||||||
|
import { mockLangGraphAPI } from "./utils/mock-api";
|
||||||
|
|
||||||
|
const EMPTY_MEMORY = {
|
||||||
|
version: "1.0",
|
||||||
|
lastUpdated: "",
|
||||||
|
user: {
|
||||||
|
workContext: { summary: "", updatedAt: "" },
|
||||||
|
personalContext: { summary: "", updatedAt: "" },
|
||||||
|
topOfMind: { summary: "", updatedAt: "" },
|
||||||
|
cognitiveStyle: { summary: "", updatedAt: "" },
|
||||||
|
},
|
||||||
|
history: {
|
||||||
|
recentMonths: { summary: "", updatedAt: "" },
|
||||||
|
earlierContext: { summary: "", updatedAt: "" },
|
||||||
|
longTermBackground: { summary: "", updatedAt: "" },
|
||||||
|
},
|
||||||
|
facts: [],
|
||||||
|
};
|
||||||
|
|
||||||
|
const LEGACY_MEMORY_WITHOUT_COGNITIVE_STYLE = {
|
||||||
|
version: "1.0",
|
||||||
|
lastUpdated: "2026-01-01T00:00:00Z",
|
||||||
|
user: {
|
||||||
|
workContext: { summary: "Works on DeerFlow", updatedAt: "" },
|
||||||
|
personalContext: { summary: "", updatedAt: "" },
|
||||||
|
topOfMind: { summary: "Memory import compatibility", updatedAt: "" },
|
||||||
|
},
|
||||||
|
history: {
|
||||||
|
recentMonths: { summary: "", updatedAt: "" },
|
||||||
|
earlierContext: { summary: "", updatedAt: "" },
|
||||||
|
longTermBackground: { summary: "", updatedAt: "" },
|
||||||
|
},
|
||||||
|
facts: [
|
||||||
|
{
|
||||||
|
content: "User prefers conclusions first.",
|
||||||
|
category: "cognitive",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
async function openMemorySettings(page: Page) {
|
||||||
|
mockLangGraphAPI(page);
|
||||||
|
await page.route(/\/api\/memory$/, async (route) => {
|
||||||
|
if (route.request().method() === "GET") {
|
||||||
|
await route.fulfill({
|
||||||
|
status: 200,
|
||||||
|
contentType: "application/json",
|
||||||
|
body: JSON.stringify(EMPTY_MEMORY),
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await route.fallback();
|
||||||
|
});
|
||||||
|
|
||||||
|
await page.goto("/workspace/chats/new");
|
||||||
|
const sidebar = page.locator("[data-sidebar='sidebar']");
|
||||||
|
await sidebar.getByRole("button", { name: /Settings and more/ }).click();
|
||||||
|
await page.getByRole("menuitem", { name: "Settings" }).click();
|
||||||
|
|
||||||
|
const settingsDialog = page.getByRole("dialog", { name: "Settings" });
|
||||||
|
await expect(settingsDialog).toBeVisible();
|
||||||
|
await settingsDialog.getByRole("button", { name: "Memory" }).click();
|
||||||
|
await expect(
|
||||||
|
settingsDialog.getByRole("button", { name: "Import memory" }),
|
||||||
|
).toBeVisible();
|
||||||
|
return settingsDialog;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function selectMemoryFile(
|
||||||
|
settingsDialog: ReturnType<Page["getByRole"]>,
|
||||||
|
fileName: string,
|
||||||
|
payload: unknown,
|
||||||
|
) {
|
||||||
|
await settingsDialog.locator('input[type="file"]').setInputFiles({
|
||||||
|
name: fileName,
|
||||||
|
mimeType: "application/json",
|
||||||
|
buffer: Buffer.from(JSON.stringify(payload)),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const invalidImports = [
|
||||||
|
{
|
||||||
|
name: "facts-only JSON",
|
||||||
|
fileName: "facts-only.json",
|
||||||
|
payload: { facts: [] },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "missing metadata",
|
||||||
|
fileName: "missing-metadata.json",
|
||||||
|
payload: { user: {}, history: {}, facts: [] },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "non-object user/history",
|
||||||
|
fileName: "invalid-sections.json",
|
||||||
|
payload: {
|
||||||
|
version: "1.0",
|
||||||
|
lastUpdated: "2026-07-17T00:00:00Z",
|
||||||
|
user: "not-an-object",
|
||||||
|
history: 123,
|
||||||
|
facts: [],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
test.describe("Memory settings import validation", () => {
|
||||||
|
for (const invalidImport of invalidImports) {
|
||||||
|
test(`does not enable confirmation for ${invalidImport.name}`, async ({
|
||||||
|
page,
|
||||||
|
}) => {
|
||||||
|
const settingsDialog = await openMemorySettings(page);
|
||||||
|
|
||||||
|
await selectMemoryFile(
|
||||||
|
settingsDialog,
|
||||||
|
invalidImport.fileName,
|
||||||
|
invalidImport.payload,
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
page.getByText(
|
||||||
|
"Failed to read the selected memory file. Please choose a valid JSON export.",
|
||||||
|
),
|
||||||
|
).toBeVisible();
|
||||||
|
await expect(
|
||||||
|
page.getByRole("dialog", { name: "Import memory?" }),
|
||||||
|
).toHaveCount(0);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
test("keeps confirmation available for a legacy export missing cognitiveStyle", async ({
|
||||||
|
page,
|
||||||
|
}) => {
|
||||||
|
const settingsDialog = await openMemorySettings(page);
|
||||||
|
|
||||||
|
await selectMemoryFile(
|
||||||
|
settingsDialog,
|
||||||
|
"legacy-without-cognitive-style.json",
|
||||||
|
LEGACY_MEMORY_WITHOUT_COGNITIVE_STYLE,
|
||||||
|
);
|
||||||
|
|
||||||
|
const confirmDialog = page.getByRole("dialog", { name: "Import memory?" });
|
||||||
|
await expect(confirmDialog).toBeVisible();
|
||||||
|
await expect(confirmDialog).toContainText(
|
||||||
|
"legacy-without-cognitive-style.json",
|
||||||
|
);
|
||||||
|
await expect(
|
||||||
|
confirmDialog.getByRole("button", { name: "Import" }),
|
||||||
|
).toBeEnabled();
|
||||||
|
});
|
||||||
|
|
||||||
|
test("round-trips unknown fields through the import request", async ({
|
||||||
|
page,
|
||||||
|
}) => {
|
||||||
|
const memoryWithExtensions = {
|
||||||
|
...LEGACY_MEMORY_WITHOUT_COGNITIVE_STYLE,
|
||||||
|
revision: 4,
|
||||||
|
display: { title: "Memory export" },
|
||||||
|
data: { future: true },
|
||||||
|
};
|
||||||
|
const importedPayloads: unknown[] = [];
|
||||||
|
await page.route(/\/api\/memory\/import$/, async (route) => {
|
||||||
|
if (route.request().method() === "POST") {
|
||||||
|
importedPayloads.push(route.request().postDataJSON());
|
||||||
|
await route.fulfill({
|
||||||
|
status: 200,
|
||||||
|
contentType: "application/json",
|
||||||
|
body: JSON.stringify(memoryWithExtensions),
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await route.fallback();
|
||||||
|
});
|
||||||
|
|
||||||
|
const settingsDialog = await openMemorySettings(page);
|
||||||
|
await selectMemoryFile(
|
||||||
|
settingsDialog,
|
||||||
|
"with-extensions.json",
|
||||||
|
memoryWithExtensions,
|
||||||
|
);
|
||||||
|
|
||||||
|
const confirmDialog = page.getByRole("dialog", { name: "Import memory?" });
|
||||||
|
await expect(confirmDialog).toBeVisible();
|
||||||
|
await confirmDialog.getByRole("button", { name: "Import" }).click();
|
||||||
|
|
||||||
|
await expect.poll(() => importedPayloads.length).toBe(1);
|
||||||
|
expect(importedPayloads[0]).toMatchObject({
|
||||||
|
revision: 4,
|
||||||
|
display: { title: "Memory export" },
|
||||||
|
data: { future: true },
|
||||||
|
user: { cognitiveStyle: { summary: "", updatedAt: "" } },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
45
frontend/tests/unit/core/memory/fixtures.ts
Normal file
45
frontend/tests/unit/core/memory/fixtures.ts
Normal file
@ -0,0 +1,45 @@
|
|||||||
|
import type { UserMemory } from "@/core/memory/types";
|
||||||
|
|
||||||
|
/** Legacy export shape before user.cognitiveStyle existed (PR #3182). */
|
||||||
|
export function legacyMemoryWithoutCognitiveStyle(): Omit<
|
||||||
|
UserMemory,
|
||||||
|
"user"
|
||||||
|
> & {
|
||||||
|
user: Omit<UserMemory["user"], "cognitiveStyle"> & {
|
||||||
|
workContext: UserMemory["user"]["workContext"];
|
||||||
|
personalContext: UserMemory["user"]["personalContext"];
|
||||||
|
topOfMind: UserMemory["user"]["topOfMind"];
|
||||||
|
};
|
||||||
|
} {
|
||||||
|
return {
|
||||||
|
version: "1.0",
|
||||||
|
lastUpdated: "2026-01-01T00:00:00Z",
|
||||||
|
user: {
|
||||||
|
workContext: {
|
||||||
|
summary: "Works on DeerFlow",
|
||||||
|
updatedAt: "2026-01-01T00:00:00Z",
|
||||||
|
},
|
||||||
|
personalContext: { summary: "", updatedAt: "" },
|
||||||
|
topOfMind: { summary: "Memory import compatibility", updatedAt: "" },
|
||||||
|
},
|
||||||
|
history: {
|
||||||
|
recentMonths: { summary: "", updatedAt: "" },
|
||||||
|
earlierContext: { summary: "", updatedAt: "" },
|
||||||
|
longTermBackground: { summary: "", updatedAt: "" },
|
||||||
|
},
|
||||||
|
facts: [],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Legacy payload whose facts predate generated ids and source/timestamp metadata. */
|
||||||
|
export function legacyMemoryWithIncompleteFacts() {
|
||||||
|
return {
|
||||||
|
...legacyMemoryWithoutCognitiveStyle(),
|
||||||
|
facts: [
|
||||||
|
{
|
||||||
|
content: "User prefers conclusions first.",
|
||||||
|
category: "cognitive",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
}
|
||||||
262
frontend/tests/unit/core/memory/import-memory.test.ts
Normal file
262
frontend/tests/unit/core/memory/import-memory.test.ts
Normal file
@ -0,0 +1,262 @@
|
|||||||
|
import { describe, expect, it } from "@rstest/core";
|
||||||
|
|
||||||
|
import { normalizeMemoryPayload } from "@/core/memory/import-memory";
|
||||||
|
|
||||||
|
import {
|
||||||
|
legacyMemoryWithIncompleteFacts,
|
||||||
|
legacyMemoryWithoutCognitiveStyle,
|
||||||
|
} from "./fixtures";
|
||||||
|
|
||||||
|
/** Legacy strict guard (pre–normalizeMemoryPayload): required every section to exist. */
|
||||||
|
function isImportedMemoryStrict(value: unknown): boolean {
|
||||||
|
if (typeof value !== "object" || value === null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
const record = value as Record<string, unknown>;
|
||||||
|
if (
|
||||||
|
typeof record.version !== "string" ||
|
||||||
|
typeof record.lastUpdated !== "string" ||
|
||||||
|
typeof record.user !== "object" ||
|
||||||
|
record.user === null ||
|
||||||
|
typeof record.history !== "object" ||
|
||||||
|
record.history === null ||
|
||||||
|
!Array.isArray(record.facts)
|
||||||
|
) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
const user = record.user as Record<string, unknown>;
|
||||||
|
const history = record.history as Record<string, unknown>;
|
||||||
|
|
||||||
|
function isSection(section: unknown): boolean {
|
||||||
|
if (typeof section !== "object" || section === null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
const s = section as Record<string, unknown>;
|
||||||
|
return typeof s.summary === "string" && typeof s.updatedAt === "string";
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
isSection(user.workContext) &&
|
||||||
|
isSection(user.personalContext) &&
|
||||||
|
isSection(user.topOfMind) &&
|
||||||
|
isSection(user.cognitiveStyle) &&
|
||||||
|
isSection(history.recentMonths) &&
|
||||||
|
isSection(history.earlierContext) &&
|
||||||
|
isSection(history.longTermBackground)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("legacy memory import compatibility (TDD)", () => {
|
||||||
|
const legacy = legacyMemoryWithoutCognitiveStyle();
|
||||||
|
|
||||||
|
it("legacy strict guard would reject exports missing cognitiveStyle", () => {
|
||||||
|
expect(isImportedMemoryStrict(legacy)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("normalizeMemoryPayload accepts legacy export without cognitiveStyle", () => {
|
||||||
|
const result = normalizeMemoryPayload(legacy);
|
||||||
|
|
||||||
|
expect(result).not.toBeNull();
|
||||||
|
expect(result!.user.cognitiveStyle).toEqual({
|
||||||
|
summary: "",
|
||||||
|
updatedAt: "",
|
||||||
|
});
|
||||||
|
expect(result!.user.workContext.summary).toBe("Works on DeerFlow");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("normalizeMemoryPayload preserves existing cognitiveStyle summary", () => {
|
||||||
|
const withStyle = {
|
||||||
|
...legacy,
|
||||||
|
user: {
|
||||||
|
...legacy.user,
|
||||||
|
cognitiveStyle: {
|
||||||
|
summary: "Conclusions first, then details.",
|
||||||
|
updatedAt: "2026-02-01T00:00:00Z",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = normalizeMemoryPayload(withStyle);
|
||||||
|
|
||||||
|
expect(result).not.toBeNull();
|
||||||
|
expect(result!.user.cognitiveStyle.summary).toBe(
|
||||||
|
"Conclusions first, then details.",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("normalizeMemoryPayload returns null for non-object payloads", () => {
|
||||||
|
expect(normalizeMemoryPayload(null)).toBeNull();
|
||||||
|
expect(normalizeMemoryPayload("not-json")).toBeNull();
|
||||||
|
expect(normalizeMemoryPayload({})).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects the truncated import envelope {"facts": []}', () => {
|
||||||
|
expect(normalizeMemoryPayload({ facts: [] })).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects imports with missing or malformed metadata", () => {
|
||||||
|
const valid = legacyMemoryWithoutCognitiveStyle();
|
||||||
|
|
||||||
|
expect(
|
||||||
|
normalizeMemoryPayload({
|
||||||
|
user: valid.user,
|
||||||
|
history: valid.history,
|
||||||
|
facts: valid.facts,
|
||||||
|
}),
|
||||||
|
).toBeNull();
|
||||||
|
expect(normalizeMemoryPayload({ ...valid, version: 1 })).toBeNull();
|
||||||
|
expect(normalizeMemoryPayload({ ...valid, lastUpdated: null })).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects imports with missing or non-record user/history envelopes", () => {
|
||||||
|
const valid = legacyMemoryWithoutCognitiveStyle();
|
||||||
|
|
||||||
|
expect(normalizeMemoryPayload({ ...valid, user: undefined })).toBeNull();
|
||||||
|
expect(normalizeMemoryPayload({ ...valid, history: undefined })).toBeNull();
|
||||||
|
expect(
|
||||||
|
normalizeMemoryPayload({ ...valid, user: "not-an-object" }),
|
||||||
|
).toBeNull();
|
||||||
|
expect(normalizeMemoryPayload({ ...valid, history: 123 })).toBeNull();
|
||||||
|
expect(normalizeMemoryPayload({ ...valid, user: [] })).toBeNull();
|
||||||
|
expect(normalizeMemoryPayload({ ...valid, history: [] })).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("normalizeMemoryPayload fills missing user sections like backend normalize", () => {
|
||||||
|
const partial = {
|
||||||
|
version: "1.0",
|
||||||
|
lastUpdated: "",
|
||||||
|
user: {
|
||||||
|
workContext: { summary: "only work", updatedAt: "" },
|
||||||
|
},
|
||||||
|
history: {},
|
||||||
|
facts: [],
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = normalizeMemoryPayload(partial);
|
||||||
|
|
||||||
|
expect(result).not.toBeNull();
|
||||||
|
expect(result!.user.workContext.summary).toBe("only work");
|
||||||
|
expect(result!.user.personalContext).toEqual({
|
||||||
|
summary: "",
|
||||||
|
updatedAt: "",
|
||||||
|
});
|
||||||
|
expect(result!.user.topOfMind).toEqual({ summary: "", updatedAt: "" });
|
||||||
|
expect(result!.user.cognitiveStyle).toEqual({
|
||||||
|
summary: "",
|
||||||
|
updatedAt: "",
|
||||||
|
});
|
||||||
|
expect(result!.history.recentMonths).toEqual({
|
||||||
|
summary: "",
|
||||||
|
updatedAt: "",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("normalizes legacy facts that are missing generated metadata", () => {
|
||||||
|
const result = normalizeMemoryPayload(legacyMemoryWithIncompleteFacts());
|
||||||
|
|
||||||
|
expect(result).not.toBeNull();
|
||||||
|
expect(result!.facts).toHaveLength(1);
|
||||||
|
expect(result!.facts[0]!.id).toMatch(/^fact_/);
|
||||||
|
expect(result!.facts[0]).toMatchObject({
|
||||||
|
content: "User prefers conclusions first.",
|
||||||
|
category: "cognitive",
|
||||||
|
confidence: 0.5,
|
||||||
|
createdAt: "",
|
||||||
|
source: "unknown",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects imports containing facts without usable content", () => {
|
||||||
|
const invalid = {
|
||||||
|
...legacyMemoryWithoutCognitiveStyle(),
|
||||||
|
facts: [{ category: "context" }],
|
||||||
|
};
|
||||||
|
|
||||||
|
expect(normalizeMemoryPayload(invalid)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("preserves unknown fields on the strict import path", () => {
|
||||||
|
const futureExport = {
|
||||||
|
version: "1.0",
|
||||||
|
revision: 4,
|
||||||
|
lastUpdated: "2026-07-01T00:00:00Z",
|
||||||
|
display: { title: "Memory export" },
|
||||||
|
data: { future: true },
|
||||||
|
user: {
|
||||||
|
workContext: {
|
||||||
|
summary: "Works on DeerFlow",
|
||||||
|
updatedAt: "2026-06-01T00:00:00Z",
|
||||||
|
confidence: 0.8,
|
||||||
|
},
|
||||||
|
providerState: { loaded: true },
|
||||||
|
},
|
||||||
|
history: {
|
||||||
|
timeline: { entries: ["2026-06"] },
|
||||||
|
},
|
||||||
|
facts: [
|
||||||
|
{
|
||||||
|
content: "User prefers conclusions first.",
|
||||||
|
category: "cognitive",
|
||||||
|
topics: ["communication"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = normalizeMemoryPayload(futureExport);
|
||||||
|
|
||||||
|
expect(result).not.toBeNull();
|
||||||
|
expect(result!.revision).toBe(4);
|
||||||
|
expect(result).toMatchObject({
|
||||||
|
display: { title: "Memory export" },
|
||||||
|
data: { future: true },
|
||||||
|
});
|
||||||
|
expect(result!.user).toMatchObject({
|
||||||
|
providerState: { loaded: true },
|
||||||
|
workContext: { confidence: 0.8 },
|
||||||
|
});
|
||||||
|
expect(result!.history).toMatchObject({
|
||||||
|
timeline: { entries: ["2026-06"] },
|
||||||
|
});
|
||||||
|
expect(result!.facts[0]).toMatchObject({
|
||||||
|
topics: ["communication"],
|
||||||
|
});
|
||||||
|
expect(result!.user.cognitiveStyle).toEqual({
|
||||||
|
summary: "",
|
||||||
|
updatedAt: "",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("legacy fact metadata parity", () => {
|
||||||
|
it.each([
|
||||||
|
[undefined, 0.5],
|
||||||
|
[null, 0.5],
|
||||||
|
[true, 0.5],
|
||||||
|
["invalid", 0.5],
|
||||||
|
[NaN, 0.5],
|
||||||
|
[Infinity, 0.5],
|
||||||
|
[0, 0],
|
||||||
|
[-1, 0],
|
||||||
|
[2, 1],
|
||||||
|
["0.8", 0.8],
|
||||||
|
])("normalizes confidence %s to %s", (confidence, expected) => {
|
||||||
|
const result = normalizeMemoryPayload({
|
||||||
|
...legacyMemoryWithoutCognitiveStyle(),
|
||||||
|
facts: [
|
||||||
|
{
|
||||||
|
id: "legacy",
|
||||||
|
content: " Keep conclusions first. ",
|
||||||
|
confidence,
|
||||||
|
source: " ",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
});
|
||||||
|
expect(result!.facts[0]).toMatchObject({
|
||||||
|
confidence: expected,
|
||||||
|
content: "Keep conclusions first.",
|
||||||
|
source: "unknown",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
41
frontend/tests/unit/core/memory/normalize.test.ts
Normal file
41
frontend/tests/unit/core/memory/normalize.test.ts
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
import { describe, expect, it } from "@rstest/core";
|
||||||
|
|
||||||
|
import { normalizeUserMemory } from "@/core/memory/normalize";
|
||||||
|
|
||||||
|
import {
|
||||||
|
legacyMemoryWithIncompleteFacts,
|
||||||
|
legacyMemoryWithoutCognitiveStyle,
|
||||||
|
} from "./fixtures";
|
||||||
|
|
||||||
|
describe("normalizeUserMemory (API read path)", () => {
|
||||||
|
it("fills cognitiveStyle when legacy payload omits it", () => {
|
||||||
|
const result = normalizeUserMemory(legacyMemoryWithoutCognitiveStyle());
|
||||||
|
|
||||||
|
expect(result.user.cognitiveStyle).toEqual({
|
||||||
|
summary: "",
|
||||||
|
updatedAt: "",
|
||||||
|
});
|
||||||
|
expect(result.user.workContext.summary).toBe("Works on DeerFlow");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("throws when payload is not a valid memory object", () => {
|
||||||
|
expect(() => normalizeUserMemory({})).toThrow("Invalid memory payload");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the API read path available when one fact is unrecoverable", () => {
|
||||||
|
const legacy = legacyMemoryWithIncompleteFacts();
|
||||||
|
const result = normalizeUserMemory({
|
||||||
|
...legacy,
|
||||||
|
facts: [...legacy.facts, { category: "context" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.facts).toHaveLength(1);
|
||||||
|
expect(result.facts[0]).toMatchObject({
|
||||||
|
content: "User prefers conclusions first.",
|
||||||
|
category: "cognitive",
|
||||||
|
confidence: 0.5,
|
||||||
|
createdAt: "",
|
||||||
|
source: "unknown",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
Loading…
x
Reference in New Issue
Block a user