betterkite b552b5015c
fix(messages): drop legacy <uploaded_files> tag handling (#4826)
* fix(messages): drop legacy <uploaded_files> tag handling (#4212)

PR #4174 unified upload-context injection on <current_uploads> (IM and web
both flow through UploadsMiddleware), and #4632 documented the current
path. This removes the remaining backward-compat parsing of the
pre-#4174 <uploaded_files> tag, the final cleanup item tracked by the
issue:

- deermem: only <current_uploads> is stripped from human turns before
  memory persistence, and the upload-sentence scrubber drops the legacy
  tag alternative.
- mem0: the mirrored message filter recognises only <current_uploads>.
- InputSanitizationMiddleware: remove the legacy tag from the blocked-tag
  denylist (it existed only because deermem parsed the old tag).
- frontend: stripUploadedFilesTag / stripInternalMarkers /
  parseUploadedFiles and the message-list fallback parse only
  <current_uploads>; demo thread fixtures are migrated to the current tag.

Scope decision: a <uploaded_files> block in pre-#4174 history is now
treated as ordinary user content (pinned by tests in both layers) instead
of being silently dropped or stripped.

* style: apply prettier formatting to stripUploadedFilesTag

* fix(uploads): keep legacy <uploaded_files> stripping for display/export only

Addresses review feedback on #4826: removing the legacy tag from the
frontend display layer made pre-#4174 threads render raw <uploaded_files>
XML (with server-side upload paths) in chat, copy data, and JSON exports.

The backend cleanup stands — memory pipelines and the sanitization denylist
treat only <current_uploads> as an internal marker. The frontend keeps the
legacy spelling in its display/export-only utilities
(stripUploadedFilesTag / INTERNAL_MARKER_TAGS / parseUploadedFiles and the
message-list fallback) so old history renders cleanly without leaking
internal paths, while the memory/sanitization scope-decision tests remain
unchanged.

Frontend tests now pin both spellings: <current_uploads> and legacy
<uploaded_files> are stripped from copy data, markdown leak-stripping, and
JSON exports.

* docs(ui): record accepted display-spoof tradeoff for legacy upload tag

Review note (willem-bd): since <uploaded_files> is off the sanitization
denylist, a live user can type the legacy spelling and fabricate file
chips / hide their own message text in display. Display-only and
self-inflicted with no backend semantics, so it is accepted for now;
documented at both the message-list fallback and stripUploadedFilesTag.
Age-gating the legacy spelling remains a possible follow-up.

---------

Co-authored-by: betterkite <313258397+betterkite@users.noreply.github.com>
2026-09-01 15:26:33 +08:00
..

mem0 memory backend

Uses mem0 (Platform hosted API, or any API-compatible self-hosted server) as DeerFlow's memory store. Fully stateless in-process: dedup, fact extraction, and storage are server-side, so it is safe for multi-worker Gateway deployments.

Configuration

memory:
  enabled: true
  injection_enabled: true
  manager_class: mem0
  mode: middleware            # or "tool"
  backend_config:
    api_key_env: MEM0_API_KEY          # key read from env, never in config.yaml
    base_url: https://api.mem0.ai      # or your self-hosted mem0 server
    allow_insecure_http: false         # true only for trusted local HTTP dev
    top_k: 8
    score_threshold: 0.1
    max_injection_chars: 12000
    timeout_seconds: 10
    startup_policy: fail_fast          # fail_fast | tolerate
    failure_policy:
      read: fail_open                  # fail_open | fail_closed
      write: log_and_drop              # log_and_drop | raise

Set the key in the environment: export MEM0_API_KEY=...

base_url must use HTTPS because every request carries the API key. For a trusted local-development server that only exposes HTTP, opt in explicitly with allow_insecure_http: true; do not use that setting across an untrusted network.

Identity mapping

DeerFlow mem0
user_id user_id
agent_name agent_id
thread_id run_id

Limitations

  • mode: middleware recall is query-less (the get_context contract carries no query): the bucket's most recent top_k memories are injected. For query-aware semantic recall use mode: tool.
  • mode: tool retains the passive per-turn write middleware for this backend, because mem0 extracts and deduplicates facts from conversations through add(). The agent still gains query-aware memory_search, while new conversations continue accumulating memory even though fact CRUD is not available.
  • Fact CRUD, import_memory, and Settings-page memory editing are not implemented (gateway returns 501). DeerMem remains the default backend.
  • No migration of existing DeerMem data.
  • log_and_drop write policy is at-most-once: a failed write is dropped.
  • memory_add/memory_update/memory_delete are backed by fact CRUD, which this backend does not implement; they return a clear unsupported-operation error. Conversation writes still happen through the retained middleware.

Async execution and failure behavior

The mem0 HTTP client is synchronous for compatibility with the MemoryManager contract. DeerFlow offloads it at every async boundary: the async middleware uses the manager's a* methods, and Gateway memory routes run sync management calls in worker threads. A slow mem0 request therefore does not block unrelated ASGI handlers or SSE heartbeats.

failure_policy.read: fail_open logs a recall failure and continues without new memory context. fail_closed propagates the backend error through prompt construction and aborts the run instead of silently degrading.