wgwfn3 c54883cce0

fix(memory): drop punctuation tokens from jieba relevance ranking (#6388) ...

* fix(memory): drop punctuation tokens from jieba relevance ranking

jieba.cut emits punctuation as standalone tokens and tokenize() only filtered whitespace, so commas and full stops counted as query matches, inflated near-duplicate similarity, and consumed the 128-token budget. Keep only tokens with an alphanumeric character, matching the regex fallback and the FTS5 adapter's query filter.

Document the rule in agents/memory/AGENTS.md.

* docs(changelog): add the DeerMem jieba punctuation fix

* docs(changelog): add the PR reference for the jieba punctuation fix (#6388)

* docs(memory): document jieba relevance token filtering

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

2026-10-07 06:58:59 +08:00

..

extension-api

feat(extensions): add host-bound Agent run control (#6190)

2026-10-03 21:16:01 +08:00

harness

fix(memory): drop punctuation tokens from jieba relevance ranking (#6388)

2026-10-07 06:58:59 +08:00