fix(llm-text): respect quoted reasoning tag delimiters (#6535)

Co-authored-by: Zhangchenhao-111 <285911435+Zhangchenhao-111@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
haochen 2026-10-09 15:57:18 +08:00 committed by GitHub
parent 999477b359
commit 157630285b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 38 additions and 3 deletions

View File

@ -1754,7 +1754,7 @@ Rebuild with `make up` after changing the managed extension set. See
Gateway-generated follow-up suggestions now normalize both plain-string model output and block/list-style rich content before parsing the JSON array response, so provider-specific content wrappers do not silently drop suggestions.
Backend response cleanup preserves unrelated tag names such as `<think-tank>` and `<think:note>` in follow-up suggestions and polished drafts; only the exact `<think>` name (optionally followed by attributes) starts a reasoning block. Self-closing `<think/>` and `<think />` tags are empty reasoning blocks and leave the following answer intact.
Backend response cleanup preserves unrelated tag names such as `<think-tank>` and `<think:note>` in follow-up suggestions and polished drafts; only the exact `<think>` name (optionally followed by attributes) starts a reasoning block. Self-closing `<think/>` and `<think />` tags are empty reasoning blocks and leave the following answer intact. Delimiters inside quoted attributes, such as `<think note=">"/>`, also leave the following answer intact.
The Web UI composer can polish draft input before sending. The rewrite runs as a short Gateway LLM request using the `input_polish` model configuration, keeps slash skill prefixes such as `/data-analysis`, and only replaces the local draft after the user clicks the polish button; it does not create a thread run or persist a message.

View File

@ -41,6 +41,8 @@ name; punctuated names such as `<think-tank>`, `<think:note>`, and `<think/other
are ordinary content. The shared prefix regex owns this boundary rule; do not
duplicate it in a manual re-check. Self-closing `<think/>` and `<think />` tags
are empty reasoning blocks: remove the tag without consuming subsequent text.
Both cleanup functions find opening delimiters outside quoted attributes; a
quoted `>` cannot terminate the tag or hide an answer after a self-closing tag.
Keep exact unfinished `<think` prefixes hidden in leading summaries. Coverage:
`tests/test_think_tag_boundaries.py`.

View File

@ -11,12 +11,27 @@ _THINK_OPEN_PREFIX_RE = re.compile(r"<think(?=[\s>]|/>|$)", re.IGNORECASE)
_THINK_CLOSE_PREFIX_RE = re.compile(r"</think", re.IGNORECASE)
def _find_tag_end(text: str, start: int) -> int:
"""Find the closing delimiter outside single- or double-quoted attributes."""
quote = ""
for index in range(start, len(text)):
char = text[index]
if quote:
if char == quote:
quote = ""
elif char in {"'", '"'}:
quote = char
elif char == ">":
return index
return -1
def _find_think_open(text: str, start: int) -> tuple[int, int] | None:
"""Find the next complete opening tag without retrying its suffix at each prefix."""
match = _THINK_OPEN_PREFIX_RE.search(text, start)
if match is None:
return None
end = text.find(">", match.end())
end = _find_tag_end(text, match.end())
if end < 0:
return None
return match.start(), end + 1
@ -99,7 +114,7 @@ def strip_leading_think_blocks(text: str) -> str:
opening = _THINK_OPEN_PREFIX_RE.match(text, start)
if opening is None:
break
end = text.find(">", opening.end())
end = _find_tag_end(text, opening.end())
if end >= 0 and _is_self_closing_think_tag(text, start, end + 1):
start = end + 1
continue

View File

@ -8,10 +8,28 @@ import pytest
from deerflow.utils.llm_text import (
extract_response_text,
strip_leading_think_blocks,
strip_markdown_code_fence,
strip_think_blocks,
)
@pytest.mark.parametrize("cleaner", [strip_think_blocks, strip_leading_think_blocks])
@pytest.mark.parametrize("tag", ['<think note=">"/>', "<think note='>'/>", '<THINK note="a > b" />'])
def test_self_closing_think_attributes_preserve_answer(cleaner, tag):
assert cleaner(tag + '{"answer":"yes"}') == '{"answer":"yes"}'
@pytest.mark.parametrize("cleaner", [strip_think_blocks, strip_leading_think_blocks])
def test_quoted_slash_angle_is_not_a_self_closing_tag(cleaner):
assert cleaner('<think note="/>">reasoning</think>answer') == "answer"
@pytest.mark.parametrize("cleaner", [strip_think_blocks, strip_leading_think_blocks])
def test_many_quoted_delimiters_in_empty_think_tag(cleaner):
assert cleaner('<think note="' + ">" * 100000 + '"/>answer') == "answer"
# ---------------------------------------------------------------------------
# strip_think_blocks
# ---------------------------------------------------------------------------