fix(models): preserve escaped newlines in MindIE XML JSON arguments (#6334)

* fix(models): preserve escaped newlines in MindIE XML JSON arguments

* fix(models): restore multiline raw MindIE tool arguments

README.md

@@ -144,6 +144,9 @@ It is disabled by default; see the linked guide to enable it.
 For Google's official Gemini OpenAI-compatible endpoint, use the
 [Gemini reasoning profile](backend/docs/CONFIGURATION.md#gemini-via-googles-openai-compatible-endpoint).
 
+For MindIE XML tool calls, see the
+[argument parsing and newline compatibility guide](backend/docs/CONFIGURATION.md#mindie-xml-tool-arguments).
+
 1. **Clone the DeerFlow repository**
 
    ```bash

backend/docs/CONFIGURATION.md

@@ -279,6 +279,26 @@ models:
           type: enabled
 ```
 
+#### MindIE XML tool arguments
+
+The MindIE adapter (`deerflow.models.mindie_provider:MindIEChatModel`) parses XML
+tool calls before decoding escaped newlines in the remaining reply text. JSON
+objects and arrays keep their original escapes during parsing, so `\n` inside a
+JSON string becomes a newline through JSON decoding and `\\n` retains a literal
+backslash. The existing Python-literal fallback also parses the original value.
+
+Non-JSON raw-string parameters retain the gateway's multiline compatibility:
+literal `\n` outside fenced code becomes a real newline, and surrounding
+whitespace is trimmed. Escapes inside fenced code remain unchanged. This keeps
+multi-line file content and commands working. Raw strings cannot distinguish an
+intended literal `\n` from a gateway-escaped newline; structured JSON parameters
+avoid that ambiguity. Numeric conversion failures and unsafe Python-literal
+containers retain the entire original argument rather than rewriting it.
+
+The same behavior applies to synchronous generation, asynchronous generation,
+and tool-enabled simulated streaming. Native tool-call arguments remain unchanged.
+No additional configuration is required.
+
 #### Gemini via Google's OpenAI-compatible endpoint
 
 When routing Gemini through an OpenAI-compatible proxy (Vertex AI OpenAI compat endpoint, AI Studio, or third-party gateways) with thinking enabled, the API attaches a `thought_signature` to each tool-call object returned in the response.  Every subsequent request that replays those assistant messages **must** echo those signatures back on the tool-call entries or the API returns:

backend/packages/harness/deerflow/models/AGENTS.md

@@ -1,3 +1,17 @@
+### MindIE XML tool arguments (`packages/harness/deerflow/models/mindie_provider.py`)
+
+Parse XML tool calls from the original model response before applying escaped-newline
+compatibility fixes to the remaining prose. Parse JSON and Python-literal arguments
+from their original escapes. For non-JSON raw-string parameters, decode literal
+`\n` outside fenced code and trim surrounding whitespace, retaining the gateway's
+existing multiline-file/command behavior. Raw strings cannot distinguish intended
+literal escapes from over-escaped newlines. Numeric conversion failures and unsafe
+Python-literal containers still retain the entire original argument. Keep the
+existing content path when no tool calls can be parsed and preserve native tool
+calls. This contract is shared by sync/async generation and tool-enabled simulated
+streaming; no-tool native streaming has its own chunk handling. Coverage:
+`tests/test_mindie_provider.py`.
+
 ### MindIE XML numeric arguments (`mindie_provider.py`)
 
 The XML parser recognizes JSON numeric syntax, including signed, fractional,

backend/packages/harness/deerflow/models/mindie_provider.py

@@ -158,13 +158,17 @@ def _parse_xml_tool_call_to_dict(content: str) -> tuple[str, list[dict]]:
                         if _safe_literal(candidate):
                             parsed_value = candidate
                     except (ValueError, SyntaxError):
-                        pass
+                        # Raw strings retain the gateway's multiline compatibility.
+                        # Structured arguments must be parsed before this decode.
+                        parsed_value = _decode_escaped_newlines_outside_fences(raw_value).strip()
                 except ValueError:
                     # Preserve the entire argument when JSON numeric conversion
                     # rejects overflow, underflow, or the integer digit limit.
                     # Retrying containers with literal_eval would turn nested
                     # underflowing numbers into zero and bypass this validation.
                     pass
+            else:
+                parsed_value = _decode_escaped_newlines_outside_fences(raw_value).strip()
 
             args[key] = parsed_value
 
@@ -248,17 +252,18 @@ class MindIEChatModel(ChatOpenAI):
             msg = gen.message
 
             if isinstance(msg.content, str):
-                # Keep escaped newlines inside fenced code blocks untouched.
-                msg.content = _decode_escaped_newlines_outside_fences(msg.content)
+                # Parse the original payload before display-only newline fixes:
+                # replacing escapes inside JSON can corrupt its syntax or values.
+                clean_content, extracted_tools = _parse_xml_tool_call_to_dict(msg.content)
 
-                if "<tool_call>" in msg.content:
-                    clean_content, extracted_tools = _parse_xml_tool_call_to_dict(msg.content)
-
-                    if extracted_tools:
-                        msg.content = clean_content
-                        if getattr(msg, "tool_calls", None) is None:
-                            msg.tool_calls = []
-                        msg.tool_calls.extend(extracted_tools)
+                if extracted_tools:
+                    msg.content = _decode_escaped_newlines_outside_fences(clean_content).strip()
+                    if getattr(msg, "tool_calls", None) is None:
+                        msg.tool_calls = []
+                    msg.tool_calls.extend(extracted_tools)
+                else:
+                    # Preserve unparseable XML and the existing prose/code behavior.
+                    msg.content = _decode_escaped_newlines_outside_fences(msg.content)
         return result
 
     def _generate(self, messages, stop=None, run_manager=None, **kwargs):

backend/tests/test_mindie_provider.py

@@ -2,6 +2,8 @@
 Unit tests for MindIEChatModel adapter.
 """
 
+import html
+import json
 from unittest.mock import AsyncMock, patch
 
 import pytest
@@ -410,6 +412,41 @@ class TestParseXmlToolCalls:
         _, calls = _parse_xml_tool_call_to_dict(content)
         assert calls[0]["args"]["bad"] == "{broken json"
 
+    @pytest.mark.parametrize(
+        ("raw_value", "expected"),
+        [
+            (r"first line\nsecond line", "first line\nsecond line"),
+            (r"echo first\necho second", "echo first\necho second"),
+            ("before\\n```python\nprint('a\\nb')\n```\\nafter", "before\n```python\nprint('a\\nb')\n```\nafter"),
+            (r"{broken json\nnext line", "{broken json\nnext line"),
+            ("first<&>\nsecond", "first<&>\nsecond"),
+        ],
+        ids=["multiline-file", "multiline-command", "fenced-code", "malformed-json", "existing-newline-and-entities"],
+    )
+    def test_raw_string_parameters_keep_multiline_compatibility(self, raw_value, expected):
+        encoded = html.escape(raw_value, quote=False)
+        content = f"<tool_call><function=write><parameter=content>{encoded}</parameter></function></tool_call>"
+
+        _, calls = _parse_xml_tool_call_to_dict(content)
+
+        assert calls[0]["args"]["content"] == expected
+
+    def test_python_literal_argument_preserves_its_own_escape_semantics(self):
+        raw_value = r"{'text': 'first\nsecond', 'literal': r'first\nsecond'}"
+        content = f"<tool_call><function=process><parameter=data>{raw_value}</parameter></function></tool_call>"
+
+        _, calls = _parse_xml_tool_call_to_dict(content)
+
+        assert calls[0]["args"]["data"] == {"text": "first\nsecond", "literal": r"first\nsecond"}
+
+    @pytest.mark.parametrize("raw_value", [r'{"n":1e400,"text":"first\nsecond"}', r"{'n':1e400,'text':'first\nsecond'}"])
+    def test_rejected_numeric_containers_keep_escaped_newlines(self, raw_value):
+        content = f"<tool_call><function=process><parameter=data>{raw_value}</parameter></function></tool_call>"
+
+        _, calls = _parse_xml_tool_call_to_dict(content)
+
+        assert calls[0]["args"]["data"] == raw_value
+
     def test_non_string_input_returned_as_is(self):
         result = _parse_xml_tool_call_to_dict(None)
         assert result == (None, [])
@@ -460,6 +497,99 @@ class TestPatchResult:
         assert len(msg.tool_calls) == 1
         assert msg.tool_calls[0]["name"] == "calc"
 
+    @pytest.mark.parametrize(
+        "value",
+        [
+            {"text": "line1\nline2"},
+            {"text": r"line1\nline2"},
+            {"items": ["line1\nline2", {"text": "<&>\nend"}]},
+            ["line1\nline2", r"literal\n"],
+        ],
+        ids=["object-newline", "literal-backslash", "nested-object", "array"],
+    )
+    def test_xml_json_arguments_preserve_escapes(self, value):
+        encoded = html.escape(json.dumps(value), quote=False)
+        content = f"before\\n<tool_call><function=process><parameter=data>{encoded}</parameter></function></tool_call>\\nafter"
+
+        msg = self._model()._patch_result_with_tools(_make_chat_result(content)).generations[0].message
+
+        assert msg.tool_calls[0]["args"]["data"] == value
+        assert msg.content == "before\n\nafter"
+
+    def test_multiple_xml_calls_preserve_payloads_and_native_calls(self):
+        value = {"text": "line1\nline2"}
+        raw_text = r"first line\nsecond line"
+        content = f"<tool_call><function=process><parameter=data>{json.dumps(value)}</parameter></function></tool_call><tool_call><function=write><parameter=content>{raw_text}</parameter></function></tool_call>"
+        native = {"name": "native", "args": {"text": r"a\nb"}, "id": "native-id"}
+
+        msg = self._model()._patch_result_with_tools(_make_chat_result(content, tool_calls=[native])).generations[0].message
+
+        assert msg.tool_calls[0] == native
+        assert msg.tool_calls[1]["args"] == {"data": value}
+        assert msg.tool_calls[2]["args"] == {"content": "first line\nsecond line"}
+        assert msg.content == ""
+
+    def test_unparseable_xml_keeps_content_and_newline_compatibility(self):
+        content = "before\\n<tool_call>not a function</tool_call>\\nafter"
+
+        msg = self._model()._patch_result_with_tools(_make_chat_result(content)).generations[0].message
+
+        assert msg.content == "before\n<tool_call>not a function</tool_call>\nafter"
+        assert msg.tool_calls == []
+
+    def test_xml_call_keeps_surrounding_escaped_whitespace_trimmed(self):
+        content = "\\n<tool_call><function=process><parameter=data>{}</parameter></function></tool_call>\\n"
+
+        msg = self._model()._patch_result_with_tools(_make_chat_result(content)).generations[0].message
+
+        assert msg.content == ""
+        assert msg.tool_calls[0]["args"]["data"] == {}
+
+    @pytest.mark.asyncio
+    @pytest.mark.parametrize("mode", ["sync", "async", "tool-stream"])
+    async def test_generation_paths_preserve_xml_json_arguments(self, mode):
+        value = {"text": "line1\nline2"}
+        content = f"<tool_call><function=process><parameter=data>{json.dumps(value)}</parameter></function></tool_call>"
+        result = _make_chat_result(content)
+        model = self._model()
+        messages = [HumanMessage(content="process the data")]
+
+        if mode == "sync":
+            with patch("deerflow.models.mindie_provider.ChatOpenAI._generate", return_value=result):
+                calls = model._generate(messages).generations[0].message.tool_calls
+        else:
+            with patch("deerflow.models.mindie_provider.ChatOpenAI._agenerate", new_callable=AsyncMock, return_value=result):
+                if mode == "async":
+                    calls = (await model._agenerate(messages)).generations[0].message.tool_calls
+                else:
+                    chunks = [chunk async for chunk in model._astream(messages, tools=[{"name": "process"}])]
+                    calls = [call for chunk in chunks for call in chunk.message.tool_calls]
+
+        assert len(calls) == 1
+        assert calls[0]["args"]["data"] == value
+
+    @pytest.mark.asyncio
+    @pytest.mark.parametrize("mode", ["sync", "async", "tool-stream"])
+    async def test_generation_paths_restore_multiline_raw_parameters(self, mode):
+        content = r"<tool_call><function=write><parameter=content>first line\nsecond line</parameter></function></tool_call>"
+        result = _make_chat_result(content)
+        model = self._model()
+        messages = [HumanMessage(content="write a two-line file")]
+
+        if mode == "sync":
+            with patch("deerflow.models.mindie_provider.ChatOpenAI._generate", return_value=result):
+                calls = model._generate(messages).generations[0].message.tool_calls
+        else:
+            with patch("deerflow.models.mindie_provider.ChatOpenAI._agenerate", new_callable=AsyncMock, return_value=result):
+                if mode == "async":
+                    calls = (await model._agenerate(messages)).generations[0].message.tool_calls
+                else:
+                    chunks = [chunk async for chunk in model._astream(messages, tools=[{"name": "write"}])]
+                    calls = [call for chunk in chunks for call in chunk.message.tool_calls]
+
+        assert len(calls) == 1
+        assert calls[0]["args"] == {"content": "first line\nsecond line"}
+
     def test_patch_result_appends_to_existing_tool_calls(self):
         model = self._model()
         existing = [{"name": "existing", "args": {}, "id": "e1"}]