Jason Cat cee0d93b60
fix(sandbox): decode Git Bash output as UTF-8 so CJK output is not garbled (#6548)
* fix(sandbox): decode Git Bash output as UTF-8 so CJK tools are not garbled

Git Bash/MSYS always writes UTF-8 to its pipes, but the MSYS branch of
LocalSandbox.execute_command called _run_windows_command without an
encoding, so the capture fell back to locale.getpreferredencoding(). On a
non-UTF-8 host (GBK on zh-CN Windows) that decodes UTF-8 bytes with the host
code page and mangles every non-ASCII tool result. The pipe decoder replaces
instead of raising, so the corruption is silent. The PowerShell branch right
next to it has pinned UTF-8 since #5440.

Decode the MSYS branch as UTF-8 too, and pin the behaviour in
tests/test_local_sandbox_encoding.py: a unit test for the decoder choice and
a live round-trip through a real Git Bash (skipped when it is absent).

* fix(sandbox): keep locale fallback for native-child output under Git Bash

Git Bash/MSYS writes its own output as UTF-8, but Windows-native children
(python.exe, CLI shims) inherit the pipe and keep writing the host code
page. Forcing UTF-8 with errors=replace turned that output into U+FFFD.

Decode the captured buffer as UTF-8 first and retry with the locale when it
is not valid UTF-8, so both MSYS UTF-8 output and native-child code-page
output survive. The PowerShell branch is unchanged (its own output is pinned
to UTF-8 by the preamble); the fallback is only wired on the MSYS path.

Adds native-Python CJK regressions through execute_command.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-09 17:25:35 +08:00
..