Skip to content

Commit 400d7a1

Browse files
committed
test(turn-driver): run the locale evidence child from stdin
Both cases passed the child program as a `-c` argument. CPython decodes `-c` text with the filesystem encoding, which this fixture forces to ASCII (`LC_ALL=C`, `-X utf8=0`, `PYTHONCOERCECLOCALE=0`), so on Linux the child died before importing LoopX: `Unable to decode the command from the command line` / `UnicodeEncodeError: ... surrogates not allowed`. The Windows lane passed only because its filesystem encoding is UTF-8. The program now travels on stdin, which CPython decodes as UTF-8 and which keeps the working directory on `sys.path` the way `-c` does. The workspace is the pytest temporary directory rather than a path with a non-ASCII component: the child addresses it as `str` under that ASCII filesystem encoding, so such a path would fail on the surrogate escape before the codec hazard this suite exists to reproduce. The non-ASCII payload stays in the CLI's UTF-8 event bytes and in the attempt record, so the codec assertions are unchanged. Reported by the independent review of a786ed7. Reproduced on Linux Python 3.12: a786ed7 fails both cases at the command-line decode; this head passes both, and reverting the observe codec pin still fails both with `UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 36`. Signed-off-by: JasonBuildAI <jasonbuildai@gmail.com>
1 parent a786ed7 commit 400d7a1

1 file changed

Lines changed: 20 additions & 8 deletions

File tree

‎tests/control_plane/test_codex_session_utf8_evidence.py‎

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,18 @@
88
99
Each case drives one real owner in a child interpreter whose default encoding is
1010
not UTF-8 (the host ANSI code page on Windows, `ascii` under `LC_ALL=C` on
11-
POSIX), with a workspace whose path carries a non-ASCII component: the refresh
12-
record keeps the observed stdout path and the binding's checkpoint root, so that
13-
workspace shape is the ordinary case rather than a synthetic one.
11+
POSIX). The child program and the workspace it addresses stay ASCII, because the
12+
forced filesystem encoding must be able to decode the source path it is started
13+
from and the `str` paths the owner hands to file I/O; the non-ASCII payload is
14+
the CLI's own UTF-8 event bytes, which is what the codec under test decodes. A
15+
non-ASCII workspace path would fail on the surrogate escape first and hide the
16+
codec hazard this suite exists to reproduce.
17+
18+
The program is transported on stdin rather than as a `-c` argument: `-c` text is
19+
decoded with the filesystem encoding, so under `LC_ALL=C` CPython rejects the
20+
program before it imports LoopX ("Unable to decode the command from the command
21+
line"), while source read from stdin is decoded as UTF-8. `-` keeps the working
22+
directory on `sys.path` the way `-c` does.
1423
1524
The child also proves the case is not vacuous: it decodes the exact bytes the
1625
CLI writes with its resolved default codec and reports whether they are
@@ -31,8 +40,6 @@
3140

3241
REPO_ROOT = Path(__file__).resolve().parents[2]
3342

34-
WORKSPACE_COMPONENT = "项目"
35-
3643
_CHILD = r'''
3744
import json, locale, sys
3845
from pathlib import Path
@@ -128,9 +135,14 @@ def _drive(base: Path, mode: str) -> subprocess.CompletedProcess[bytes]:
128135
"PYTHONUTF8": "0",
129136
}
130137
return subprocess.run(
131-
[sys.executable, "-X", "utf8=0", "-c", _CHILD, str(base), mode],
138+
[sys.executable, "-X", "utf8=0", "-", str(base), mode],
132139
cwd=REPO_ROOT,
133140
env=environment,
141+
# `-c` arguments are decoded with the ASCII filesystem encoding this
142+
# environment forces, which rejects a non-ASCII program before LoopX is
143+
# imported; stdin source is decoded as UTF-8 and keeps the working
144+
# directory on `sys.path`, as `-c` would.
145+
input=_CHILD.encode("utf-8"),
134146
capture_output=True,
135147
timeout=180,
136148
)
@@ -161,7 +173,7 @@ def test_codex_events_are_read_back_with_a_locale_codec(tmp_path: Path) -> None:
161173
makes this case fail with the reproduced `UnicodeDecodeError`.
162174
"""
163175

164-
base = tmp_path / WORKSPACE_COMPONENT
176+
base = tmp_path
165177
payload = _payload(base, "observe")
166178

167179
assert payload["session_id"] == "fresh"
@@ -178,7 +190,7 @@ def test_checkpoint_refresh_attempt_record_is_readable_again(tmp_path: Path) ->
178190
`tests/test_loopx_text_io_utf8.py` is what protects them.
179191
"""
180192

181-
base = tmp_path / WORKSPACE_COMPONENT
193+
base = tmp_path
182194
payload = _payload(base, "recover")
183195

184196
assert payload["session_id"] == "resumed"

0 commit comments

Comments
 (0)