mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 21:26:39 +00:00
fix(tests): pass Windows base system env to the isolated live-policy pytest child (#6439)
* fix(tests): pass Windows base system env to isolated pytest child
The live-test policy regression tests spawn a pytest child with a minimal,
from-scratch env. On Windows that env lacks SystemRoot/TEMP/USERPROFILE, so
the child dies during import asyncio (WinError 10106, winsock provider init)
and tempfile cannot resolve a directory, making the tests fail for
environment reasons instead of testing the opt-in policy.
Extract the env assembly into _child_pytest_env() and pass through a small
allowlist of Windows base system variables (SystemRoot, windir, COMSPEC,
PATHEXT, TEMP, TMP, USERPROFILE) when present. These carry no credentials,
so the isolation contract (no API keys, no real .env) is unchanged, and
POSIX behavior is byte-identical. A new unit test pins the contract: only
the allowlisted base vars may pass through, and SystemRoot must be present
on Windows.
* test(live-policy): seed Windows base env vars in the bootability contract
The contract test asserted TEMP/TMP pass-through without seeding them, so
on Windows hosts that do not export those optional variables (CPython then
falls back to %USERPROFILE%\AppData\Local\Temp) it failed on host
configuration rather than on the helper's contract.
Seed the representative base variables with monkeypatch and assert the
forwarded set exactly, so the isolation contract is deterministic on every
host. Red/green evidence: with TEMP/TMP stripped from the environment the
old assertion fails (extra = {COMSPEC, PATHEXT, SystemRoot, USERPROFILE,
windir}) while the seeded assertion passes.
Addresses the P3 finding in the review of PR #6439.
* ci(tests): cover Windows live-test policy
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
parent
e1f20a6099
commit
829edac602
60
.github/workflows/backend-windows-live-policy.yml
vendored
Normal file
60
.github/workflows/backend-windows-live-policy.yml
vendored
Normal file
@ -0,0 +1,60 @@
|
||||
name: Windows Live Test Policy
|
||||
|
||||
# These offline policy tests spawn pytest children that collect live tests.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
- cron: '17 6 * * 1'
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened, ready_for_review]
|
||||
paths:
|
||||
- '.github/workflows/backend-windows-live-policy.yml'
|
||||
- 'backend/tests/test_client_live_policy.py'
|
||||
- 'backend/tests/test_client_live.py'
|
||||
- 'backend/pyproject.toml'
|
||||
- 'backend/uv.lock'
|
||||
|
||||
concurrency:
|
||||
group: windows-live-policy-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
windows-live-policy:
|
||||
if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
|
||||
runs-on: windows-latest
|
||||
timeout-minutes: 15
|
||||
defaults:
|
||||
run:
|
||||
working-directory: backend
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
# Must match backend/Dockerfile's UV_IMAGE tag.
|
||||
version: '0.11.1'
|
||||
|
||||
- name: Install backend dependencies
|
||||
run: uv sync --locked --group dev
|
||||
|
||||
- name: Check isolated child environment and live-test collection policy
|
||||
# Explicit nodes include all four collection cases and avoid make.
|
||||
run: >-
|
||||
uv run python -m pytest
|
||||
tests/test_client_live_policy.py::test_child_env_stays_isolated_but_bootable_on_windows
|
||||
tests/test_client_live_policy.py::test_config_alone_does_not_enable_live_collection
|
||||
tests/test_client_live_policy.py::test_explicit_opt_in_collects_live_tests
|
||||
tests/test_client_live_policy.py::test_ci_blocks_live_collection_even_with_opt_in
|
||||
tests/test_client_live_policy.py::test_opt_in_without_config_reports_missing_config
|
||||
-q
|
||||
@ -10,6 +10,13 @@ ownership and capture cleanup. Retain the real XLSX conversion coverage.
|
||||
|
||||
Backend tests must preserve the runtime invariants they exercise without changing production execution topology.
|
||||
|
||||
`.github/workflows/backend-windows-live-policy.yml` runs the child-env contract and all
|
||||
four live-test collection policy cases on Windows/Python 3.12, weekly, manually,
|
||||
and for relevant PR changes. These children only collect live tests. Keep the
|
||||
make-dependent checks in the default suite. The workflow-selection regression
|
||||
in `test_client_live_policy.py` collects the configured pytest nodes and pins
|
||||
all five policy checks, so a partial name filter cannot silently omit cases.
|
||||
|
||||
Runtime-callback cleanup tests keep DEBUG capture enabled and implement graph
|
||||
checkpoint reads. Fixture-only exception tracebacks can retain the journal;
|
||||
preserve the strict weak-reference assertion rather than clearing captured logs.
|
||||
|
||||
@ -19,6 +19,47 @@ REPRESENTATIVE_LIVE_NODE_IDS = (
|
||||
)
|
||||
|
||||
|
||||
# A Python child on Windows cannot boot from a scrubbed environment: without
|
||||
# SystemRoot, `import asyncio` dies with WinError 10106 (winsock provider init),
|
||||
# and tempfile cannot resolve a usable directory without TEMP/TMP/USERPROFILE.
|
||||
# These base system variables carry no credentials, so passing them through
|
||||
# keeps the isolation contract (no API keys, no real .env) intact while letting
|
||||
# the child start at all. POSIX behavior is unchanged.
|
||||
_WINDOWS_BASE_ENV_VARS = (
|
||||
"SystemRoot",
|
||||
"windir",
|
||||
"COMSPEC",
|
||||
"PATHEXT",
|
||||
"TEMP",
|
||||
"TMP",
|
||||
"USERPROFILE",
|
||||
)
|
||||
|
||||
|
||||
def _child_pytest_env(*, opt_in: bool, ci: bool) -> dict[str, str]:
|
||||
"""Build the minimal pytest child env without inheriting the real one."""
|
||||
env = {
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
"PYTHONDONTWRITEBYTECODE": "1",
|
||||
"PYTHONIOENCODING": "utf-8",
|
||||
"PYTHONUTF8": "1",
|
||||
"PYTHONPATH": os.pathsep.join(
|
||||
[
|
||||
str(BACKEND_ROOT),
|
||||
str(BACKEND_ROOT / "packages" / "harness"),
|
||||
]
|
||||
),
|
||||
"PYTEST_DISABLE_PLUGIN_AUTOLOAD": "1",
|
||||
}
|
||||
if opt_in:
|
||||
env[LIVE_OPT_IN] = "1"
|
||||
if ci:
|
||||
env["CI"] = "1"
|
||||
if os.name == "nt":
|
||||
env.update({name: os.environ[name] for name in _WINDOWS_BASE_ENV_VARS if name in os.environ})
|
||||
return env
|
||||
|
||||
|
||||
def _collect_live_tests(
|
||||
tmp_path: Path,
|
||||
*,
|
||||
@ -37,23 +78,7 @@ def _collect_live_tests(
|
||||
if config_exists:
|
||||
(temp_repo / "config.yaml").write_text("models: []\n", encoding="utf-8")
|
||||
|
||||
env = {
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
"PYTHONDONTWRITEBYTECODE": "1",
|
||||
"PYTHONIOENCODING": "utf-8",
|
||||
"PYTHONUTF8": "1",
|
||||
"PYTHONPATH": os.pathsep.join(
|
||||
[
|
||||
str(BACKEND_ROOT),
|
||||
str(BACKEND_ROOT / "packages" / "harness"),
|
||||
]
|
||||
),
|
||||
"PYTEST_DISABLE_PLUGIN_AUTOLOAD": "1",
|
||||
}
|
||||
if opt_in:
|
||||
env[LIVE_OPT_IN] = "1"
|
||||
if ci:
|
||||
env["CI"] = "1"
|
||||
env = _child_pytest_env(opt_in=opt_in, ci=ci)
|
||||
|
||||
return subprocess.run(
|
||||
[
|
||||
@ -124,6 +149,57 @@ def test_opt_in_without_config_reports_missing_config(tmp_path: Path) -> None:
|
||||
_assert_collection_skipped(result, "No config.yaml found")
|
||||
|
||||
|
||||
_BASE_ENV_KEYS = frozenset(
|
||||
{
|
||||
"PATH",
|
||||
"PYTHONDONTWRITEBYTECODE",
|
||||
"PYTHONIOENCODING",
|
||||
"PYTHONUTF8",
|
||||
"PYTHONPATH",
|
||||
"PYTEST_DISABLE_PLUGIN_AUTOLOAD",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Representative values for the Windows base variables. The contract test
|
||||
# below seeds them so its assertions do not depend on which optional
|
||||
# variables the host happens to export: a host without TEMP/TMP still lets
|
||||
# CPython fall back to %USERPROFILE%\AppData\Local\Temp, so an unseeded
|
||||
# assertion would fail on valid host configurations.
|
||||
_SEEDED_WINDOWS_BASE_ENV = {
|
||||
"SystemRoot": r"C:\Windows",
|
||||
"windir": r"C:\Windows",
|
||||
"COMSPEC": r"C:\Windows\System32\cmd.exe",
|
||||
"PATHEXT": ".COM;.EXE;.BAT;.CMD",
|
||||
"TEMP": r"C:\Windows\Temp",
|
||||
"TMP": r"C:\Windows\Temp",
|
||||
"USERPROFILE": r"C:\Users\runneradmin",
|
||||
}
|
||||
|
||||
|
||||
def test_child_env_stays_isolated_but_bootable_on_windows(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The scrubbed child env must not starve the Windows interpreter.
|
||||
|
||||
A child spawned with only the pytest-relevant variables fails on Windows
|
||||
during `import asyncio` (WinError 10106, missing SystemRoot) and cannot
|
||||
resolve a temp directory, so the live-policy tests below spuriously fail.
|
||||
"""
|
||||
if os.name == "nt":
|
||||
for name, value in _SEEDED_WINDOWS_BASE_ENV.items():
|
||||
monkeypatch.setenv(name, value)
|
||||
|
||||
env = _child_pytest_env(opt_in=True, ci=True)
|
||||
|
||||
extra = set(env) - _BASE_ENV_KEYS - {LIVE_OPT_IN, "CI"}
|
||||
if os.name == "nt":
|
||||
# Exactly the base system variables pass through — never credentials.
|
||||
# The seeded set covers the bootability-critical names: SystemRoot for
|
||||
# winsock init, TEMP/TMP/USERPROFILE for tempfile resolution.
|
||||
assert extra == set(_WINDOWS_BASE_ENV_VARS)
|
||||
else:
|
||||
assert not extra
|
||||
|
||||
|
||||
def test_make_targets_keep_default_tests_offline_and_support_live_opt_in() -> None:
|
||||
default_command = _dry_run_make_target("test")
|
||||
live_command = _dry_run_make_target("test-live")
|
||||
@ -208,3 +284,40 @@ def test_blocking_io_ci_workflow_owns_dedicated_suite() -> None:
|
||||
|
||||
assert "pytest tests/blocking_io" in command
|
||||
assert "make test-blocking-io" in workflow
|
||||
|
||||
|
||||
def test_windows_workflow_collects_all_five_policy_checks() -> None:
|
||||
"""Execute CI's selection so missing cases or accidental make tests fail."""
|
||||
import shlex
|
||||
|
||||
import yaml
|
||||
|
||||
workflow = yaml.safe_load((REPO_ROOT / ".github" / "workflows" / "backend-windows-live-policy.yml").read_text(encoding="utf-8"))
|
||||
job = workflow["jobs"]["windows-live-policy"]
|
||||
assert job["runs-on"] == "windows-latest"
|
||||
assert workflow["permissions"] == {"contents": "read"}
|
||||
assert job["defaults"]["run"]["working-directory"] == "backend"
|
||||
assert not job.get("services")
|
||||
step = next(step for step in job["steps"] if "python -m pytest" in step.get("run", ""))
|
||||
command = shlex.split(step["run"])
|
||||
assert command[:5] == ["uv", "run", "python", "-m", "pytest"]
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "pytest", *command[5:], "--collect-only", "-p", "no:cacheprovider"],
|
||||
cwd=BACKEND_ROOT,
|
||||
env=_child_pytest_env(opt_in=False, ci=True),
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
check=False,
|
||||
)
|
||||
output = result.stdout + result.stderr
|
||||
assert result.returncode == 0, output
|
||||
collected = {line.strip() for line in output.splitlines() if line.startswith("tests/test_client_live_policy.py::")}
|
||||
assert collected == {
|
||||
"tests/test_client_live_policy.py::test_child_env_stays_isolated_but_bootable_on_windows",
|
||||
"tests/test_client_live_policy.py::test_config_alone_does_not_enable_live_collection",
|
||||
"tests/test_client_live_policy.py::test_explicit_opt_in_collects_live_tests",
|
||||
"tests/test_client_live_policy.py::test_ci_blocks_live_collection_even_with_opt_in",
|
||||
"tests/test_client_live_policy.py::test_opt_in_without_config_reports_missing_config",
|
||||
}, output
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user