fix(test): exclude blocking I/O suite from make test (#5105)

* fix(test): exclude blocking I/O suite from make test

Keep make test-blocking-io as the dedicated suite owner.

Add regression coverage for the Makefile contract.

Refs #5088

* test: pin blocking I/O workflow ownership

Document both targets required for full offline validation.

Keep the dedicated workflow and Makefile target under contract coverage.

Refs #5088

* docs(test): align blocking-I/O test guidance
This commit is contained in:
luo jiyin 2026-08-30 22:05:24 +08:00 committed by GitHub
parent 8c8c5ac246
commit 3820515155
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
8 changed files with 36 additions and 14 deletions

View File

@ -146,7 +146,8 @@ Run `make help` for the full list.
```bash
# Backend (see backend/AGENTS.md for the full set)
cd backend && make dev # Gateway API with reload (port 8001)
cd backend && make test # Backend test suite
cd backend && make test # Default backend suite; excludes live and blocking-I/O tests
cd backend && make test-blocking-io # Strict blocking-I/O suite
cd backend && make lint # ruff check
cd backend && make format # ruff format

View File

@ -335,10 +335,13 @@ before review.
## Testing
```bash
# Backend tests (offline by default; excludes live external-API tests)
# Default backend tests (excludes live and blocking-I/O tests)
cd backend
make test
# Strict blocking-I/O tests
make test-blocking-io
# Live DeerFlowClient integration tests (explicit opt-in)
# Requires a valid root config.yaml and API credentials.
make test-live

View File

@ -1508,12 +1508,12 @@ immediately after starting any deployment that is not loopback-only.
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, workflow, and guidelines.
Backend `make test` is offline by default and excludes live external-API
coverage. Maintainers can explicitly run the real `DeerFlowClient` integration
suite with `cd backend && make test-live` after providing a valid root
`config.yaml` and API credentials; this may incur API costs and create local
sandboxes, artifacts, or files. Direct pytest runs additionally require
`DEER_FLOW_RUN_LIVE_TESTS=1`.
Backend `make test` excludes live external-API and blocking-I/O coverage.
Run `cd backend && make test-blocking-io` for strict blocking-I/O checks.
Maintainers can run the real `DeerFlowClient` suite with `cd backend && make test-live`.
This command requires a valid root `config.yaml` and API credentials.
It can incur API costs and create local sandboxes, artifacts, or files.
Direct pytest runs additionally require `DEER_FLOW_RUN_LIVE_TESTS=1`.
Regression coverage includes Docker sandbox mode detection and provisioner kubeconfig-path handling tests in `backend/tests/`.
Backend blocking-IO diagnostics are available from the repository root with

View File

@ -151,7 +151,7 @@ make stop # Stop all services
make install # Install backend dependencies
make dev # Run Gateway API with runtime-safe reload (port 8001)
make gateway # Run Gateway API only (port 8001)
make test # Run offline backend tests (excludes live external-API tests)
make test # Run offline backend tests (excludes live and blocking-I/O tests)
make test-live # Explicitly run live DeerFlowClient tests with real APIs
make test-blocking-io # Run strict Blockbuster runtime gate on tests/blocking_io/
make lint # Lint with ruff
@ -216,15 +216,18 @@ float filters accept integer or real JSON numbers through `json_value_matches`.
**Every new feature or bug fix MUST be accompanied by unit tests. No exceptions.**
- Write tests in `backend/tests/` following the existing naming convention `test_<feature>.py`
- Run the full offline suite before and after your change: `make test`
- Run both offline targets before and after your change: `make test` and `make test-blocking-io`
- Tests must pass before a feature is considered complete
- For lightweight config/utility modules, prefer pure unit tests with no external dependencies
- If a module causes circular import issues in tests, add a `sys.modules` mock in `tests/conftest.py` (see existing example for `deerflow.subagents.executor`)
```bash
# Run all offline tests
# Run default offline tests
make test
# Run strict blocking-I/O tests
make test-blocking-io
# Explicit live integration tests (requires config.yaml and credentials;
# calls real APIs and may create local side effects)
make test-live

View File

@ -20,7 +20,7 @@ gateway:
PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run --locked uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001
test:
PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run pytest -m "not live" tests/ -v
PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run pytest -m "not live" --ignore=tests/blocking_io tests/ -v
test-live:
DEER_FLOW_RUN_LIVE_TESTS=1 PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run pytest -m live tests/ -v -s

View File

@ -490,9 +490,12 @@ the only execution path, which keeps operational mistakes off the table. See
### Testing
```bash
# Offline backend suite (live external-API tests are excluded)
# Default offline backend suite (live external-API and blocking-I/O tests are excluded)
make test
# Strict blocking-I/O suite
make test-blocking-io
# Explicit real-API DeerFlowClient integration suite
make test-live
```

View File

@ -441,9 +441,12 @@ make migrate-rev MSG="add foo column to runs"
### 测试
```bash
# 离线后端测试套件(排除调用外部真实 API 的测试)
# 默认离线后端测试套件(排除调用外部真实 API 和阻塞式 I/O 的测试)
make test
# 严格的阻塞式 I/O 测试套件
make test-blocking-io
# 显式运行使用真实 API 的 DeerFlowClient 集成测试套件
make test-live
```

View File

@ -129,6 +129,7 @@ def test_make_targets_keep_default_tests_offline_and_support_live_opt_in() -> No
live_command = _dry_run_make_target("test-live")
assert 'pytest -m "not live"' in default_command
assert "--ignore=tests/blocking_io" in default_command
assert "tests/" in default_command
assert LIVE_OPT_IN not in default_command
@ -159,3 +160,11 @@ def test_default_ci_workflow_does_not_opt_in_to_live_tests() -> None:
assert "make test" in workflow
assert LIVE_OPT_IN not in workflow
def test_blocking_io_ci_workflow_owns_dedicated_suite() -> None:
command = _dry_run_make_target("test-blocking-io")
workflow = (REPO_ROOT / ".github" / "workflows" / "backend-blocking-io-tests.yml").read_text(encoding="utf-8")
assert "pytest tests/blocking_io" in command
assert "make test-blocking-io" in workflow