diff --git a/AGENTS.md b/AGENTS.md index 84dbadfa4..b42f1e50a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -146,7 +146,8 @@ Run `make help` for the full list. ```bash # Backend (see backend/AGENTS.md for the full set) cd backend && make dev # Gateway API with reload (port 8001) -cd backend && make test # Backend test suite +cd backend && make test # Default backend suite; excludes live and blocking-I/O tests +cd backend && make test-blocking-io # Strict blocking-I/O suite cd backend && make lint # ruff check cd backend && make format # ruff format diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index acb2a417e..04a36792a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -335,10 +335,13 @@ before review. ## Testing ```bash -# Backend tests (offline by default; excludes live external-API tests) +# Default backend tests (excludes live and blocking-I/O tests) cd backend make test +# Strict blocking-I/O tests +make test-blocking-io + # Live DeerFlowClient integration tests (explicit opt-in) # Requires a valid root config.yaml and API credentials. make test-live diff --git a/README.md b/README.md index 01ceac3cc..45df84a8c 100644 --- a/README.md +++ b/README.md @@ -1508,12 +1508,12 @@ immediately after starting any deployment that is not loopback-only. We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, workflow, and guidelines. -Backend `make test` is offline by default and excludes live external-API -coverage. Maintainers can explicitly run the real `DeerFlowClient` integration -suite with `cd backend && make test-live` after providing a valid root -`config.yaml` and API credentials; this may incur API costs and create local -sandboxes, artifacts, or files. Direct pytest runs additionally require -`DEER_FLOW_RUN_LIVE_TESTS=1`. +Backend `make test` excludes live external-API and blocking-I/O coverage. +Run `cd backend && make test-blocking-io` for strict blocking-I/O checks. +Maintainers can run the real `DeerFlowClient` suite with `cd backend && make test-live`. +This command requires a valid root `config.yaml` and API credentials. +It can incur API costs and create local sandboxes, artifacts, or files. +Direct pytest runs additionally require `DEER_FLOW_RUN_LIVE_TESTS=1`. Regression coverage includes Docker sandbox mode detection and provisioner kubeconfig-path handling tests in `backend/tests/`. Backend blocking-IO diagnostics are available from the repository root with diff --git a/backend/AGENTS.md b/backend/AGENTS.md index 4f650c3c6..69724fc99 100644 --- a/backend/AGENTS.md +++ b/backend/AGENTS.md @@ -151,7 +151,7 @@ make stop # Stop all services make install # Install backend dependencies make dev # Run Gateway API with runtime-safe reload (port 8001) make gateway # Run Gateway API only (port 8001) -make test # Run offline backend tests (excludes live external-API tests) +make test # Run offline backend tests (excludes live and blocking-I/O tests) make test-live # Explicitly run live DeerFlowClient tests with real APIs make test-blocking-io # Run strict Blockbuster runtime gate on tests/blocking_io/ make lint # Lint with ruff @@ -216,15 +216,18 @@ float filters accept integer or real JSON numbers through `json_value_matches`. **Every new feature or bug fix MUST be accompanied by unit tests. No exceptions.** - Write tests in `backend/tests/` following the existing naming convention `test_.py` -- Run the full offline suite before and after your change: `make test` +- Run both offline targets before and after your change: `make test` and `make test-blocking-io` - Tests must pass before a feature is considered complete - For lightweight config/utility modules, prefer pure unit tests with no external dependencies - If a module causes circular import issues in tests, add a `sys.modules` mock in `tests/conftest.py` (see existing example for `deerflow.subagents.executor`) ```bash -# Run all offline tests +# Run default offline tests make test +# Run strict blocking-I/O tests +make test-blocking-io + # Explicit live integration tests (requires config.yaml and credentials; # calls real APIs and may create local side effects) make test-live diff --git a/backend/Makefile b/backend/Makefile index 19aa66d00..4260d6d63 100644 --- a/backend/Makefile +++ b/backend/Makefile @@ -20,7 +20,7 @@ gateway: PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run --locked uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 test: - PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run pytest -m "not live" tests/ -v + PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run pytest -m "not live" --ignore=tests/blocking_io tests/ -v test-live: DEER_FLOW_RUN_LIVE_TESTS=1 PYTHONPATH=. PYTHONIOENCODING=utf-8 PYTHONUTF8=1 uv run pytest -m live tests/ -v -s diff --git a/backend/README.md b/backend/README.md index 07c260c1d..4b8999877 100644 --- a/backend/README.md +++ b/backend/README.md @@ -490,9 +490,12 @@ the only execution path, which keeps operational mistakes off the table. See ### Testing ```bash -# Offline backend suite (live external-API tests are excluded) +# Default offline backend suite (live external-API and blocking-I/O tests are excluded) make test +# Strict blocking-I/O suite +make test-blocking-io + # Explicit real-API DeerFlowClient integration suite make test-live ``` diff --git a/backend/README_zh.md b/backend/README_zh.md index 14dfa781a..26bcb3791 100644 --- a/backend/README_zh.md +++ b/backend/README_zh.md @@ -441,9 +441,12 @@ make migrate-rev MSG="add foo column to runs" ### 测试 ```bash -# 离线后端测试套件(排除调用外部真实 API 的测试) +# 默认离线后端测试套件(排除调用外部真实 API 和阻塞式 I/O 的测试) make test +# 严格的阻塞式 I/O 测试套件 +make test-blocking-io + # 显式运行使用真实 API 的 DeerFlowClient 集成测试套件 make test-live ``` diff --git a/backend/tests/test_client_live_policy.py b/backend/tests/test_client_live_policy.py index a63db634f..95bc858fb 100644 --- a/backend/tests/test_client_live_policy.py +++ b/backend/tests/test_client_live_policy.py @@ -129,6 +129,7 @@ def test_make_targets_keep_default_tests_offline_and_support_live_opt_in() -> No live_command = _dry_run_make_target("test-live") assert 'pytest -m "not live"' in default_command + assert "--ignore=tests/blocking_io" in default_command assert "tests/" in default_command assert LIVE_OPT_IN not in default_command @@ -159,3 +160,11 @@ def test_default_ci_workflow_does_not_opt_in_to_live_tests() -> None: assert "make test" in workflow assert LIVE_OPT_IN not in workflow + + +def test_blocking_io_ci_workflow_owns_dedicated_suite() -> None: + command = _dry_run_make_target("test-blocking-io") + workflow = (REPO_ROOT / ".github" / "workflows" / "backend-blocking-io-tests.yml").read_text(encoding="utf-8") + + assert "pytest tests/blocking_io" in command + assert "make test-blocking-io" in workflow