mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
457aa3a492
* test(blocking-io): catch socket.getaddrinfo on the event loop Blockbuster wraps socket.socket methods but not the module-level resolver, so a synchronous hostname lookup on the loop passes the strict gate. The SSRF guard in deerflow.community.url_safety resolves with socket.getaddrinfo from async web tools, and the anchors added next stay green against that unfixed code without this rule. Add socket.getaddrinfo to _PROJECT_BLOCKING_RULES, scoped to app.* and deerflow.* callers. loop.getaddrinfo and asyncio.to_thread resolve in a worker thread and are not flagged. The full make test-blocking-io suite stays green with the rule active (187 passed). * fix(community): resolve SSRF-guarded hostnames off the event loop validate_public_http_url resolves hostnames with blocking socket.getaddrinfo. The crawl4ai and Browserless web_fetch, web_capture, browser_navigate, the Gateway browser navigate route, and the Live stream's navigate input and seed all called it directly from async code, so a slow DNS answer for a model- or user-chosen URL stalled the Gateway event loop for the whole lookup. Run the guard through asyncio.to_thread at each of those call sites, the same way personal_network already offloads it. The screening result is unchanged. fastcrw's web_fetch is a sync tool and personal MCP validation already runs in a worker, so neither needed a change. * docs(changelog): reference #6140 in the SSRF guard event-loop entry * fix(browser): run the Playwright request guard off the shared browser loop BrowserSession._install_request_guard screens every redirect hop, iframe and subresource by calling the SSRF guard inside the async route callback. The guard resolves hostnames with blocking socket.getaddrinfo, and that callback runs on the process-wide Playwright loop, which also pumps Live frames and input for every browser session, so one slow lookup stalled all of them. Run the guard through asyncio.to_thread there as well, and pin it with a strict-gate anchor that drives the real route handler. Also note in the url_safety docstring that the screen is check-time only: a caller that re-resolves at connect time is not pinned to the vetted IPs. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
65 lines
2.2 KiB
Python
65 lines
2.2 KiB
Python
"""Smoke test: the strict Blockbuster gate is wired up and actively catching.
|
|
|
|
Independent of any specific production code path, asserts that calling a
|
|
known blocking IO function directly from an `async def` (without an
|
|
`asyncio.to_thread` wrapper) raises `BlockingError`. If this test ever
|
|
stops raising, the gate machinery itself is broken — typical causes are
|
|
`scanned_modules` misconfiguration, accidental removal of the Blockbuster
|
|
dev dependency, or the conftest hookwrapper no longer firing.
|
|
|
|
This is the meta-test that protects every other test in this directory
|
|
from silent regressions (a green gate that no longer catches anything is
|
|
worse than no gate at all).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import os
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
from blockbuster import BlockingError
|
|
from support.detectors.blocking_io_runtime import detect_blocking_io_strict
|
|
|
|
pytestmark = pytest.mark.asyncio
|
|
|
|
|
|
async def test_gate_catches_unoffloaded_blocking_io_in_deerflow_module(tmp_path: Path) -> None:
|
|
from deerflow.runtime.store._sqlite_utils import ensure_sqlite_parent_dir
|
|
|
|
db_file = tmp_path / "subdir" / "store.db"
|
|
|
|
with pytest.raises(BlockingError):
|
|
ensure_sqlite_parent_dir(str(db_file))
|
|
|
|
|
|
async def test_gate_catches_hostname_resolution_on_the_loop() -> None:
|
|
# Blockbuster's default rules miss module-level socket.getaddrinfo; the
|
|
# project rule must catch a synchronous SSRF-guard lookup on the loop.
|
|
from deerflow.community.url_safety import resolve_host_addresses
|
|
|
|
with pytest.raises(BlockingError):
|
|
resolve_host_addresses("127.1")
|
|
|
|
|
|
async def test_gate_restores_blockbuster_patches_after_exceptions() -> None:
|
|
original_stat = os.stat
|
|
|
|
with pytest.raises(RuntimeError, match="boom"):
|
|
with detect_blocking_io_strict():
|
|
raise RuntimeError("boom")
|
|
|
|
assert os.stat is original_stat
|
|
|
|
|
|
@pytest.mark.allow_blocking_io
|
|
async def test_allow_blocking_io_marker_opts_out_of_gate(tmp_path: Path) -> None:
|
|
"""Verify the @pytest.mark.allow_blocking_io opt-out actually disables the gate."""
|
|
from deerflow.runtime.store._sqlite_utils import ensure_sqlite_parent_dir
|
|
|
|
db_file = tmp_path / "subdir" / "store.db"
|
|
|
|
ensure_sqlite_parent_dir(str(db_file))
|
|
|
|
assert db_file.parent.exists()
|