mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 21:26:39 +00:00
457aa3a492
* test(blocking-io): catch socket.getaddrinfo on the event loop Blockbuster wraps socket.socket methods but not the module-level resolver, so a synchronous hostname lookup on the loop passes the strict gate. The SSRF guard in deerflow.community.url_safety resolves with socket.getaddrinfo from async web tools, and the anchors added next stay green against that unfixed code without this rule. Add socket.getaddrinfo to _PROJECT_BLOCKING_RULES, scoped to app.* and deerflow.* callers. loop.getaddrinfo and asyncio.to_thread resolve in a worker thread and are not flagged. The full make test-blocking-io suite stays green with the rule active (187 passed). * fix(community): resolve SSRF-guarded hostnames off the event loop validate_public_http_url resolves hostnames with blocking socket.getaddrinfo. The crawl4ai and Browserless web_fetch, web_capture, browser_navigate, the Gateway browser navigate route, and the Live stream's navigate input and seed all called it directly from async code, so a slow DNS answer for a model- or user-chosen URL stalled the Gateway event loop for the whole lookup. Run the guard through asyncio.to_thread at each of those call sites, the same way personal_network already offloads it. The screening result is unchanged. fastcrw's web_fetch is a sync tool and personal MCP validation already runs in a worker, so neither needed a change. * docs(changelog): reference #6140 in the SSRF guard event-loop entry * fix(browser): run the Playwright request guard off the shared browser loop BrowserSession._install_request_guard screens every redirect hop, iframe and subresource by calling the SSRF guard inside the async route callback. The guard resolves hostnames with blocking socket.getaddrinfo, and that callback runs on the process-wide Playwright loop, which also pumps Live frames and input for every browser session, so one slow lookup stalled all of them. Run the guard through asyncio.to_thread there as well, and pin it with a strict-gate anchor that drives the real route handler. Also note in the url_safety docstring that the screen is check-time only: a caller that re-resolves at connect time is not pinned to the vetted IPs. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
49 lines
1.8 KiB
Python
49 lines
1.8 KiB
Python
"""Strict Blockbuster runtime context scoped to DeerFlow business code.
|
|
|
|
Creates a `BlockBuster` instance with `scanned_modules=("app", "deerflow")`
|
|
so that test infrastructure (pytest, langchain, importlib, third-party libs)
|
|
is out of scope and does not produce false positives. Only loop-blocking
|
|
sync IO whose caller stack passes through `app.*` or `deerflow.*` raises
|
|
`BlockingError`.
|
|
|
|
Used by `backend/tests/blocking_io/conftest.py` to gate the regression suite.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from collections.abc import Iterator
|
|
from contextlib import contextmanager
|
|
|
|
from blockbuster import BlockBuster, BlockBusterFunction, BlockingError
|
|
|
|
_SCANNED_MODULES: tuple[str, ...] = ("app", "deerflow")
|
|
|
|
# Add DeerFlow-local rules here only when Blockbuster's default rule set misses
|
|
# a generic blocking primitive used by production code. If a path is invisible
|
|
# because no test exercises it, add a production-path runtime anchor instead.
|
|
_PROJECT_BLOCKING_RULES: tuple[tuple[str, BlockBusterFunction], ...] = (
|
|
# Blockbuster wraps socket.socket methods but not the module-level resolver,
|
|
# and the SSRF guard in deerflow.community.url_safety resolves hostnames.
|
|
("socket.getaddrinfo", BlockBusterFunction(None, "socket.getaddrinfo", scanned_modules=list(_SCANNED_MODULES))),
|
|
)
|
|
|
|
|
|
def _install_project_rules(bb: BlockBuster) -> None:
|
|
for name, rule in _PROJECT_BLOCKING_RULES:
|
|
bb.functions[name] = rule
|
|
|
|
|
|
@contextmanager
|
|
def detect_blocking_io_strict() -> Iterator[BlockBuster]:
|
|
"""Activate Blockbuster scoped to app.* and deerflow.* callers only."""
|
|
bb = BlockBuster(scanned_modules=list(_SCANNED_MODULES))
|
|
_install_project_rules(bb)
|
|
try:
|
|
bb.activate()
|
|
yield bb
|
|
finally:
|
|
bb.deactivate()
|
|
|
|
|
|
__all__ = ["BlockingError", "detect_blocking_io_strict"]
|