3 Commits

Author SHA1 Message Date
siye566
d8ca2c9358
test(community): make URL offload regressions DNS-independent (#6314)
* test(community): make URL offload regressions DNS-independent

* test(community): reject unsupported DNS fixture families

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 21:53:55 +08:00
Hyeonsang Cho
bb4f7ac5e2
fix(browser): pin browser connections to the SSRF-vetted addresses (#6201)
* fix(browser): pin browser connections to the SSRF-vetted addresses

The navigate screen and the per-request guard resolve a hostname to vet
it, but Chromium resolved the name again to open the connection. A
rebinding DNS server could answer both checks with a public address and
the connection with a private or cloud-metadata one, and the browser
followed it.

Launch each private browser through a per-session loopback SOCKS5 proxy.
Over SOCKS5 Chromium sends the hostname itself, so the proxy resolves it
exactly once through the session's egress resolver and connects only to
the addresses that resolver vetted. resolve_browser_egress applies the
same allow_private_addresses policy as validate_browser_url, on top of a
new url_safety.resolve_public_addresses helper that
validate_public_http_url now shares. Loopback is proxied explicitly with
bypass "<-loopback>". The tools, navigate_and_capture and the Live
WebSocket all pass the resolver, because a thread's first caller fixes
its launch options.

CDP-attached Chrome is unchanged. Delegated fetch services (crawl4ai,
Browserless, fastcrw) still resolve on their own side, which the Gateway
cannot pin.

* docs(changelog): reference #6201 in the browser egress pinning entry

* fix(browser): bound egress proxy faults and close, and record the WebRTC gap

Review follow-ups for the pinned browser egress proxy:

- A resolver exception other than ValueError escaped the connection
  handler as an unretrieved task error. Log it with its traceback and
  reply SOCKS5 general failure (0x01), so a resolver fault stays
  distinguishable from a policy refusal (0x02).
- Bound the resolve with its own timeout. The handshake timeout did not
  cover it, so a wedged getaddrinfo held the half-open tunnel until
  Chromium gave up. A timeout replies host unreachable.
- Bound close(): Server.wait_closed() also waits for a connection whose
  handler registered after the writer sweep, which could stall session
  close and eviction for the 15 s handshake. A handler that starts after
  close now drops its connection without opening a tunnel.
- Docs said a launched browser connects "solely" through the proxy.
  WebRTC UDP does not traverse a SOCKS proxy, so AGENTS.md, README (en/zh)
  and both changelogs now scope the guarantee to TCP and name the gap.
2026-10-03 10:05:47 +08:00
Hyeonsang Cho
457aa3a492
fix(community): resolve SSRF-guarded hostnames off the event loop (#6140)
* test(blocking-io): catch socket.getaddrinfo on the event loop

Blockbuster wraps socket.socket methods but not the module-level
resolver, so a synchronous hostname lookup on the loop passes the strict
gate. The SSRF guard in deerflow.community.url_safety resolves with
socket.getaddrinfo from async web tools, and the anchors added next stay
green against that unfixed code without this rule.

Add socket.getaddrinfo to _PROJECT_BLOCKING_RULES, scoped to app.* and
deerflow.* callers. loop.getaddrinfo and asyncio.to_thread resolve in a
worker thread and are not flagged. The full make test-blocking-io suite
stays green with the rule active (187 passed).

* fix(community): resolve SSRF-guarded hostnames off the event loop

validate_public_http_url resolves hostnames with blocking
socket.getaddrinfo. The crawl4ai and Browserless web_fetch, web_capture,
browser_navigate, the Gateway browser navigate route, and the Live
stream's navigate input and seed all called it directly from async code,
so a slow DNS answer for a model- or user-chosen URL stalled the Gateway
event loop for the whole lookup.

Run the guard through asyncio.to_thread at each of those call sites, the
same way personal_network already offloads it. The screening result is
unchanged. fastcrw's web_fetch is a sync tool and personal MCP validation
already runs in a worker, so neither needed a change.

* docs(changelog): reference #6140 in the SSRF guard event-loop entry

* fix(browser): run the Playwright request guard off the shared browser loop

BrowserSession._install_request_guard screens every redirect hop, iframe
and subresource by calling the SSRF guard inside the async route
callback. The guard resolves hostnames with blocking socket.getaddrinfo,
and that callback runs on the process-wide Playwright loop, which also
pumps Live frames and input for every browser session, so one slow
lookup stalled all of them.

Run the guard through asyncio.to_thread there as well, and pin it with a
strict-gate anchor that drives the real route handler. Also note in the
url_safety docstring that the screen is check-time only: a caller that
re-resolves at connect time is not pinned to the vetted IPs.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-02 22:33:48 +08:00