mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-30 15:10:41 +00:00
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Hyeonsang Cho
|
bb4f7ac5e2
|
fix(browser): pin browser connections to the SSRF-vetted addresses (#6201)
* fix(browser): pin browser connections to the SSRF-vetted addresses The navigate screen and the per-request guard resolve a hostname to vet it, but Chromium resolved the name again to open the connection. A rebinding DNS server could answer both checks with a public address and the connection with a private or cloud-metadata one, and the browser followed it. Launch each private browser through a per-session loopback SOCKS5 proxy. Over SOCKS5 Chromium sends the hostname itself, so the proxy resolves it exactly once through the session's egress resolver and connects only to the addresses that resolver vetted. resolve_browser_egress applies the same allow_private_addresses policy as validate_browser_url, on top of a new url_safety.resolve_public_addresses helper that validate_public_http_url now shares. Loopback is proxied explicitly with bypass "<-loopback>". The tools, navigate_and_capture and the Live WebSocket all pass the resolver, because a thread's first caller fixes its launch options. CDP-attached Chrome is unchanged. Delegated fetch services (crawl4ai, Browserless, fastcrw) still resolve on their own side, which the Gateway cannot pin. * docs(changelog): reference #6201 in the browser egress pinning entry * fix(browser): bound egress proxy faults and close, and record the WebRTC gap Review follow-ups for the pinned browser egress proxy: - A resolver exception other than ValueError escaped the connection handler as an unretrieved task error. Log it with its traceback and reply SOCKS5 general failure (0x01), so a resolver fault stays distinguishable from a policy refusal (0x02). - Bound the resolve with its own timeout. The handshake timeout did not cover it, so a wedged getaddrinfo held the half-open tunnel until Chromium gave up. A timeout replies host unreachable. - Bound close(): Server.wait_closed() also waits for a connection whose handler registered after the writer sweep, which could stall session close and eviction for the 15 s handshake. A handler that starts after close now drops its connection without opening a tunnel. - Docs said a launched browser connects "solely" through the proxy. WebRTC UDP does not traverse a SOCKS proxy, so AGENTS.md, README (en/zh) and both changelogs now scope the guarantee to TCP and name the gap. |
||
|
Hyeonsang Cho
|
457aa3a492
|
fix(community): resolve SSRF-guarded hostnames off the event loop (#6140)
* test(blocking-io): catch socket.getaddrinfo on the event loop Blockbuster wraps socket.socket methods but not the module-level resolver, so a synchronous hostname lookup on the loop passes the strict gate. The SSRF guard in deerflow.community.url_safety resolves with socket.getaddrinfo from async web tools, and the anchors added next stay green against that unfixed code without this rule. Add socket.getaddrinfo to _PROJECT_BLOCKING_RULES, scoped to app.* and deerflow.* callers. loop.getaddrinfo and asyncio.to_thread resolve in a worker thread and are not flagged. The full make test-blocking-io suite stays green with the rule active (187 passed). * fix(community): resolve SSRF-guarded hostnames off the event loop validate_public_http_url resolves hostnames with blocking socket.getaddrinfo. The crawl4ai and Browserless web_fetch, web_capture, browser_navigate, the Gateway browser navigate route, and the Live stream's navigate input and seed all called it directly from async code, so a slow DNS answer for a model- or user-chosen URL stalled the Gateway event loop for the whole lookup. Run the guard through asyncio.to_thread at each of those call sites, the same way personal_network already offloads it. The screening result is unchanged. fastcrw's web_fetch is a sync tool and personal MCP validation already runs in a worker, so neither needed a change. * docs(changelog): reference #6140 in the SSRF guard event-loop entry * fix(browser): run the Playwright request guard off the shared browser loop BrowserSession._install_request_guard screens every redirect hop, iframe and subresource by calling the SSRF guard inside the async route callback. The guard resolves hostnames with blocking socket.getaddrinfo, and that callback runs on the process-wide Playwright loop, which also pumps Live frames and input for every browser session, so one slow lookup stalled all of them. Run the guard through asyncio.to_thread there as well, and pin it with a strict-gate anchor that drives the real route handler. Also note in the url_safety docstring that the screen is check-time only: a caller that re-resolves at connect time is not pinned to the vetted IPs. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
YxinMiracle
|
ce50a28dfd
|
fix(auth): enforce write permission for Live Browser WebSockets (#5621)
* fix(auth): enforce write permission for Live Browser WebSockets Resolve route permissions before accepting browser streams and require threads:write, matching the existing HTTP navigation endpoint. Preserve shared authorization failure semantics and reject unexpected setup errors before acquiring a browser session. Add authorization, frame delivery, input dispatch, cancellation, and ownership regressions. Document the admission-only permission check. * fix(auth): improve browser authorization diagnostics --------- Co-authored-by: YxinMiracle <“939157765@qq.com”> |
||
|
DanielWalnut
|
459dd78707
|
perf(frontend): bound delivery, bundles, and long-running UI work (#4622)
* docs: design frontend performance remediation * docs: plan frontend performance remediation * test(frontend): add route asset performance budgets * perf(nginx): compress textual responses safely * perf(frontend): lazy load case study media * perf(frontend): bound static demo file tracing * perf(frontend): restore static locale boundaries * perf(frontend): defer closed workspace panels * perf(frontend): split editors and deduplicate highlighting * perf(frontend): index incremental message derivation * perf(frontend): stabilize paged history cache policy * perf(frontend): bound streaming markdown renders * perf(frontend): virtualize message history * perf(frontend): bound and virtualize chat lists * perf(frontend): suspend inactive decorative animation * perf(browser): stream latest frames as binary * perf(artifacts): stream bounded text previews * docs: finalize performance runtime boundaries * style(backend): apply test formatting * fix(frontend): keep translation functions client-side * perf(frontend): defer decorative animation bundles * test(frontend): lock optimized route budgets * fix: harden frontend performance boundaries * test(frontend): update i18n provider fixture * fix(frontend): preserve sidebar pagination position * style(backend): format artifact range test |
||
|
Ryker_Feng
|
fa496c0c8d
|
feat(browser): add agentic browser control (#4187)
* feat(browser): add agentic browser control
* fix(frontend): format browser view changes
* fix(browser): keep browser optional and isolate sidecar layout
* fix(browser): address PR review security and IME findings
- Nginx: add a browser-stream WebSocket location before the generic
/api/threads regex so Live upgrades instead of downgrading to HTTP
(both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
navigate, and tear down the browser session on thread deletion so a
later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
context-level route guard covering redirects, popups, iframes, and
subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
CJK candidate with Enter no longer submits the remote page form.
Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.
* fix(frontend): smooth streaming in long tool threads
* Revert "fix(frontend): smooth streaming in long tool threads"
This reverts commit f0462516eabe77f138d4027ea1c714fb226683cf.
* fix(browser): address review security and lifecycle findings
- Reject cross-origin WebSocket upgrades on the live browser stream
(Origin allow-list reuse of CORS/same-origin helpers) to close a
WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
(browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
can't permanently stall after the cumulative attempt cap.
* fix(frontend): reduce long tool thread render stalls
Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.
* fix(browser): keep live control responsive during continuous input
Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").
What:
- Frontend forwards one `click` per physical click instead of also emitting
`down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
start a rate-limited background refresh loop (leading frame + bounded cadence)
that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
inputs coalesce, and continuous input keeps refreshing before it stops.
Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.
* fix(browser): harden worker and session lifecycle
* fix(browser): address latest review feedback
* fix(frontend): preserve optimistic new-chat message
* test(e2e): preserve mocked message run ids
* fix(browser): address capability review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
|