deer-flow/docs/superpowers/plans/2026-07-31-frontend-performance-live-artifacts.md

Frontend Live Media and Artifact Performance Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Stop hidden animations, remove Browser Live base64/JSON/frame-state overhead, and make large text artifact preview bounded end to end.

Architecture: A shared visibility hook gates animation loops. Browser Live negotiates binary JPEG frames while retaining legacy JSON/base64 fallback. The browser client owns an object URL outside React render cadence and presents at most once per animation frame. Artifact text uses FileResponse range semantics and a 1 MiB preview contract with an explicit full-load action.

Tech Stack: React 19, WebSocket, FastAPI/Starlette, Playwright CDP, TypeScript/Rstest, Python/pytest.

Global Constraints: Read the relevant Browser Automation and Artifact sections of backend/AGENTS.md before edits. Preserve legacy Browser Live clients. Revoke every object URL. Preserve artifact download/content-disposition security and path ownership checks. Backend changes are test-first and must pass Ruff.

Task 1: Pause decorative work when hidden or offscreen

Files:

• Create: frontend/src/core/dom/use-render-activity.ts

• Create: frontend/tests/unit/core/dom/use-render-activity.dom.test.tsx

• Modify: frontend/src/components/ui/galaxy.jsx

• Modify: frontend/src/components/ui/magic-bento.tsx

• Create: frontend/tests/unit/components/ui/galaxy.dom.test.tsx

• Create: frontend/tests/unit/components/ui/magic-bento.dom.test.tsx

• Write failing tests for useRenderActivity(ref): false when document hidden or intersection false, true only when both visible, and observer/listener cleanup on unmount.

• Add component tests proving Galaxy cancels RAF while inactive and Magic Bento attaches pointer movement only to its container and coalesces work to one RAF.

• Run focused tests and capture RED.

• Implement the shared hook using visibilitychange plus IntersectionObserver. Gate Galaxy's RAF lifecycle. Replace Magic Bento's document-wide mousemove with container pointermove and one pending RAF.

• Run GREEN; disable the visibility gate to prove the RAF test RED, restore.

• Commit: perf(frontend): suspend hidden landing effects.

Task 2: Negotiate binary Browser Live frames in the backend

Files:

• Modify: backend/packages/harness/deerflow/community/browser_automation/session.py

• Modify: backend/app/gateway/routers/browser.py

• Modify: backend/tests/test_browser_automation.py

• Modify: backend/tests/test_browser_router.py

• Modify: backend/AGENTS.md

• Add failing session tests asserting _live_frame() returns raw JPEG bytes, never base64 text.

• Add router tests for ?frame_format=binary: capability is accepted, frame events use websocket.send_bytes, control/status events remain JSON, and a client without the query receives the legacy {"type":"frame","data":"..."} JSON payload.

• Run cd backend && uv run pytest tests/test_browser_automation.py tests/test_browser_router.py -q and capture RED.

• Change frame_queue to asyncio.Queue[bytes]; keep drop-oldest backpressure. Encode base64 only inside the legacy gateway send path. Reject unsupported capability values with a JSON error and close code 1008.

• Update backend/AGENTS.md with the wire contract and compatibility boundary.

• Run focused tests GREEN. Reintroduce session-layer base64 to prove the raw-byte assertion RED, restore.

• Commit: perf(browser): stream negotiated binary live frames.

Task 3: Present Browser Live frames outside React state cadence

Files:

• Modify: frontend/src/components/workspace/browser-view/use-browser-stream.ts

• Create: frontend/src/components/workspace/browser-view/frame-buffer.ts

• Create: frontend/tests/unit/components/workspace/browser-view/frame-buffer.dom.test.ts

• Modify: frontend/tests/e2e/browser-feature.spec.ts

• Write failing tests for FrameBuffer: multiple binary frames before RAF expose only the latest frame, replaced/dropped URLs are revoked, close revokes the current URL, and legacy JSON/base64 still renders.

• Assert the WebSocket URL includes frame_format=binary and binary frames are not passed through JSON.parse.

• Run focused tests and capture RED.

• Set binaryType="blob". Keep status/control in React state, but feed frame blobs to FrameBuffer, which owns one pending RAF and one object URL. Expose the current URL through useSyncExternalStore or an imperative image ref; do not allocate a data URL.

• Run tests and browser E2E GREEN. Revert to per-message state to prove the coalescing test RED, restore.

• Commit: perf(browser): coalesce binary frame presentation.

Task 4: Serve text artifacts with byte ranges

Files:

• Modify: backend/app/gateway/routers/artifacts.py

• Modify: backend/tests/test_artifacts_router.py

• Modify: backend/tests/blocking_io/test_artifacts_router.py

• Add failing API tests for a UTF-8 text file: full GET remains inline with the correct media type, Range: bytes=0-1048575 returns 206/Content-Range, invalid ranges return 416, and active HTML/SVG content remains forced-download.

• Add a blocking-I/O regression asserting the async route does not call Path.read_text or Path.read_bytes for normal text preview.

• Run focused backend tests and capture RED.

• Use Starlette FileResponse for safe text and binary inline responses, passing the detected textual media type and existing security headers. Retain attachment handling for active content and explicit downloads.

• Run focused tests GREEN. Revert safe text to PlainTextResponse, prove range/blocking tests RED, restore.

• Commit: perf(artifacts): stream ranged text responses.

Task 5: Bound the client artifact preview to 1 MiB

Files:

• Modify: frontend/src/core/artifacts/loader.ts

• Modify: frontend/src/core/artifacts/preview.ts

• Modify: frontend/src/components/workspace/artifacts/artifact-file-detail.tsx

• Create: frontend/tests/unit/core/artifacts/loader.test.ts

• Modify: frontend/tests/unit/core/artifacts/preview.test.ts

• Modify: frontend/tests/e2e/artifact-preview.spec.ts

• Write failing tests that require Range: bytes=0-1048575, parse status 206 and Content-Range, mark the result { truncated: true, totalBytes }, and preserve full/small responses.

• Add a DOM/E2E assertion that truncated text shows byte counts plus an explicit “Load full file” action, and CodeMirror is not mounted before that action.

• Run focused tests and capture RED.

• Add ARTIFACT_PREVIEW_MAX_BYTES = 1_048_576. Decode only the returned prefix, display a truncation banner, and refetch without Range only after explicit consent. Plain/Markdown preview remains available for the prefix; editable CodeMirror requires full content.

• Run tests GREEN. Remove the Range header temporarily to prove RED, restore.

• Commit: perf(artifacts): bound large text previews.

Task 6: Documentation and cross-stack verification

Files:

• Modify: README.md

• Modify: frontend/AGENTS.md

• Modify: backend/AGENTS.md

• Modify: CHANGELOG.md

• Document Browser Live binary negotiation/fallback, the 1 MiB artifact preview behavior, and performance verification commands.

• Run cd backend && make format && make lint && uv run pytest tests/test_browser_automation.py tests/test_browser_router.py tests/test_artifacts_router.py tests/blocking_io/test_artifacts_router.py -q.

• Run cd frontend && pnpm check && pnpm test && pnpm test:e2e -- tests/e2e/browser-feature.spec.ts tests/e2e/artifact-preview.spec.ts.

• Run the full root-supported checks that are practical locally, then cd frontend && NEXT_PUBLIC_STATIC_WEBSITE_ONLY=true pnpm build && pnpm perf:check.

• Use browser DevTools to verify binary websocket frames, bounded frame presentation, object URL cleanup after panel close, a 206 artifact preview, and an explicit full-load request.

• Commit: docs: document frontend performance boundaries.