mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-25 06:08:50 +00:00
459dd78707
* docs: design frontend performance remediation * docs: plan frontend performance remediation * test(frontend): add route asset performance budgets * perf(nginx): compress textual responses safely * perf(frontend): lazy load case study media * perf(frontend): bound static demo file tracing * perf(frontend): restore static locale boundaries * perf(frontend): defer closed workspace panels * perf(frontend): split editors and deduplicate highlighting * perf(frontend): index incremental message derivation * perf(frontend): stabilize paged history cache policy * perf(frontend): bound streaming markdown renders * perf(frontend): virtualize message history * perf(frontend): bound and virtualize chat lists * perf(frontend): suspend inactive decorative animation * perf(browser): stream latest frames as binary * perf(artifacts): stream bounded text previews * docs: finalize performance runtime boundaries * style(backend): apply test formatting * fix(frontend): keep translation functions client-side * perf(frontend): defer decorative animation bundles * test(frontend): lock optimized route budgets * fix: harden frontend performance boundaries * test(frontend): update i18n provider fixture * fix(frontend): preserve sidebar pagination position * style(backend): format artifact range test
107 lines
8.1 KiB
Markdown
107 lines
8.1 KiB
Markdown
# Frontend Live Media and Artifact Performance Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Stop hidden animations, remove Browser Live base64/JSON/frame-state overhead, and make large text artifact preview bounded end to end.
|
|
|
|
**Architecture:** A shared visibility hook gates animation loops. Browser Live negotiates binary JPEG frames while retaining legacy JSON/base64 fallback. The browser client owns an object URL outside React render cadence and presents at most once per animation frame. Artifact text uses `FileResponse` range semantics and a 1 MiB preview contract with an explicit full-load action.
|
|
|
|
**Tech Stack:** React 19, WebSocket, FastAPI/Starlette, Playwright CDP, TypeScript/Rstest, Python/pytest.
|
|
|
|
**Global Constraints:** Read the relevant Browser Automation and Artifact sections of `backend/AGENTS.md` before edits. Preserve legacy Browser Live clients. Revoke every object URL. Preserve artifact download/content-disposition security and path ownership checks. Backend changes are test-first and must pass Ruff.
|
|
|
|
## Task 1: Pause decorative work when hidden or offscreen
|
|
|
|
**Files:**
|
|
- Create: `frontend/src/core/dom/use-render-activity.ts`
|
|
- Create: `frontend/tests/unit/core/dom/use-render-activity.dom.test.tsx`
|
|
- Modify: `frontend/src/components/ui/galaxy.jsx`
|
|
- Modify: `frontend/src/components/ui/magic-bento.tsx`
|
|
- Create: `frontend/tests/unit/components/ui/galaxy.dom.test.tsx`
|
|
- Create: `frontend/tests/unit/components/ui/magic-bento.dom.test.tsx`
|
|
|
|
- [ ] Write failing tests for `useRenderActivity(ref)`: false when document hidden or intersection false, true only when both visible, and observer/listener cleanup on unmount.
|
|
- [ ] Add component tests proving Galaxy cancels RAF while inactive and Magic Bento attaches pointer movement only to its container and coalesces work to one RAF.
|
|
- [ ] Run focused tests and capture RED.
|
|
- [ ] Implement the shared hook using `visibilitychange` plus `IntersectionObserver`. Gate Galaxy's RAF lifecycle. Replace Magic Bento's document-wide mousemove with container `pointermove` and one pending RAF.
|
|
- [ ] Run GREEN; disable the visibility gate to prove the RAF test RED, restore.
|
|
- [ ] Commit: `perf(frontend): suspend hidden landing effects`.
|
|
|
|
## Task 2: Negotiate binary Browser Live frames in the backend
|
|
|
|
**Files:**
|
|
- Modify: `backend/packages/harness/deerflow/community/browser_automation/session.py`
|
|
- Modify: `backend/app/gateway/routers/browser.py`
|
|
- Modify: `backend/tests/test_browser_automation.py`
|
|
- Modify: `backend/tests/test_browser_router.py`
|
|
- Modify: `backend/AGENTS.md`
|
|
|
|
- [ ] Add failing session tests asserting `_live_frame()` returns raw JPEG `bytes`, never base64 text.
|
|
- [ ] Add router tests for `?frame_format=binary`: capability is accepted, frame events use `websocket.send_bytes`, control/status events remain JSON, and a client without the query receives the legacy `{"type":"frame","data":"..."}` JSON payload.
|
|
- [ ] Run `cd backend && uv run pytest tests/test_browser_automation.py tests/test_browser_router.py -q` and capture RED.
|
|
- [ ] Change `frame_queue` to `asyncio.Queue[bytes]`; keep drop-oldest backpressure. Encode base64 only inside the legacy gateway send path. Reject unsupported capability values with a JSON error and close code 1008.
|
|
- [ ] Update `backend/AGENTS.md` with the wire contract and compatibility boundary.
|
|
- [ ] Run focused tests GREEN. Reintroduce session-layer base64 to prove the raw-byte assertion RED, restore.
|
|
- [ ] Commit: `perf(browser): stream negotiated binary live frames`.
|
|
|
|
## Task 3: Present Browser Live frames outside React state cadence
|
|
|
|
**Files:**
|
|
- Modify: `frontend/src/components/workspace/browser-view/use-browser-stream.ts`
|
|
- Create: `frontend/src/components/workspace/browser-view/frame-buffer.ts`
|
|
- Create: `frontend/tests/unit/components/workspace/browser-view/frame-buffer.dom.test.ts`
|
|
- Modify: `frontend/tests/e2e/browser-feature.spec.ts`
|
|
|
|
- [ ] Write failing tests for `FrameBuffer`: multiple binary frames before RAF expose only the latest frame, replaced/dropped URLs are revoked, close revokes the current URL, and legacy JSON/base64 still renders.
|
|
- [ ] Assert the WebSocket URL includes `frame_format=binary` and binary frames are not passed through `JSON.parse`.
|
|
- [ ] Run focused tests and capture RED.
|
|
- [ ] Set `binaryType="blob"`. Keep status/control in React state, but feed frame blobs to `FrameBuffer`, which owns one pending RAF and one object URL. Expose the current URL through `useSyncExternalStore` or an imperative image ref; do not allocate a data URL.
|
|
- [ ] Run tests and browser E2E GREEN. Revert to per-message state to prove the coalescing test RED, restore.
|
|
- [ ] Commit: `perf(browser): coalesce binary frame presentation`.
|
|
|
|
## Task 4: Serve text artifacts with byte ranges
|
|
|
|
**Files:**
|
|
- Modify: `backend/app/gateway/routers/artifacts.py`
|
|
- Modify: `backend/tests/test_artifacts_router.py`
|
|
- Modify: `backend/tests/blocking_io/test_artifacts_router.py`
|
|
|
|
- [ ] Add failing API tests for a UTF-8 text file: full GET remains inline with the correct media type, `Range: bytes=0-1048575` returns 206/`Content-Range`, invalid ranges return 416, and active HTML/SVG content remains forced-download.
|
|
- [ ] Add a blocking-I/O regression asserting the async route does not call `Path.read_text` or `Path.read_bytes` for normal text preview.
|
|
- [ ] Run focused backend tests and capture RED.
|
|
- [ ] Use Starlette `FileResponse` for safe text and binary inline responses, passing the detected textual media type and existing security headers. Retain attachment handling for active content and explicit downloads.
|
|
- [ ] Run focused tests GREEN. Revert safe text to `PlainTextResponse`, prove range/blocking tests RED, restore.
|
|
- [ ] Commit: `perf(artifacts): stream ranged text responses`.
|
|
|
|
## Task 5: Bound the client artifact preview to 1 MiB
|
|
|
|
**Files:**
|
|
- Modify: `frontend/src/core/artifacts/loader.ts`
|
|
- Modify: `frontend/src/core/artifacts/preview.ts`
|
|
- Modify: `frontend/src/components/workspace/artifacts/artifact-file-detail.tsx`
|
|
- Create: `frontend/tests/unit/core/artifacts/loader.test.ts`
|
|
- Modify: `frontend/tests/unit/core/artifacts/preview.test.ts`
|
|
- Modify: `frontend/tests/e2e/artifact-preview.spec.ts`
|
|
|
|
- [ ] Write failing tests that require `Range: bytes=0-1048575`, parse status 206 and `Content-Range`, mark the result `{ truncated: true, totalBytes }`, and preserve full/small responses.
|
|
- [ ] Add a DOM/E2E assertion that truncated text shows byte counts plus an explicit “Load full file” action, and CodeMirror is not mounted before that action.
|
|
- [ ] Run focused tests and capture RED.
|
|
- [ ] Add `ARTIFACT_PREVIEW_MAX_BYTES = 1_048_576`. Decode only the returned prefix, display a truncation banner, and refetch without Range only after explicit consent. Plain/Markdown preview remains available for the prefix; editable CodeMirror requires full content.
|
|
- [ ] Run tests GREEN. Remove the Range header temporarily to prove RED, restore.
|
|
- [ ] Commit: `perf(artifacts): bound large text previews`.
|
|
|
|
## Task 6: Documentation and cross-stack verification
|
|
|
|
**Files:**
|
|
- Modify: `README.md`
|
|
- Modify: `frontend/AGENTS.md`
|
|
- Modify: `backend/AGENTS.md`
|
|
- Modify: `CHANGELOG.md`
|
|
|
|
- [ ] Document Browser Live binary negotiation/fallback, the 1 MiB artifact preview behavior, and performance verification commands.
|
|
- [ ] Run `cd backend && make format && make lint && uv run pytest tests/test_browser_automation.py tests/test_browser_router.py tests/test_artifacts_router.py tests/blocking_io/test_artifacts_router.py -q`.
|
|
- [ ] Run `cd frontend && pnpm check && pnpm test && pnpm test:e2e -- tests/e2e/browser-feature.spec.ts tests/e2e/artifact-preview.spec.ts`.
|
|
- [ ] Run the full root-supported checks that are practical locally, then `cd frontend && NEXT_PUBLIC_STATIC_WEBSITE_ONLY=true pnpm build && pnpm perf:check`.
|
|
- [ ] Use browser DevTools to verify binary websocket frames, bounded frame presentation, object URL cleanup after panel close, a 206 artifact preview, and an explicit full-load request.
|
|
- [ ] Commit: `docs: document frontend performance boundaries`.
|