mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-27 15:06:23 +00:00
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zeren Wang
|
a58ab484a6
|
feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash (#5443)
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md (issue #5160, tracker #5129) in the slice order of the spec's §16. Slices: - A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY admission pinning (both server-owned sets + worker hoist); latest-only request-scoped <project> block via DynamicContextMiddleware wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID prefix + marker + provenance, never persisted); journal audit fingerprints; Instructions tab. - B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with locked check-and-set; hash-qualified immutable shelf storage with Paths helpers; upload/list/content/delete-to-trash routes; project delete trashes the shelf in-transaction; request-scoped bounded <documents> index with honest count/shown + actionable overflow note; list_project_documents/read_project_document tools registered only on pinned runs; PAT allowlist + drift guards; blocking-IO anchors. - C: shared thread-upload ingestion service (uploads router refactored to parity); POST from-thread with provenance; attach-to-thread with lock-staged copy (archived source allowed); read-only thread-files view with per-group truncation reporting. - D: restore (restored/merged/not_found/no_target/content_missing; no file moves), purge (continuous row lock across unlink/delete/commit, retryable on FS errors), retention sweep (lazy + startup, 24h orphan guard, row-side reconciliation never deletes). - E: Documents tab (shelf + conversation-files browser, provenance, archived banner, content-missing rows), /workspace/trash route, sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks + specs. Review hardening folded in (10 rounds, all with tests): - force active shelf content (HTML/XML family) to download; nosniff on artifact + content responses; unified unsandboxed-iframe PDF preview (fixes the pre-existing Chromium sandbox blank in the artifact viewer) - scope document trash to the URL project under the document lock - atomic no-overwrite filename reservation for ALL ingestion (seeded claims + os.link commit with suffix retry; same-name re-upload now unique-names instead of replacing); hidden staging only, no visible placeholders; lease cleanup on setup failure - serialize conversion under the document lock with post-lock active revalidation; drain locked filesystem work on cancellation; preserve bytes when an insert's commit state is uncertain (including trashed rows) - original-integrity checks before serving text or cached conversions; content_missing surfaced in list responses (UI reads the flag, no 409-probe); downloads always serve original bytes - bounded streaming document reads with cached char counts; shelf limits declared in middleware release identity - thread-root confinement for from-thread sources; config fallback rejects fractional/infinite values; composer counts staged attachments; pending attachments persist until submission or removal; in-flight instruction/rename edits survive save refetches; shelf and trash pagination; conversation-file and thread-files pages stay subscribed to refetches Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md contracts, config.example.yaml projects block. Review follow-ups (head b4807477 → this revision): - The trash retention sweep is split so repeated lazy triggers stay bounded: the indexed expiry purge still runs on every trigger (GET /api/trash/documents, POST /api/trash/purge) while the O(all rows + all files) reconciliation is throttled to one run per user per 15 minutes (process-local, per-user window). The startup sweep now runs as a background task instead of blocking gateway readiness, and shutdown awaits it (bounded). - The export scrub (stripInternalMarkers) is fence- and indentation-aware like the render path, so a pasted, fenced <project>/<documents> snippet survives markdown export while real injected blocks (never fenced) are still removed. Fence regexes moved to a dependency-free leaf module to avoid the messages↔streamdown import cycle. - The artifact viewer's PDF iframe no longer carries an added title attribute (the upstream e2e contract locates it via :not([title])), and the upstream artifact-preview spec now pins the new contract: PDFs render unsandboxed, images keep sandbox="". * fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title - Shutdown cancelled only the shield around the background startup sweep, so an all-users reconciliation that outlived the 5s budget kept walking rows and files while the document repo and DB engine were disposed underneath it. The wait now lives in `_shutdown_startup_trash_sweep`, which cancels the task and drains it before worker exit: the shield keeps the wait bounded, the cancel makes it final (CancelledError lands at the sweep's next await, and `_run_startup_trash_sweep` only catches `Exception`, so nothing swallows it). - The browser-preview iframe lost `title={getFileName(filepath)}` in the previous fix round, leaving the PDF frame without an accessible name while its siblings keep theirs. Restore it (WCAG frame titles), assert it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]` instead of `iframe:not([title])`. * fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel `Task.cancel()` returns False when the sweep already finished inside the window between the deadline firing and the cancel, so the shutdown log claimed a cancellation that never happened. Branch on that outcome: the warning stays for a real cancel, a late finish is logged at info, and both paths still reap the task before worker exit. * fix(projects): round-4 review — make Empty trash delete what it confirms `POST /api/trash/purge` only ran the retention sweep, and the sweep's candidate selection is age-gated, so a freshly trashed document survived "Empty trash" even though the confirmation promises that every listed document is permanently deleted. With one trashed row the route answered `{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps expired rows before listing, so the visible rows were normally ineligible for the action by construction. Empty trash now drives `purge_all_trashed`: the caller's trashed rows (`list_all_trashed`, no age filter) each go through the same guarded, row-locked `purge` as the single-document delete — bytes first, then the row, in one transaction — so a row restored mid-flight is skipped instead of force-deleted, and an unlink failure rolls that row back and answers 500 with a retryable message. Retention expiry stays where it was: the sweep's `purge_candidates` is now the only age-gated selection, and the lazy retention sweep still runs on the listing and at startup. Tests: the router suite replaces the retention-gated expectation with the reviewer's repro (fresh row purged, bytes unlinked, shelf and other users' trash untouched, a failing unlink stays retryable and 500); a blocking-I/O anchor drives the new entry point through the offload; the mocked e2e covers the action end to end; a new real-backend spec performs it against the real gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE and the phase-2 design docs (en+zh) state the age-independent contract. |
||
|
Beverly621
|
0464502af1
|
fix(frontend): keep human input cards with their turn (#4892)
* fix(frontend): keep human input cards with their turn * fix(frontend): keep human input cards with the correct turn Multi-turn ordering in restoreLocalTurnMessageOrder could place an `ask_clarification` (needYourHelp) card on the wrong side of a newly submitted human message, and after an interrupt/stop it could move the current run's own already-executed steps above the human that started them. - Restore established messages that a live checkpoint tail wove after the new human (displacedBaselineMessages). - Treat cards/messages confirmed only by the REST history page as established-past-turn too (confirmedHistoryIdentities), not as in-flight pending steps (displacedHistoryMessages). - Never displace the CURRENT run's own steps after an interrupt/stop; they belong after the human even once canonical history confirms them (currentTurnRunIds, anchored by the pending human's run_id). Fixes #4889 * fix(frontend): preserve ordering across displaced messages * fix(frontend): close canonical history ordering gaps * fix(frontend): preserve current turn anchor after compaction * fix(frontend): anchor the local turn on the submitted human identity Follow-up to #4892. R2 is reachable through the full hook chain: when the checkpoint baseline covers only the latest turn, the server echo of the submitted human confirms the optimistic copy against the unthrottled SDK state while the ~80ms render snapshot cannot show it yet; the baseline-only anchor scan then promoted an older history-only human into the current turn's anchor and moved established history behind it. - Record a LocalTurnAnchor at dispatch: one client-generated human id is shared by the optimistic display copy and the submitted message, so the server X__user echo confirms the exact identity already on screen. - restoreLocalTurnMessageOrder repairs only when that identity is present in the display; a null anchor (hidden human-input reply, regenerate replay) or a not-yet-rendered identity keeps established history untouched. - Optimistic confirmation now observes the same coalesced render snapshot (identity match first, rendered human-count growth as fallback for runtime-re-keyed first turns) instead of the per-chunk array. - Edit replays adopt the prepare response's replacement identity; the render ledger excludes unconfirmed optimistic copies by identity now that the local input no longer uses an opt- prefix, so a failed send cannot pin a message the server never saw. - Anchor lifecycle matches the previous baseline: kept across finish/stop/error until canonical data takes over, replaced by the next local submit, cleared on send failure, thread switch, and replay gaps. * test(threads): type submit mock calls in local-turn-order dom tests * fix(frontend): bound local turn repair to pre-submit history * fix(frontend): preserve pre-submit bridge ordering --------- Co-authored-by: 肘子香香 <hyh112300@163.com> Co-authored-by: 霍英豪 <huoyinghao250707@credithc.com> Co-authored-by: wangzeren <1004695029@qq.com> |
||
|
AnoobFeng
|
47b0f604f4
|
feat(frontend):enhance the ask_clarification interaction with visualized card (#3956)
* feat(frontend): add structured human input cards for ask_clarification Implement a reusable Human Input Card flow for ask_clarification while keeping the existing text fallback for older clients and IM channels. Backend: - Add structured ToolMessage.artifact.human_input payloads for clarification requests. - Preserve ToolMessage.content as the readable Markdown/text fallback. - Normalize clarification options from native lists, JSON strings, plain strings, mixed scalar values, None, and missing options. - Derive input_mode as choice_with_other when options exist, otherwise free_text. - Keep disable_clarification non-interactive behavior as a plain ToolMessage with no human_input artifact. - Cover artifact persistence and Gateway message metadata preservation in tests. Frontend: - Add human input protocol types, runtime guards, extractors, response builders, and thread-state helpers. - Add reusable HumanInputCard with option buttons, free-text input, pending, read-only, disabled, and answered states. - Render structured clarification cards from artifact.human_input, with Markdown fallback for malformed or legacy tool messages. - Preserve line breaks in structured question/context/option text. - Hide submitted clarification bridge messages from the chat UI via additional_kwargs.hide_from_ui. - Send structured human_input_response metadata through the fourth sendMessage options argument, preserving run context in the third argument. - Wire submissions for normal chats, custom agent chats, agent bootstrap chats, and sidecar chats. - Derive answered state from raw thread.messages so hidden replies still update the original card. - Clear pending state when the hidden reply arrives, dispatch is dropped, or a later async stream failure appears on thread.error. * perf(frontend): optimize HumanInputCard UI interactions - Support Enter key to submit text input (Shift+Enter for newline) - Render question and context fields as Markdown instead of plain text - Replace deprecated FormEventHandler type with structural typing * test(frontend): add unit test cover optimize HumanInputCard UI interactions * feat(frontend): disabled chatbox when has new human-input-card * fix(style): lint error fix * fix: sanitize hidden human input replies - Preserve IME composition safety for human input card Enter submits - Treat hidden human input responses as genuine user messages for sanitization - Keep hidden card replies in memory filtering while excluding malformed/internal hidden messages - Add regression coverage for card IME handling and hidden reply sanitization * fix: tighten human input response validation - Reject empty hidden human input response values - Remove invalid list ARIA role from human input card options - Add backend coverage for empty response payloads --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> |
||
|
Ryker_Feng
|
6a4e5a3bb2
|
feat(frontend): add side conversations for quoted follow-ups (#3934)
* feat(frontend): add side conversations for quoted follow-ups * style(frontend): apply prettier formatting to sidecar-chat files * fix(frontend): surface sidecar cascade cleanup failures via console.warn Previously deleteSidecarThreadsForParent silently swallowed both lookup errors and per-thread deletion failures, so parent thread deletions could succeed while orphaning sidecar threads with no signal to the caller. Log a warning that includes the parent id and the failed thread ids/reasons so the leak is discoverable in telemetry, matching the existing console.warn/error pattern in this file. * fix(frontend): address all sidecar review feedback Resolve every reviewer comment on PR #3934: - input-box/hooks/sidecar-panel: clear quoted references only via an `onSent` callback that fires after the in-flight guard, so a dropped send no longer silently discards quotes (willem-bd #3550). - message-list: flip the selection toolbar below the selection when it would clip above the viewport (willem-bd #3551). - reference-metadata/thread/input-box: keep referenced ids, roles, and count arrays 1:1 parallel instead of deduping ids (willem-bd #3552). - message-list: widen selection containment to the shared assistant-turn container and hint when a selection crosses messages (willem-bd #3553). - sidecar/api: coalesce concurrent sidecar creates for one parent behind a single in-flight promise to prevent duplicates (willem-bd #3554). - sidecar-trigger/context: force-restore on trigger click so a sidecar deleted elsewhere self-heals instead of opening a dead thread (willem-bd #3555). - threads/hooks: surface sidecar cascade cleanup failures via console.warn for both lookup and per-thread deletes (Copilot). Add unit + e2e coverage for parallel metadata, atomic create, and trigger self-healing. |