* fix(frontend): read web_fetch titles that start with blank lines or indented headings
* docs(frontend): describe the indented-code guard as it actually behaves
* fix(frontend): reject mixed code indentation in web-fetch titles
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(authz): gate composer send on runs:create (Phase 4 follow-up, #4063)
Mirrors #5294's stop gating on the send side: both chat routes pass
canCreateRuns (from PERMISSIONS.RUNS_CREATE, lockstep with the backend
enum) into the shared composer. The gate sits at the top of
submitThreadMessage — the single choke point every composer entry
(submit button, Enter, goal-set-triggered run) funnels through — and
denies with a toast plus a rejected promise so PromptInput keeps the
text. The idle submit button is disabled and explains the boundary via
conditionally-spread aria-label/title (startTurnUnavailable, en/zh);
while streaming the button stays the runs:cancel stop affordance.
Also removes the unreachable kind === "stop" branch in handleSubmit
(flagged during #5294's review): the Enter path early-returns with the
streaming toast before the classifier runs.
* fix(frontend): reject denied goal starts before saving state
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): add WeChat QR login and binding recovery
* fix(channels): enforce single-worker WeChat QR login and preserve bot ID
Reject QR login endpoints when multiple Gateway workers are configured, while keeping manual token setup available.
Preserve the configured bot ID when the provider omits it or returns an empty value. Add regression coverage for worker guards and credential persistence.
* fix(channels): sync WeChat completion state on provider updates
Show the connected step when refreshed provider data confirms the binding, so the dialog no longer waits indefinitely after polling is cancelled.
Add regression tests for provider updates during pending poll and binding requests, including late responses and expiry.
* fix(channels): preserve WeChat pairing codes across waits and redirects
---------
Co-authored-by: YxinMiracle <“939157765@qq.com”>
* fix(frontend): rejoin active runs after reopening chats
* test(frontend): mock thread runs query in stream options test
* fix(frontend): avoid rejoining completed runs from stale cache
* fix(frontend): tighten active run recovery cleanup
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(knowledge): add verifiable RAGFlow source citations
* docs(knowledge): scope RAGFlow guidance to its own directory
* fix(knowledge): preserve citations through rendering and budgets
* fix(frontend): gate tool-step links through the href scheme allowlist
The chain-of-thought renderer turned web_fetch args and web_search /
image_search result URLs straight into <a href>. Markdown links already
pass isSafeHref, but these tool-step links bypassed it, so a
prompt-injected tool call could put file:, ms-msdt:, vscode: or other
OS protocol-handler links into the chat. React 19 only rewrites
javascript: hrefs.
All three sites now reuse the markdown allowlist and render an unsafe
URL as plain text (the image thumbnail stays, unlinked). Tests render
MessageGroup for each tool with unsafe schemes plus a web-URL control.
* docs(changelog): note tool-step link scheme gating (#5526)
* fix(frontend): mark omitted tool-step links and guard web_fetch url type
Review follow-up. Tool steps dropped an unsafe URL to bare text, while
markdown and artifact links show a dotted "Unsafe link omitted" span, so
the two surfaces applying the same rule degraded differently. That span
was already duplicated between markdown-link.tsx and artifact-link.tsx;
it is now one UnsafeLink component used by all three renderers. It
passes extra props through so the image tile still works as a Radix
tooltip trigger.
web_fetch also read args.url with a cast only. A non-string url (models
occasionally emit one mid-stream) reached JSX as an object and threw,
taking down the message list. It is now typeof-guarded.
* fix(frontend): default missing tool-call args before rendering steps
Review follow-up. The web_fetch typeof guard dropped the optional
chaining of the cast it replaced, so a tool call without an args object
threw again. Other branches were already exposed the same way: seven
tool kinds (web_fetch, web_search, image_search, read_file, write_file,
str_replace, browser_*) threw on a missing or null args while building
their labels. convertToSteps now defaults args to {} once, so every
ToolCall branch receives an object.
A role whose skills policy allows nothing (or a fresh install with no
skills) gets an empty catalog from GET /api/skills once per-caller
filtering lands. Pin the composer wiring against that shape: the
dropdown still offers the builtin commands, an unmatched query hides it
quietly, and the matcher returns builtins-only for an empty catalog.
* fix(frontend): scope skill suggestions by agent
Filter composer slash-skill suggestions through the active custom agent allowlist so restricted agents do not offer unavailable skills.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(frontend): wait for agent scope before draft hydration
Keep agent loading distinct from explicit empty and inherited skill scopes so saved skill chips are restored only after the active scope settles.
Co-Authored-By: Claude Code <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
* feat(frontend): reference conversations from the composer
Adds a "Reference a conversation" button next to the attachment button,
shown only while GET /api/features reports read_conversation enabled. It
opens a picker over the recent-conversation list (current thread excluded,
capped at max_references) and shows removable chips in the composer.
On send the thread IDs ride SendMessageOptions.conversationReferences into
run context.conversation_references, which the Gateway consumes at
admission; the LangGraph SDK drops unknown top-level body fields. A
display-only copy ({thread_id, title}) on the visible human message lets
the transcript render read-only chips linking to the source.
References are per message: not persisted with the draft and cleared on
send or thread switch; regenerating or editing a turn runs without them
unless they are attached again.
Related to #5398. Depends on #5463.
* fix(frontend): pin the run-context contract and finish the picker states
Both thread.submit paths now build their run context through one exported
buildRunContext helper, tested directly: attached references travel as a
plain string[] under context.conversation_references only when the caller
passed them, a stray key in local settings is dropped instead of forwarded,
and the regenerate/edit replay path never carries references.
The picker shows a loading row while the conversation list is still in
flight instead of claiming there are no conversations, and the transcript
chip group is labelled with the previously unused referencedConversations
translation.
* fix(frontend): route conversation-reference chips to custom-agent sources
The picker offered custom-agent conversations but kept only the thread ID
and title, so transcript chips always linked to /workspace/chats/{id} and
dropped the source's custom-agent context on navigation.
Preserve the agent identity end to end: the picker now attaches
agentNameOfThread() (context first, then metadata.agent_name, mirroring
pathOfThread) to the selection, the display-only additional_kwargs metadata
round-trips it as agent_name, and the transcript chip passes it to
pathOfThread so custom-agent sources resolve to
/workspace/agents/{agent}/chats/{id}.
Tests: agent_name metadata round-trip and malformed-entry tolerance, picker
toggle carrying the metadata agent with run context winning, and a
picker-to-transcript regression pinning the /workspace/agents/writer/chats/
source-1 href.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash
Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.
Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
admission pinning (both server-owned sets + worker hoist); latest-only
request-scoped <project> block via DynamicContextMiddleware
wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
prefix + marker + provenance, never persisted); journal audit
fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
locked check-and-set; hash-qualified immutable shelf storage with
Paths helpers; upload/list/content/delete-to-trash routes; project
delete trashes the shelf in-transaction; request-scoped bounded
<documents> index with honest count/shown + actionable overflow note;
list_project_documents/read_project_document tools registered only on
pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
parity); POST from-thread with provenance; attach-to-thread with
lock-staged copy (archived source allowed); read-only thread-files
view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
file moves), purge (continuous row lock across unlink/delete/commit,
retryable on FS errors), retention sweep (lazy + startup, 24h orphan
guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
archived banner, content-missing rows), /workspace/trash route,
sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
+ specs.
Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
artifact + content responses; unified unsandboxed-iframe PDF preview
(fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
claims + os.link commit with suffix retry; same-name re-upload now
unique-names instead of replacing); hidden staging only, no visible
placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
revalidation; drain locked filesystem work on cancellation; preserve
bytes when an insert's commit state is uncertain (including trashed
rows)
- original-integrity checks before serving text or cached conversions;
content_missing surfaced in list responses (UI reads the flag, no
409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
rejects fractional/infinite values; composer counts staged
attachments; pending attachments persist until submission or removal;
in-flight instruction/rename edits survive save refetches; shelf and
trash pagination; conversation-file and thread-files pages stay
subscribed to refetches
Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.
Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
bounded: the indexed expiry purge still runs on every trigger
(GET /api/trash/documents, POST /api/trash/purge) while the
O(all rows + all files) reconciliation is throttled to one run per
user per 15 minutes (process-local, per-user window). The startup
sweep now runs as a background task instead of blocking gateway
readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
like the render path, so a pasted, fenced <project>/<documents> snippet
survives markdown export while real injected blocks (never fenced) are
still removed. Fence regexes moved to a dependency-free leaf module to
avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
attribute (the upstream e2e contract locates it via :not([title])), and
the upstream artifact-preview spec now pins the new contract: PDFs
render unsandboxed, images keep sandbox="".
* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title
- Shutdown cancelled only the shield around the background startup sweep,
so an all-users reconciliation that outlived the 5s budget kept walking
rows and files while the document repo and DB engine were disposed
underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
which cancels the task and drains it before worker exit: the shield
keeps the wait bounded, the cancel makes it final (CancelledError lands
at the sweep's next await, and `_run_startup_trash_sweep` only catches
`Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
previous fix round, leaving the PDF frame without an accessible name
while its siblings keep theirs. Restore it (WCAG frame titles), assert
it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
instead of `iframe:not([title])`.
* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel
`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.
* fix(projects): round-4 review — make Empty trash delete what it confirms
`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.
Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.
Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
* feat(settings): persist account preferences across browsers
* docs(settings): scope preference guidance to user persistence
* fix(settings): preserve SSR and fence custom-agent defaults
* test: include user persistence in scoped guidance inventory
* fix(settings): sync explicit edits and preserve local tab updates
* fix(frontend): keep human input cards with their turn
* fix(frontend): keep human input cards with the correct turn
Multi-turn ordering in restoreLocalTurnMessageOrder could place an
`ask_clarification` (needYourHelp) card on the wrong side of a newly
submitted human message, and after an interrupt/stop it could move the
current run's own already-executed steps above the human that started them.
- Restore established messages that a live checkpoint tail wove after the
new human (displacedBaselineMessages).
- Treat cards/messages confirmed only by the REST history page as
established-past-turn too (confirmedHistoryIdentities), not as in-flight
pending steps (displacedHistoryMessages).
- Never displace the CURRENT run's own steps after an interrupt/stop; they
belong after the human even once canonical history confirms them
(currentTurnRunIds, anchored by the pending human's run_id).
Fixes#4889
* fix(frontend): preserve ordering across displaced messages
* fix(frontend): close canonical history ordering gaps
* fix(frontend): preserve current turn anchor after compaction
* fix(frontend): anchor the local turn on the submitted human identity
Follow-up to #4892. R2 is reachable through the full hook chain: when the
checkpoint baseline covers only the latest turn, the server echo of the
submitted human confirms the optimistic copy against the unthrottled SDK
state while the ~80ms render snapshot cannot show it yet; the baseline-only
anchor scan then promoted an older history-only human into the current
turn's anchor and moved established history behind it.
- Record a LocalTurnAnchor at dispatch: one client-generated human id is
shared by the optimistic display copy and the submitted message, so the
server X__user echo confirms the exact identity already on screen.
- restoreLocalTurnMessageOrder repairs only when that identity is present
in the display; a null anchor (hidden human-input reply, regenerate
replay) or a not-yet-rendered identity keeps established history
untouched.
- Optimistic confirmation now observes the same coalesced render snapshot
(identity match first, rendered human-count growth as fallback for
runtime-re-keyed first turns) instead of the per-chunk array.
- Edit replays adopt the prepare response's replacement identity; the
render ledger excludes unconfirmed optimistic copies by identity now
that the local input no longer uses an opt- prefix, so a failed send
cannot pin a message the server never saw.
- Anchor lifecycle matches the previous baseline: kept across
finish/stop/error until canonical data takes over, replaced by the next
local submit, cleared on send failure, thread switch, and replay gaps.
* test(threads): type submit mock calls in local-turn-order dom tests
* fix(frontend): bound local turn repair to pre-submit history
* fix(frontend): preserve pre-submit bridge ordering
---------
Co-authored-by: 肘子香香 <hyh112300@163.com>
Co-authored-by: 霍英豪 <huoyinghao250707@credithc.com>
Co-authored-by: wangzeren <1004695029@qq.com>
* feat(authz): gate thread-delete and run-cancel UI on effective permissions (Phase 4, #4063)
Consume the effective route permissions surfaced by #5228 so the UI hides
actions the caller's role cannot perform:
- threads:delete hides the sidebar thread-row Delete menu item and the
sidecar panel delete button (every useDeleteThread consumer)
- runs:cancel disables the composer stop affordance; all three stop entry
points converge on one check inside handleStopStreaming
hasPermission treats an absent/null/unresolved permission list as
permissive, so a mixed old-backend/new-frontend deploy never hides actions
the caller can still perform. The Gateway @require_permission guards
remain the single enforcement point.
* fix(authz): review follow-ups for stop gating (comment accuracy, a11y, tests)
- Correct the defense-in-depth comment: the submit-button click is the
only live entry into handleStopStreaming (handleSubmit returns early
with the pleaseWaitStreaming toast while streaming, so the kind==="stop"
branch is unreachable); the handler gate stays as future-proofing.
- Explain the disabled stop affordance with aria-label + title (Radix
tooltips don't fire on disabled buttons), with en-US/zh-CN strings.
- Add the composer stop-gating DOM tests (disabled + onStop never fires +
permissive default) and the sidebar delete-menu gating tests, so all
gated surfaces carry wiring tests.
* fix(authz): stop conditional aria-label from stripping the submit name
The stop-gating follow-up (1612855b) explained the disabled stop
affordance with aria-label/title but passed explicitly-undefined
values in the non-denied case. PromptInputSubmit declares its default
aria-label="Submit" before {...props}, so the undefined key landed in
the spread and clobbered the default: React omits the attribute
entirely and the submit control lost its accessible name in every
state, which broke the sidecar e2e layout helper (it locates the
button by its "Submit" label).
Spread the attributes conditionally so they only attach when
stopDenied, and lock the invariant with a DOM test asserting the base
"Submit" name survives when stop is not denied (mutation-verified:
reverting the conditional spread turns the new test red).
* test(authz): drop unused rerenderWith helper, guard accessible name by role query
Address the review nit on the stop-gating DOM tests: the
rerenderWith helper was never called, and a second render() would
append a composer instead of updating the first one anyway — drop it
(the sidecar-delete-gating tests already demonstrate the correct
rerender pattern if a granted->denied flip test is ever needed).
Also resolve the accessible-name regression guard through
getByRole("button", { name: "Submit" }) so it fails exactly the way
e2e and assistive tech consume the control (mutation-verified: the
explicitly-undefined aria-label form turns it red).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): add interval schedule type
Allow scheduled tasks to fire every N seconds from last dispatch, not
only wall-clock cron or a single run_at. Cadence is UTC now+N with no
missed-beat catch-up, bounded by min_once_delay_seconds and 30 days.
* fix(scheduler): let interval tasks create, edit, and keep next run
Create/edit now keep every_seconds. Unchanged interval spec no longer
resets next_run_at, including timezone-only PATCH.
* fix(scheduler): keep non-minute intervals on edit
Stop rounding every_seconds to whole minutes in the form. Values that
are not whole minutes or hours now use a seconds unit so edit/duplicate
round-trips the stored cadence instead of rewriting it and resetting
next_run_at. Document that min_once_delay_seconds is also the interval
floor.
* fix(scheduler): clamp interval seconds to the default 60s floor
The new seconds unit allowed 1–59, which the API rejects under the
default min_once_delay_seconds. Clamp the form to >= 60 and show the
floor next to the preview. Also mention interval in the scheduler
field_doc, matching config.example.yaml.
* fix(scheduler): do not clamp interval amount while typing
Keystroke clamp made 90 become 9 -> 60, then 600, and backspace could
not leave 60. Keep the raw field text and apply the 60s floor on blur
and emit only.
* test(scheduler): cover interval input editing
* fix(frontend): preserve saved interval cadence until edited
* style(tests): format scheduled task router tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): let scheduled tasks pin a custom agent
Create and update accept optional assistant_id, defaulting to lead_agent.
Custom names are normalized and must already exist for the task owner.
The workspace form exposes the same choice, and duplicate copies it.
Fixes#5286
* fix(scheduler): keep assistant-id PR free of interval tests
Drop the six interval tests that belonged to the interval schedule PR
and fail here because this tree still only accepts once/cron.
Treat lead_agent case-insensitively so LEAD_AGENT / lead-agent store
as the default. Omit unchanged assistant_id on edit so a deleted custom
agent does not 422 unrelated PATCH (rename, reschedule).
* fix(scheduler): format task page and browser tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* test(threads): add red R3/R4 merge ordering regressions
* fix(threads): preserve trusted seq positions through content merge (R3/R4)
Extract the message ordering/identity logic into a pure
core/threads/message-order.ts module. Each normalized identity now tracks
latest visible content and trusted position separately: content replacement
no longer drops deerflow_seq/run_id/turn_duration (R3), and a seq-carrying
live message is placed by the ascending seq skeleton instead of the next
shared identity anchor (R4: 1,3,2,5 -> 1,2,3,5).
buildVisibleHistoryMessages converges repeated identities to the earliest
visible feed row (mirroring backend get_message_seqs), and the
summarization transient bridge plus rendered ledger share the same
position priority: trusted seq outranks anchor weaving, bridge refreshes
keep known seqs, and hidden control copies never contribute a visible
position.
* test(e2e): add long-thread ordering regression with compaction and pagination
Add tests/e2e/thread-ordering.spec.ts: a deterministic 68-row, 33-turn
fixture with two hidden compaction summaries, a paginated /messages/page
mock, and a live compaction during submit (real SSE frame shapes). Asserts
DOM group order at stage barriers, outline/scroll navigation across the
virtualized list, tool-card association, and order stability across reload.
Also close three mock gaps in mockLangGraphAPI (token-usage, mcp-tasks,
workspace-changes): unmocked they fell through to the absent gateway and
the 401 redirected thread pages to /login, breaking every thread-page spec
in a gateway-less Playwright environment.
* test(threads): address review on ordering regression coverage
- e2e: actually expand the collapsed web_search step and assert the
intermediate result payload (realistic JSON array fixture); assert the
new turn's DOM relative order via compareDocumentPosition instead of
racing viewport coordinates; add a Custom Agent route regression sharing
the same paginated fixture.
- Add a unit test for the hidden-control-only seq fallback path.
- Keep isNonEmptyString in hooks.ts (message-order.ts does not use it).
- Document the seq-first position authority contract in frontend/src/AGENTS.md.
* test(threads): type run_id fixtures via getMessageRunId accessor
* test(e2e): exercise the real collapsed-steps region for the tool payload check
The previous toolStep.click() was a no-op: as the last tool call, the
web_search step rendered unconditionally. Add a second tool call to the
turn-30 fixture so web_search falls into the collapsed moreSteps region,
assert the intermediate result payload is hidden while collapsed, then
click the "1 more step" button and assert it becomes visible.
* docs(frontend): prettier-format AGENTS.md merge contract
* fix(frontend): anchor mixed-sequence message segments
* test(auth): include project permissions in me contracts
* fix(frontend): anchor trailing steps to positioned live results
* fix(frontend): preserve prefixes before rescued sequence anchors
* fix(frontend): support standalone demo APIs and runtime GitHub stars
* Update API origin URL to use environment variables
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* test(frontend): align static demo tests with runtime origin
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* feat(artifacts): preview CSV and TSV files as bounded tables
* chore: keep preview screenshots out of the PR file diff
* fix(artifacts): detect record newlines outside quoted fields
* test(auth): include project permissions in me contract expectations
* feat(projects): project workspaces with scoped chats and thread membership
Backend:
- projects table model and migration; fail-closed ProjectRepository with
ownership checks, CRUD/archive/restore/delete router, and atomic thread
move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
metadata; project-aware thread create/search with pagination bounds and
membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
at write time and dropped when invalid; serialized against project
deletion and thread assignment
- branch creation inherits the source thread's project membership (an
archived/deleted project degrades the branch to unassigned instead of
failing the request)
Frontend:
- projects data layer, thread move API, and sidebar projects section with
flat/grouped modes, archived-project threads, and stable virtual-list
offsets
- project detail page with project-scoped new chat
(/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
continuation is dropped instead of saving the goal or launching the
abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
an open project page never keeps stale titles, recency, or pagination
* fix(chats): keep archive undo toast when the sidebar row unmounts
The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.
Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.
* fix(projects): pin project thread listing contract and exclude archived chats
GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.
The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.
Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.
* docs(migrations): record the 0019/0020 chain against the bootstrap reservation
The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.
Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.
* fix(chats): invalidate project thread lists on archive/restore
useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.
An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.
Regression test asserts the project-list prefix is invalidated on success.
* fix(projects): fetch project discovery only in grouped sidebar mode
RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.
Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.
* fix(threads): fail closed on project-scoped create in memory mode
MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).
Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.
Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.
* fix(projects): window the project page thread list
ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.
Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.
* fix(projects): restore row dividers and pin them with a render test
The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.
The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.
* fix(projects): validate forward schemas and fence membership reads
* fix(frontend): default to Webpack over Turbopack in dev to avoid PostCSS worker leak on macOS
On macOS arm64, Turbopack + Next.js 16.2.11 + Tailwind CSS v4 causes an
unbounded spawn of PostCSS evaluator processes that consume high CPU and
memory and never return a response. Webpack is unaffected.
Change the no-override default in getDevBundler() from platform-dependent
Turbopack (all non-Windows) to Webpack. DEER_FLOW_DEV_BUNDLER=turbo
continues to work as an explicit opt-in for local diagnosis.
Fixes#5132
* docs(frontend): address webpack default review feedback
* docs(frontend): clarify webpack default rationale
* fix(history): stop dropping user messages that fall outside the loaded page window
Two independent paths made a user's own message disappear from a long thread
(#4666, #4508, #4363). Both are reproduced by a real two-round run: once the
thread passes the 50-row `/messages/page` window AND context compaction fires,
the two sources of truth stop overlapping at the head.
1. Middleware-answered tool results never reached the event store. A middleware
that short-circuits a tool call (e.g. ReadBeforeWriteMiddleware's blocked
write) returns a user-visible ToolMessage, but LangChain never emits
`on_tool_end`, so RunJournal never persisted it — the user saw it during the
run and it vanished on reload. RunJournal already reconciles final-output
tool messages, but only for an `ask_clarification` allowlist. The allowlist
is removed; scope stays bounded by the three conditions that actually matter
(visible, this run's lead agent, not already persisted), so subagent results
still stay in their own step feed.
2. mergeMessages discarded the checkpoint prefix before the first shared anchor.
#4065 correctly established that a summarization-rescued early message must
not be appended to the tail, and suppressed it instead. That suppression is
what deletes the message when the first history page no longer reaches back
to it. It is now woven in before the first shared anchor — the one position
both the checkpoint and seq-sorted history agree on — so #4065's invariant
(never the tail) still holds. A collapsed unloaded gap is recoverable by
paging; a dropped message is not.
Verified against real captured payloads from the reproducing run: the first user
message returns to the transcript. Its exact position is still approximate —
after compaction the live window carries too few anchors to place it precisely,
which only seq-based ordering can close.
Backend: 10809 passed (baseline 10808; same 15 pre-existing failures in
browser/crawler community tools). Frontend: 986 passed, typecheck + eslint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(events): look up a persisted message's seq by identity
Groundwork for placing checkpoint messages in the seq-ordered thread feed
(#4666). A checkpoint carries no seq of its own and loses messages to
summarization, so once the feed's 50-row page window no longer reaches back to a
surviving old message, a client has nothing to place it by. The seq already
exists in run_events keyed by the message id — this exposes it without paging
the whole feed.
`message_identity` is the backend half of the identity rule the frontend applies
in `hooks.ts::messageIdentity`: a ToolMessage is keyed by `tool_call_id`, and
DynamicContextMiddleware's `X` / `X__user` human copies collapse to one identity.
The two halves must stay in sync — a mismatch is silent, degrading placement
rather than raising.
`get_message_seqs` is implemented for all three stores. Misses are absent from
the result rather than an error, so callers degrade to their own placement rule;
the earliest seq wins when one identity resolves to several rows, so a
re-persisted message keeps the position it first occupied. The DB store decodes
rows in Python because `content` is a TEXT column holding a JSON string, not a
JSON column — the identity fields cannot be projected in SQL.
Nothing consumes this yet; no behavior change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(runtime): carry each persisted message's feed seq on values frames
Attaches `additional_kwargs.deerflow_seq` to messages in a root `values` frame
that the thread feed already holds, so a client can place a message the
checkpoint kept but its loaded history page window no longer reaches (#4666).
Nothing is written back to the checkpoint: the seq is added when the frame is
serialized and belongs to that frame only.
Cost is bounded to frames introducing identities the run has not resolved yet.
Messages this run produces are not in the feed while streaming, so they are
looked up once, recorded as misses, and never retried — in a real run the only
frame that pays for a query is the one where compaction brings older messages
back into view. Measured on a reproducing two-round run: 1 lookup across 25
values frames.
The stamper is built once per run rather than per `_stream_once`, or a goal
continuation would discard the resolved seqs. Subgraph frames are not stamped:
a subagent's snapshot is not part of this thread's feed ordering. A lookup
failure logs and leaves the frame unstamped rather than failing it — placement
is an enhancement and clients fall back to their own ordering rule.
Frontend does not read the field yet; no behavior change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(gateway): strip the server-owned message seq from untrusted input
`deerflow_seq` is display metadata the Gateway attaches when it serializes a
values frame. A client replaying messages (regenerate / edit-and-rerun) would
otherwise write it into the checkpoint, where it becomes wrong the moment the
thread is forked — a branch re-seeds its feed and reassigns seq (#4380).
Joins the existing server-owned key set, so it follows the same trusted-internal
rule as the dynamic-context and view-image markers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(frontend): place a checkpoint message by its feed seq, not its nearest anchor
Completes #4666. Weaving a compaction-rescued message before the first shared
anchor keeps it in the transcript, but not in the right place: after compaction
the live window carries too few anchors, and the nearest one can sit deep inside
the loaded page window — measured at row 25 of 50 on a reproducing run, which is
why the first user turn rendered mid-transcript instead of at the head.
Both sides now carry the backend's thread-global seq. `buildVisibleHistoryMessages`
copies each row's `seq` onto the message (same shape as the existing `run_id`),
and the Gateway stamps it onto `values` frame messages it has already persisted.
A live message whose seq is below the loaded window's lower bound is placed ahead
of everything on screen rather than before the nearest anchor. A message with no
seq — still streaming, so not in the feed yet — keeps the weaving path, since the
tail is already its correct position.
Verified against the captured payloads of the reproducing run: the first user
message goes from absent, to #13 (behind the second question), to #0.
Frontend: 988 passed, typecheck + eslint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(frontend): place a pre-window checkpoint message even when no anchor is shared
Also #4666. Placing a compaction-rescued message by its feed seq was gated on
reaching a shared anchor, because the split ran inside the anchor walk. When the
loaded page and the live checkpoint share no identity at all, that walk never
runs and the message fell through to `[...canonical, ...live]` — appended after
the entire window, the one arrangement #4065 proved wrong, with its seq known
the whole time.
That is not a corner case. Open an old, already-summarized conversation and send
a message: the page on screen is the newest rows from before that turn, while
the checkpoint holds the rescued first user turn plus steps of the new run that
are not in the feed yet. On a reproducing run the two sides shared zero anchors
and the user's own first question rendered at row 50 of 50 — the reported
"first message jumps to the bottom".
Split `beforeWindow` out of `live` before walking anchors, walk `liveInWindow`,
and use it for the no-anchor branch as well, so a message routed ahead of the
window is not re-appended at the tail by dedup.
Measured on captured payloads of a reproducing run (real gateway, real
compaction), first user message position:
no shared anchor: row 50 -> row 0, seq order monotonic again
shared anchors: row 0 -> row 0 (unchanged)
paged to the top: row 0 -> row 0 (unchanged)
Regression test verified red-green: reverting the fix fails it with the message
rendered after the window.
Frontend: 989 passed, eslint + tsc clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(gateway): stamp the message feed seq on checkpoint reads, not only on stream frames
Completes #4666. `_MessageSeqStamper` sits on the streaming publish path, so a
client that joins a live run learns where a summarization-rescued turn belongs
while a client that merely opens the conversation does not — and opening is the
common case. `GET /threads/{id}/state` and `POST /threads/{id}/history` returned
the checkpoint with no seq at all, so the merge fell back to the nearest shared
anchor, which after summarization sits deep inside the loaded page.
Reproduced in a browser against a real gateway, on a thread that had already
compacted: the user's first question rendered at row 320 of 389, behind the
newest question instead of at the head. Both reads showed 0 of 13 messages
carrying a seq. That is the reported symptom, still present after the streaming
fix.
Add `stamp_messages_with_seq`, the request-scoped counterpart of the stamper:
everything a checkpoint still holds is already persisted, so one batched lookup
resolves the whole list and there is nothing to retry later. Resolve the store
through `_optional_run_event_store` rather than `get_run_event_store`, because
seq is placement metadata — a deployment without a feed must still be able to
read a thread.
After the fix, on the same thread in the same browser: 13 of 13 messages carry a
seq and the first question renders at the head, ahead of the newest one.
Backend: ruff clean, 326 passed across the touched suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(harness): move the injected-user-id suffix helpers to utils.messages to break an import cycle
message_identity imported strip_injected_user_message_id_suffix from the
dynamic-context middleware, closing a cycle (middleware -> deerflow.runtime
-> worker -> events -> middleware) that only stayed hidden while an earlier
import happened to break it. Define INJECTED_USER_MESSAGE_ID_SUFFIX and the
strip helper in deerflow.utils.messages and re-export them from the
middleware so existing importers keep working.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(docs): improve formatting and clarity in AGENTS.md and message-merge.test.ts
* perf(events): stop the seq scan once every wanted identity is resolved
Rows past the last wanted seq can only be re-persisted copies that
already lose the earliest-seq-wins tiebreak, so all three stores now
break out of the scan (and the db store out of its per-row JSON
decoding) once found covers wanted. Matters most for /state and
/history reads of long threads, where this lookup runs with no run
cache and a typically tiny wanted set.
Raised by review on #4696.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(events): share the seq-stamping expression between the two stampers
The walrus-plus-merge expression was duplicated verbatim between
stamp_messages_with_seq and _MessageSeqStamper.stamp — two counterparts
of one rule where silent divergence is the likely failure mode if only
one side is edited. Both now call attach_message_seq next to
MESSAGE_SEQ_KEY in message_identity.py. The trailing
isinstance(message, Mapping) guard was unreachable (a non-Mapping entry
already got identity = None) and is gone with the extraction.
Raised by review on #4696.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(events): seq stamping survives launch paths without user context
The db store's get_message_seqs defaults to user_id=AUTO, which raises
when no user is in the contextvar — the first strict-AUTO read ever
called from the worker context. On a launch path that never inherits
the auth context (e.g. a null-owner scheduled task), stamp()'s except
clause swallowed that into a per-frame warning and silently disabled
seq stamping for exactly the background runs that need it.
The stamper now soft-resolves the user id once at build time — the
same rule as the worker's write paths beside it (unset -> no filter)
— and passes it explicitly. jsonl/memory stores gain the same
user_id kwarg the base list_messages contract already carries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* perf(events): SQL-prefilter the message seq lookup's candidate rows
get_message_seqs scanned and JSON-decoded every message row of the
thread: the early exit never fires when a wanted identity is absent
from the feed (a message still streaming, or checkpoint-only), and
/state / /history reads want the newest messages, so the ascending
scan traversed essentially the whole feed — with the content column
carrying full tool outputs, that is heavy I/O plus N JSON parses on
exactly the long threads this lookup exists for.
A LIKE prefilter now keeps that cost in SQL: only rows containing a
wanted raw id as a substring are fetched and decoded. False positives
are re-checked by message_identity; LIKE wildcards are escaped; an id
json.dumps would escape (breaking the verbatim-substring guarantee)
falls the whole set back to the full scan rather than silently
missing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(agents): sink runtime mechanism docs below the gateway guidance budget
Merging main pushed backend/app/gateway/AGENTS.md past its 40KB soft
budget (main had left 81 bytes of headroom). Per the nearest-file rule,
move the mechanism detail of the message-seq stamping and run-delivery
receipt sections — both owned by runtime/ code — into
packages/harness/deerflow/runtime/AGENTS.md, leaving the gateway file
the REST-surface summary and a pointer. The seq section also documents
the stamper's build-time soft user-id resolution and the db store's SQL
prefilter from the review follow-ups.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(agents): sink durable-MCP task detail below the backend guidance budget
Merging main pushed backend/AGENTS.md past its 24KB module soft budget
(main itself is at 24762 after #4848 — this branch adds zero net bytes
to the file). Per the nearest-file rule, move the two durable-MCP task
runtime bullets' mechanism detail into
packages/harness/deerflow/mcp/AGENTS.md, leaving summaries and
pointers; this also restores ~2KB of headroom so the next merge does
not trip the same wire.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(events): re-ask a message-seq miss once the feed advances
The run-scoped stamper cached lookup misses for the whole run. A message
this run produces reaches a values frame before RunJournal flushes it, so
its first lookup legitimately misses — and the journal persists it moments
later, giving it a feed seq the stamper never asks for again. A long run
that afterwards rolls past the history page and compacts then carries that
message unstamped, back to the approximate anchor placement this stamper
exists to replace (#4666). A transient store error had the same permanent
effect, since the except clause degrades to an empty result.
A miss is now provisional while a hit stays final: RunJournal counts its
successful event-store writes as `feed_generation`, and the stamper re-asks
a missed identity only once that counter moves. Retrying is therefore
bounded by feed writes rather than by frames — the per-frame query the
run-scoped cache was built to avoid — and a failed lookup costs one
generation instead of the run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
