* feat(authz): gate composer send on runs:create (Phase 4 follow-up, #4063)
Mirrors #5294's stop gating on the send side: both chat routes pass
canCreateRuns (from PERMISSIONS.RUNS_CREATE, lockstep with the backend
enum) into the shared composer. The gate sits at the top of
submitThreadMessage — the single choke point every composer entry
(submit button, Enter, goal-set-triggered run) funnels through — and
denies with a toast plus a rejected promise so PromptInput keeps the
text. The idle submit button is disabled and explains the boundary via
conditionally-spread aria-label/title (startTurnUnavailable, en/zh);
while streaming the button stays the runs:cancel stop affordance.
Also removes the unreachable kind === "stop" branch in handleSubmit
(flagged during #5294's review): the Enter path early-returns with the
streaming toast before the classifier runs.
* fix(frontend): reject denied goal starts before saving state
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): add WeChat QR login and binding recovery
* fix(channels): enforce single-worker WeChat QR login and preserve bot ID
Reject QR login endpoints when multiple Gateway workers are configured, while keeping manual token setup available.
Preserve the configured bot ID when the provider omits it or returns an empty value. Add regression coverage for worker guards and credential persistence.
* fix(channels): sync WeChat completion state on provider updates
Show the connected step when refreshed provider data confirms the binding, so the dialog no longer waits indefinitely after polling is cancelled.
Add regression tests for provider updates during pending poll and binding requests, including late responses and expiry.
* fix(channels): preserve WeChat pairing codes across waits and redirects
---------
Co-authored-by: YxinMiracle <“939157765@qq.com”>
* feat(knowledge): add verifiable RAGFlow source citations
* docs(knowledge): scope RAGFlow guidance to its own directory
* fix(knowledge): preserve citations through rendering and budgets
* feat(frontend): reference conversations from the composer
Adds a "Reference a conversation" button next to the attachment button,
shown only while GET /api/features reports read_conversation enabled. It
opens a picker over the recent-conversation list (current thread excluded,
capped at max_references) and shows removable chips in the composer.
On send the thread IDs ride SendMessageOptions.conversationReferences into
run context.conversation_references, which the Gateway consumes at
admission; the LangGraph SDK drops unknown top-level body fields. A
display-only copy ({thread_id, title}) on the visible human message lets
the transcript render read-only chips linking to the source.
References are per message: not persisted with the draft and cleared on
send or thread switch; regenerating or editing a turn runs without them
unless they are attached again.
Related to #5398. Depends on #5463.
* fix(frontend): pin the run-context contract and finish the picker states
Both thread.submit paths now build their run context through one exported
buildRunContext helper, tested directly: attached references travel as a
plain string[] under context.conversation_references only when the caller
passed them, a stray key in local settings is dropped instead of forwarded,
and the regenerate/edit replay path never carries references.
The picker shows a loading row while the conversation list is still in
flight instead of claiming there are no conversations, and the transcript
chip group is labelled with the previously unused referencedConversations
translation.
* fix(frontend): route conversation-reference chips to custom-agent sources
The picker offered custom-agent conversations but kept only the thread ID
and title, so transcript chips always linked to /workspace/chats/{id} and
dropped the source's custom-agent context on navigation.
Preserve the agent identity end to end: the picker now attaches
agentNameOfThread() (context first, then metadata.agent_name, mirroring
pathOfThread) to the selection, the display-only additional_kwargs metadata
round-trips it as agent_name, and the transcript chip passes it to
pathOfThread so custom-agent sources resolve to
/workspace/agents/{agent}/chats/{id}.
Tests: agent_name metadata round-trip and malformed-entry tolerance, picker
toggle carrying the metadata agent with run context winning, and a
picker-to-transcript regression pinning the /workspace/agents/writer/chats/
source-1 href.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash
Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.
Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
admission pinning (both server-owned sets + worker hoist); latest-only
request-scoped <project> block via DynamicContextMiddleware
wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
prefix + marker + provenance, never persisted); journal audit
fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
locked check-and-set; hash-qualified immutable shelf storage with
Paths helpers; upload/list/content/delete-to-trash routes; project
delete trashes the shelf in-transaction; request-scoped bounded
<documents> index with honest count/shown + actionable overflow note;
list_project_documents/read_project_document tools registered only on
pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
parity); POST from-thread with provenance; attach-to-thread with
lock-staged copy (archived source allowed); read-only thread-files
view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
file moves), purge (continuous row lock across unlink/delete/commit,
retryable on FS errors), retention sweep (lazy + startup, 24h orphan
guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
archived banner, content-missing rows), /workspace/trash route,
sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
+ specs.
Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
artifact + content responses; unified unsandboxed-iframe PDF preview
(fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
claims + os.link commit with suffix retry; same-name re-upload now
unique-names instead of replacing); hidden staging only, no visible
placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
revalidation; drain locked filesystem work on cancellation; preserve
bytes when an insert's commit state is uncertain (including trashed
rows)
- original-integrity checks before serving text or cached conversions;
content_missing surfaced in list responses (UI reads the flag, no
409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
rejects fractional/infinite values; composer counts staged
attachments; pending attachments persist until submission or removal;
in-flight instruction/rename edits survive save refetches; shelf and
trash pagination; conversation-file and thread-files pages stay
subscribed to refetches
Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.
Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
bounded: the indexed expiry purge still runs on every trigger
(GET /api/trash/documents, POST /api/trash/purge) while the
O(all rows + all files) reconciliation is throttled to one run per
user per 15 minutes (process-local, per-user window). The startup
sweep now runs as a background task instead of blocking gateway
readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
like the render path, so a pasted, fenced <project>/<documents> snippet
survives markdown export while real injected blocks (never fenced) are
still removed. Fence regexes moved to a dependency-free leaf module to
avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
attribute (the upstream e2e contract locates it via :not([title])), and
the upstream artifact-preview spec now pins the new contract: PDFs
render unsandboxed, images keep sandbox="".
* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title
- Shutdown cancelled only the shield around the background startup sweep,
so an all-users reconciliation that outlived the 5s budget kept walking
rows and files while the document repo and DB engine were disposed
underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
which cancels the task and drains it before worker exit: the shield
keeps the wait bounded, the cancel makes it final (CancelledError lands
at the sweep's next await, and `_run_startup_trash_sweep` only catches
`Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
previous fix round, leaving the PDF frame without an accessible name
while its siblings keep theirs. Restore it (WCAG frame titles), assert
it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
instead of `iframe:not([title])`.
* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel
`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.
* fix(projects): round-4 review — make Empty trash delete what it confirms
`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.
Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.
Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
* feat(authz): gate thread-delete and run-cancel UI on effective permissions (Phase 4, #4063)
Consume the effective route permissions surfaced by #5228 so the UI hides
actions the caller's role cannot perform:
- threads:delete hides the sidebar thread-row Delete menu item and the
sidecar panel delete button (every useDeleteThread consumer)
- runs:cancel disables the composer stop affordance; all three stop entry
points converge on one check inside handleStopStreaming
hasPermission treats an absent/null/unresolved permission list as
permissive, so a mixed old-backend/new-frontend deploy never hides actions
the caller can still perform. The Gateway @require_permission guards
remain the single enforcement point.
* fix(authz): review follow-ups for stop gating (comment accuracy, a11y, tests)
- Correct the defense-in-depth comment: the submit-button click is the
only live entry into handleStopStreaming (handleSubmit returns early
with the pleaseWaitStreaming toast while streaming, so the kind==="stop"
branch is unreachable); the handler gate stays as future-proofing.
- Explain the disabled stop affordance with aria-label + title (Radix
tooltips don't fire on disabled buttons), with en-US/zh-CN strings.
- Add the composer stop-gating DOM tests (disabled + onStop never fires +
permissive default) and the sidebar delete-menu gating tests, so all
gated surfaces carry wiring tests.
* fix(authz): stop conditional aria-label from stripping the submit name
The stop-gating follow-up (1612855b) explained the disabled stop
affordance with aria-label/title but passed explicitly-undefined
values in the non-denied case. PromptInputSubmit declares its default
aria-label="Submit" before {...props}, so the undefined key landed in
the spread and clobbered the default: React omits the attribute
entirely and the submit control lost its accessible name in every
state, which broke the sidecar e2e layout helper (it locates the
button by its "Submit" label).
Spread the attributes conditionally so they only attach when
stopDenied, and lock the invariant with a DOM test asserting the base
"Submit" name survives when stop is not denied (mutation-verified:
reverting the conditional spread turns the new test red).
* test(authz): drop unused rerenderWith helper, guard accessible name by role query
Address the review nit on the stop-gating DOM tests: the
rerenderWith helper was never called, and a second render() would
append a composer instead of updating the first one anyway — drop it
(the sidecar-delete-gating tests already demonstrate the correct
rerender pattern if a granted->denied flip test is ever needed).
Also resolve the accessible-name regression guard through
getByRole("button", { name: "Submit" }) so it fails exactly the way
e2e and assistive tech consume the control (mutation-verified: the
explicitly-undefined aria-label form turns it red).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): add interval schedule type
Allow scheduled tasks to fire every N seconds from last dispatch, not
only wall-clock cron or a single run_at. Cadence is UTC now+N with no
missed-beat catch-up, bounded by min_once_delay_seconds and 30 days.
* fix(scheduler): let interval tasks create, edit, and keep next run
Create/edit now keep every_seconds. Unchanged interval spec no longer
resets next_run_at, including timezone-only PATCH.
* fix(scheduler): keep non-minute intervals on edit
Stop rounding every_seconds to whole minutes in the form. Values that
are not whole minutes or hours now use a seconds unit so edit/duplicate
round-trips the stored cadence instead of rewriting it and resetting
next_run_at. Document that min_once_delay_seconds is also the interval
floor.
* fix(scheduler): clamp interval seconds to the default 60s floor
The new seconds unit allowed 1–59, which the API rejects under the
default min_once_delay_seconds. Clamp the form to >= 60 and show the
floor next to the preview. Also mention interval in the scheduler
field_doc, matching config.example.yaml.
* fix(scheduler): do not clamp interval amount while typing
Keystroke clamp made 90 become 9 -> 60, then 600, and backspace could
not leave 60. Keep the raw field text and apply the 60s floor on blur
and emit only.
* test(scheduler): cover interval input editing
* fix(frontend): preserve saved interval cadence until edited
* style(tests): format scheduled task router tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): let scheduled tasks pin a custom agent
Create and update accept optional assistant_id, defaulting to lead_agent.
Custom names are normalized and must already exist for the task owner.
The workspace form exposes the same choice, and duplicate copies it.
Fixes#5286
* fix(scheduler): keep assistant-id PR free of interval tests
Drop the six interval tests that belonged to the interval schedule PR
and fail here because this tree still only accepts once/cron.
Treat lead_agent case-insensitively so LEAD_AGENT / lead-agent store
as the default. Omit unchanged assistant_id on edit so a deleted custom
agent does not 422 unrelated PATCH (rename, reschedule).
* fix(scheduler): format task page and browser tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(artifacts): preview CSV and TSV files as bounded tables
* chore: keep preview screenshots out of the PR file diff
* fix(artifacts): detect record newlines outside quoted fields
* test(auth): include project permissions in me contract expectations
* feat(projects): project workspaces with scoped chats and thread membership
Backend:
- projects table model and migration; fail-closed ProjectRepository with
ownership checks, CRUD/archive/restore/delete router, and atomic thread
move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
metadata; project-aware thread create/search with pagination bounds and
membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
at write time and dropped when invalid; serialized against project
deletion and thread assignment
- branch creation inherits the source thread's project membership (an
archived/deleted project degrades the branch to unassigned instead of
failing the request)
Frontend:
- projects data layer, thread move API, and sidebar projects section with
flat/grouped modes, archived-project threads, and stable virtual-list
offsets
- project detail page with project-scoped new chat
(/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
continuation is dropped instead of saving the goal or launching the
abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
an open project page never keeps stale titles, recency, or pagination
* fix(chats): keep archive undo toast when the sidebar row unmounts
The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.
Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.
* fix(projects): pin project thread listing contract and exclude archived chats
GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.
The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.
Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.
* docs(migrations): record the 0019/0020 chain against the bootstrap reservation
The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.
Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.
* fix(chats): invalidate project thread lists on archive/restore
useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.
An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.
Regression test asserts the project-list prefix is invalidated on success.
* fix(projects): fetch project discovery only in grouped sidebar mode
RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.
Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.
* fix(threads): fail closed on project-scoped create in memory mode
MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).
Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.
Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.
* fix(projects): window the project page thread list
ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.
Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.
* fix(projects): restore row dividers and pin them with a render test
The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.
The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.
* fix(projects): validate forward schemas and fence membership reads
* feat(frontend): render markdown artifacts in the new window
The artifacts panel's "open in new window" action handed the browser the
raw Gateway response. For markdown that is a `text/markdown` body the
browser can only show as source, so the new window was a text dump rather
than a reader.
Route markdown artifacts to a new `/artifacts/view` page that renders them
with the same components the panel uses (SafeStreamdown + the artifact
rehype chain + citation links/panel), including the truncated-preview
banner and its "load full file" action. Everything else keeps the raw
Gateway URL — notably HTML/SVG, which the Gateway deliberately serves as a
download so active content never executes in the application origin.
- `core/artifacts/viewer.ts` centralizes which stored artifacts are
markdown (`.skill` archives included, since they hold a SKILL.md), so
the panel and the viewer route cannot drift.
- `ArtifactFilePreview` and its siblings move out of
`artifact-file-detail.tsx` into `artifact-file-preview.tsx`; otherwise
the standalone route would pull the CodeMirror editor into its bundle.
- The window title comes from the route's `generateMetadata`, not
`document.title`, which the App Router overwrites after hydration.
- The viewer reads content through `useStandaloneArtifactContent`, which
shares `useArtifactContent`'s query key but not its `useThread`
dependency, since a detached window has no thread context.
Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC
* fix(frontend): keep the artifact target across re-authentication
Review found the standalone viewer unrecoverable from an expired session.
The window's target lives entirely in `?path=...&thread_id=...`, and both
auth paths dropped it:
- The layout guard redirected to `/login` with no `next` at all. A layout
cannot read `searchParams`, so the guard moves into the page, which can
— and rebuilds the full viewer address for `next`. The layout loses its
AuthProvider along the way: nothing under this route reads `useAuth`,
and the guard now makes a single `getServerSideUser` call per request.
- The shared fetch wrapper built `next` from `window.location.pathname`,
which silently truncated the query string. It now carries `search` too,
so any route holding state in the query survives a 401, not just this
one. `validateAuthNextPath` already accepts a query string.
`buildArtifactViewerURL` is split out of `resolveArtifactOpenURL`: the
guard needs the route itself, never the Gateway fallback that the latter
takes for non-markdown targets.
Tests: the login round trip (unit — the rebuilt URL survives
`validateAuthNextPath` and parses back to the same target), the fetch
wrapper preserving the query on 401 (unit), and the expired-session
window reaching `/login` with the artifact intact (E2E). The E2E asserts
on the popup's navigation *requests*, since `(auth)/layout` answers
`/login` with a server redirect under DEER_FLOW_AUTH_DISABLED and no
navigation commits.
`tests/unit/core/models/api.test.ts` stubbed `window.location` without
`search`; a real Location always has it.
Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC
* fix(frontend): keep public showcase artifacts out of the auth gate
Review found that the viewer's access check regressed `/showcase`. Those
pages render with `isMock`, their artifacts are served by the
unauthenticated demo route, and the raw artifact URL this window replaced
stayed public — so gating the window unconditionally bounced every
logged-out showcase visitor to /login for a document that is already
public.
`requiresAuthenticatedViewer` exempts a mock target only when
`resolveStaticDemoArtifact` would actually serve it. The allowlist is the
authority rather than the flag: `mock=true` is caller-supplied, so a
target the demo route answers with 404 — a non-allowlisted path, or a
thread that is not a demo thread — still needs a session.
Covered in `tests/e2e-auth/`, since the default E2E config disables auth
and cannot see this: a public showcase artifact renders without a
session, while a non-allowlisted path and a missing mock flag both land
on /login. Verified the positive case goes red without the exemption.
Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC
* fix: DeerFlow brand casing in disclaimer, 'What's New' title, setup-sandbox .PHONY
- en-US disclaimer said 'Deerflow'; every other user-facing string
brands it 'DeerFlow' (rendered under the chat composer)
- landing section title 'Whats New in DeerFlow 2.0' -> 'What's New'
- setup-sandbox was the only Makefile target missing from .PHONY
* test: align disclaimer assertions with DeerFlow brand casing
The unit and e2e suites still expected the old "Deerflow" spelling in
inputBox.disclaimer; update both to the corrected locale string so the
i18n load test and the chat footer assertion pass again.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(artifacts): inline editing for text artifacts in the panel
Add a PUT /api/threads/{id}/artifacts/{path} endpoint that atomically
replaces an existing UTF-8 text file under /mnt/user-data/outputs after
verifying its SHA-256 revision. Active runs conflict (409); binary,
symlink, oversized, and non-output paths are rejected.
Frontend: edit/save/discard buttons, draft state with conflict detection,
CodeEditor onChange/onSave, loader SHA-256 from ETag, i18n, beforeunload guard.
Backend: PUT endpoint with thread reservation, atomic temp-file replacement,
sandbox sync for non-mounted providers, rollback on failure, ETag on GET.
Tests: 8 backend + 1 blocking-IO + 3 frontend test files.
* fix(artifacts): scope replacement permissions and release sandboxes
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: show real-time context window usage in chat UI (#3125)
Adds a `context_usage` block to `GET /api/threads/{id}/token-usage`
(token count from the live checkpoint, the thread model's
`context_window`, and a percentage), introduces a new
`ModelConfig.context_window` distinct from the per-call `max_tokens`
output cap, and surfaces the percentage in the chat header — inside
`TokenUsageIndicator` when token-usage tracking is on, or as a
standalone badge when it's off so context capacity stays visible
independent of cost tracking.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat: per-category breakdown for context window usage
Replace the single-number context_usage payload with a Claude-Code-style
breakdown — messages, system prompt, skills, system/MCP tools (active +
deferred), custom agents, memory injection, autocompact buffer, and free
space — and surface it in the chat UI with a segmented progress bar and
per-row table.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(config): document context_window across model examples
Add `context_window` to every example model in config.example.yaml so the
new chat-UI "% context used" indicator works out of the box for whichever
example a user adopts. Each value is the published default at the time of
writing; users are pointed at the official model spec to verify. Bumps
config_version to 11 so `make config-upgrade` flags outdated user configs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* style: ruff format (line-length 240)
No behavior change — collapses two multi-line expressions that fit on
one line under the project's 240-char limit. Picked up by `make format`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* review: address Copilot bot comments on #3183
- token-usage-indicator: switch `{contextPercentage && (...)}` to an
explicit `!= null` check. (The string `"0"` is actually truthy in JS so
the original code wasn't buggy, but the explicit check is clearer.)
- context-usage-breakdown: drop the `useMemo` around segments/totals — the
computation is O(n) over a handful of rows and the previous memo deps
omitted `t.contextUsage.categories`, so the bar's tooltips/aria-labels
could stay in the old language after a locale switch.
- context_usage._split_tools: snapshot MCP names from
`get_cached_mcp_tools()` directly instead of re-reading
`extensions_config.json` after `get_available_tools()` already loaded
it. Removes redundant file I/O on every `/token-usage` poll.
(`get_available_tools()` still emits its own INFO logs — silencing
those is out of scope here.)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* style(frontend): prettier --write context-usage-breakdown
CI's `pnpm format` (prettier --check) caught two lines previously
formatted by hand. Collapses one comma to fit on one line; no behavior
change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(gateway): correct context-usage breakdown + add exact token counting
The context-usage indicator shipped two bugs that silently zeroed whole
breakdown rows (both caught by try/except, so the feature looked alive but
produced wrong numbers):
1. _count_system_prompt passed app_config= to get_deferred_tools_prompt_section,
which only accepts deferred_names -> TypeError swallowed -> system_prompt
row always 0, and used_tokens/percentage undercounted by the full prompt.
Also subtracted the deferred section twice (the rendered prompt already
excluded it). Fix: derive deferred names deterministically and pass them to
apply_prompt_template; drop the redundant subtraction.
2. _split_tools imported a non-existent get_deferred_registry -> ImportError
swallowed -> all four tool-category rows always 0. Fix: classify via the
public is_mcp_tool predicate + tool_search.enabled (mirrors
build_deferred_tool_setup); the MCP tag is set by get_available_tools.
Added token_usage.counting (approximate|exact). 'exact' routes text/schema/
message counting through the model tokenizer (tiktoken cl100k_base) via the
existing memory-module machinery (lazy load + cache + cooldown + CJK-aware
fallback), so CJK-heavy threads stop being undercounted by chars//4.
Regression + e2e tests added; 6621 backend tests pass.
* fix(gateway): harden context usage accounting
* fix(gateway): count promoted MCP tools as active in context usage
Promoted tools (deferred MCP tools the thread has fetched via tool_search)
have their full schema bound on every subsequent turn by
DeferredToolFilterMiddleware, so they consume context like any active tool.
The breakdown previously left them in the reserved *_deferred rows, under-
counting the thread's used_tokens.
Classification now treats a tool as deferred only when tool_search is enabled,
it is MCP-sourced, AND it has not been promoted. The promoted set is read from
the checkpoint's channel_values and scoped by catalog hash — matching the
runtime middleware, so a stale promotion from MCP-config drift cannot inflate
the active count.
The static system prompt still lists all deferred tool names (promotions only
affect schema binding, not the prompt), so _count_system_prompt's deferred
rendering is intentionally left unchanged.
8 new tests cover classification, catalog-hash scoping (match / drift /
compute-failure / malformed), and checkpoint extraction.
* fix(context): address review feedback
* fix(context): count structured message payloads
* fix(context): harden usage accounting
* fix(config): bump schema for context usage fields
* refactor: narrow context usage to core indicator
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(lark): sidecar credential broker for sandbox lark-cli (Pattern B)
Removes the plaintext Lark credential mounts (appSecret + OAuth tokens)
from the sandbox container. A long-running broker sidecar owns lark-cli
and the per-user config/data dirs and serves the command surface over
Pod loopback; the sandbox gets only a forwarding shim on PATH, so the
raw credential files never exist in the sandbox filesystem.
- lark_broker.py: stdlib-only loopback broker (argv passthrough with
shell=False, server-injected credential env, bounded I/O) + shim
script constant + install-shim mode.
- docker/lark-cli-broker: init(install-shim) + serve image.
- provisioner: LARK_CLI_BROKER_IMAGE + provision_lark_cli_broker →
shim init container + lark-cli-broker sidecar (config/data mounted
sidecar-only); credentials dropped from the sandbox container;
/api/capabilities reports lark_cli_broker_image. Broker supersedes
the Pattern A init-container binary when both are configured.
- gateway: lark_cli_env_overlay(broker=True) omits config/data env;
sandbox_lark_broker_active() TTL-cached mode resolver; broker added
to sandbox_runtime_mode / readiness and the settings UI.
Opt-in and off by default (empty LARK_CLI_BROKER_IMAGE ⇒ no change).
Closes#4338
* fix(lark): address Pattern B broker review findings (#4501)
Follow-up to the sidecar credential broker addressing the PR #4501 review:
- shim: split the on-PATH lark-cli into a /bin/sh launcher + Python shim body
so broker mode fails loudly (exit 127, actionable message) instead of ENOEXEC
when the sandbox image ships no python3; interpreter pinnable via
DEERFLOW_LARK_BROKER_PYTHON. Launcher bakes in the shim's absolute path since
$0 is the bare command name when run off PATH.
- broker: drop the dead cwd payload field (broker can't see the sandbox FS) and
document the command-surface-only / no-file-IO limitation.
- broker: return a structured 500 JSON on unexpected exec errors so the shim
gets a meaningful message, not an opaque transport failure; set a handler
socket timeout to bound slow/stuck connections.
- broker: add an opt-in DEERFLOW_LARK_BROKER_DENY_SUBCOMMANDS denylist that
refuses secret-dumping subcommands before spawning the binary, forwarded from
the provisioner sidecar.
- gateway: tighten the per-bash-call broker probe timeout (1.5s) and cache
negatives longer (300s) so non-broker remote-provisioner users don't pay a
latency hit; guard the mode cache with a lock; drop the dead
_probe_provisioner_lark_cli_init_image wrapper.
- docs: remove the broken design-doc link from the broker README.
Adds tests for launcher python resolution, cwd omission, denylist enforcement,
500-on-error, hot-path probe timeout + negative caching, and provisioner
denylist-env wiring.
* feat(clarification): structured form fields for human-input cards
Add a request-side v2 `form` mode to the ask_clarification protocol so
business flows (e.g. expense reimbursement) can collect several values
in one card instead of sequential free-text questions:
- `ask_clarification` gains a restricted `fields` parameter (text /
textarea / number / select / multi_select / checkbox / date)
- ClarificationMiddleware validates and normalizes fields explicitly
(whitelisted types, unknown -> text, select-likes without options ->
text, duplicate/invalid entries dropped, all-invalid falls back to
the legacy modes) since the middleware short-circuits before tool
execution; the plain-text fallback lists fields for IM channels
- Form payloads carry `version: 2` so older frontends degrade to the
text fallback; replies stay on the v1 response protocol — the card
submits a readable summary as `response_kind: "text"`, so journal
persistence and answered-card recovery are unchanged
- Frontend renders typed field controls with required-field validation
and compact multi-select chips
Part of #4400 (scope narrowed per maintainer feedback: request-side
only, no new response kinds, no top-level multi_choice).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(clarification): harden form protocol per review feedback
Address the five review points on #4406:
- Reject field names colliding with JS Object.prototype members on both
sides; frontend reads form values via own-property access only, so
`constructor`/`toString`-style names can no longer leak inherited
members into required validation or the submitted summary
- Close open requests answered through the legacy text fallback: a
visible plain human reply (no response metadata) now marks every
previously-opened request as answered, so upgrading to a v2-aware
frontend cannot leave the composer locked on an already-answered card
- Give checkbox fields deterministic boolean semantics: values are
seeded to an explicit false ("no" in the summary) and `required` means
must-agree/consent; documented in the tool schema
- Make middleware field validation atomic: structurally broken entries
(bad/duplicate/reserved names, over-cap field/option counts or text
lengths) degrade the whole form instead of silently dropping fields;
options are trimmed/deduped with blanks removed so the backend never
emits payloads the frontend parser rejects
- Associate form labels/controls (htmlFor/id), aria-required,
aria-invalid, and error descriptions for accessibility
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(clarification): type the fields item schema via TypedDict
Replace `fields: list[dict[str, Any]]` with `list[ClarificationFormField]`
(a TypedDict with `name` required and the type whitelist as a Literal) so
the provider-facing tool schema documents the item shape instead of an
opaque object relying on the docstring. Runtime validation is unchanged
and stays in ClarificationMiddleware, which intercepts the call before
tool execution. Addresses the non-blocking review suggestion on #4406.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(frontend): drop unsupported aria-invalid from multi-select group
jsx-a11y: role=group does not support aria-invalid; the error linkage
stays via aria-describedby.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(clarification): coerce numeric required flags and normalize fields once
- `_normalize_bool` now coerces 1/0 (some providers serialize booleans
as integers), so `required: 1` no longer silently flips to optional
- `_handle_clarification` normalizes `fields` once and passes the result
to both the text fallback and the payload builder
Addresses the non-blocking review nits on #4406.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(clarification): harden form protocol per contract review round 2
Backend:
- Guard unhashable JSON in the intercept path: `type: []`/`{}` degrades
the field to text and `clarification_type: []` coerces to str instead
of raising TypeError (which, with return_direct, ended the turn with
an error and no card or fallback)
- Add a total budget over the serialized normalized fields (16KB UTF-8
bytes): per-item caps alone admitted forms whose IM text fallback
exceeded channel delivery limits (Slack 40k chars, Feishu ~30KB card),
silently truncating trailing fields; a boundary test proves any
accepted form's fallback stays deliverable
Frontend:
- Submission value now appends a JSON block keyed by stable field names
(readable summary alone is delimiter-ambiguous), with a collision
regression test
- Parser boundary tightened to match backend constraints: empty option
values (Radix SelectItem crash), duplicate option ids/values,
duplicate field names, and the form<->version-2 binding are rejected
- Keep the error node mounted while any field is still invalid so
aria-describedby never points at a removed element (happy-dom
interaction test)
- Required semantics are now accessible: native checkbox control (no
HTML required attribute — it would intercept the custom submit path),
visually-hidden localized "required" markers next to the aria-hidden
asterisks
- Legacy-fallback closure narrowed to the latest unanswered request:
nothing guarantees a single outstanding clarification across runs, and
closing all would silently swallow older decisions; an older request
left open becomes the active card again
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(frontend): keep clarification selects controlled
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(frontend): pin recent chats
* fix(threads): address pin-chat review feedback
- Stop bumping updated_at on metadata-only PATCH (pin/unpin) via a new
update_metadata(touch=False) path so unpinning no longer jumps a chat
to the top of the updated_at-sorted recent list.
- Narrow patchThreadMetadata to a ThreadMetadataPatchResponse matching
the Gateway's actual response (no values/context).
- Namespace the pinned metadata key as deerflow_pinned for consistency
with deerflow_sidecar / deerflow_branch.
- Cover touch/touch=False behavior in repo + router tests; document the
e2e mock's updated_at preservation now mirrors production.
* style(frontend): format thread utils test
* fix(threads): make pinned ordering server-side
* test(frontend): keep infinite-scroll fixture order stable
* test(frontend): stabilize lark reconnect e2e
* docs: clarify thread pin metadata contract
* fix(auth): recover from setup status timeouts
* test(auth): cover setup status recovery flows
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
