* feat(authz): gate composer send on runs:create (Phase 4 follow-up, #4063)
Mirrors #5294's stop gating on the send side: both chat routes pass
canCreateRuns (from PERMISSIONS.RUNS_CREATE, lockstep with the backend
enum) into the shared composer. The gate sits at the top of
submitThreadMessage — the single choke point every composer entry
(submit button, Enter, goal-set-triggered run) funnels through — and
denies with a toast plus a rejected promise so PromptInput keeps the
text. The idle submit button is disabled and explains the boundary via
conditionally-spread aria-label/title (startTurnUnavailable, en/zh);
while streaming the button stays the runs:cancel stop affordance.
Also removes the unreachable kind === "stop" branch in handleSubmit
(flagged during #5294's review): the Enter path early-returns with the
streaming toast before the classifier runs.
* fix(frontend): reject denied goal starts before saving state
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): scope skill suggestions by agent
Filter composer slash-skill suggestions through the active custom agent allowlist so restricted agents do not offer unavailable skills.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(frontend): wait for agent scope before draft hydration
Keep agent loading distinct from explicit empty and inherited skill scopes so saved skill chips are restored only after the active scope settles.
Co-Authored-By: Claude Code <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
* feat(projects): Projects MVP Phase 2 — instructions, document shelf, promotion, trash
Implements docs/superpowers/specs/2026-09-12-projects-mvp-phase2-design.md
(issue #5160, tracker #5129) in the slice order of the spec's §16.
Slices:
- A: ProjectsConfig + write-time 422 UTF-8 byte cap; PROJECT_CONTEXT_KEY
admission pinning (both server-owned sets + worker hoist); latest-only
request-scoped <project> block via DynamicContextMiddleware
wrap_model_call/awrap_model_call (idempotent reassembly, reserved ID
prefix + marker + provenance, never persisted); journal audit
fingerprints; Instructions tab.
- B: ProjectDocumentRow + migration 0023; ProjectDocumentRepository with
locked check-and-set; hash-qualified immutable shelf storage with
Paths helpers; upload/list/content/delete-to-trash routes; project
delete trashes the shelf in-transaction; request-scoped bounded
<documents> index with honest count/shown + actionable overflow note;
list_project_documents/read_project_document tools registered only on
pinned runs; PAT allowlist + drift guards; blocking-IO anchors.
- C: shared thread-upload ingestion service (uploads router refactored to
parity); POST from-thread with provenance; attach-to-thread with
lock-staged copy (archived source allowed); read-only thread-files
view with per-group truncation reporting.
- D: restore (restored/merged/not_found/no_target/content_missing; no
file moves), purge (continuous row lock across unlink/delete/commit,
retryable on FS errors), retention sweep (lazy + startup, 24h orphan
guard, row-side reconciliation never deletes).
- E: Documents tab (shelf + conversation-files browser, provenance,
archived banner, content-missing rows), /workspace/trash route,
sidebar entry, composer attach handoff, i18n (en-US/zh-CN), e2e mocks
+ specs.
Review hardening folded in (10 rounds, all with tests):
- force active shelf content (HTML/XML family) to download; nosniff on
artifact + content responses; unified unsandboxed-iframe PDF preview
(fixes the pre-existing Chromium sandbox blank in the artifact viewer)
- scope document trash to the URL project under the document lock
- atomic no-overwrite filename reservation for ALL ingestion (seeded
claims + os.link commit with suffix retry; same-name re-upload now
unique-names instead of replacing); hidden staging only, no visible
placeholders; lease cleanup on setup failure
- serialize conversion under the document lock with post-lock active
revalidation; drain locked filesystem work on cancellation; preserve
bytes when an insert's commit state is uncertain (including trashed
rows)
- original-integrity checks before serving text or cached conversions;
content_missing surfaced in list responses (UI reads the flag, no
409-probe); downloads always serve original bytes
- bounded streaming document reads with cached char counts; shelf limits
declared in middleware release identity
- thread-root confinement for from-thread sources; config fallback
rejects fractional/infinite values; composer counts staged
attachments; pending attachments persist until submission or removal;
in-flight instruction/rename edits survive save refetches; shelf and
trash pagination; conversation-file and thread-files pages stay
subscribed to refetches
Docs: README/README_zh, backend API.md/ARCHITECTURE.md, AGENTS.md
contracts, config.example.yaml projects block.
Review follow-ups (head b4807477 → this revision):
- The trash retention sweep is split so repeated lazy triggers stay
bounded: the indexed expiry purge still runs on every trigger
(GET /api/trash/documents, POST /api/trash/purge) while the
O(all rows + all files) reconciliation is throttled to one run per
user per 15 minutes (process-local, per-user window). The startup
sweep now runs as a background task instead of blocking gateway
readiness, and shutdown awaits it (bounded).
- The export scrub (stripInternalMarkers) is fence- and indentation-aware
like the render path, so a pasted, fenced <project>/<documents> snippet
survives markdown export while real injected blocks (never fenced) are
still removed. Fence regexes moved to a dependency-free leaf module to
avoid the messages↔streamdown import cycle.
- The artifact viewer's PDF iframe no longer carries an added title
attribute (the upstream e2e contract locates it via :not([title])), and
the upstream artifact-preview spec now pins the new contract: PDFs
render unsandboxed, images keep sandbox="".
* fix(projects): round-2 review — cancel an overrun trash sweep, restore the PDF frame title
- Shutdown cancelled only the shield around the background startup sweep,
so an all-users reconciliation that outlived the 5s budget kept walking
rows and files while the document repo and DB engine were disposed
underneath it. The wait now lives in `_shutdown_startup_trash_sweep`,
which cancels the task and drains it before worker exit: the shield
keeps the wait bounded, the cancel makes it final (CancelledError lands
at the sweep's next await, and `_run_startup_trash_sweep` only catches
`Exception`, so nothing swallows it).
- The browser-preview iframe lost `title={getFileName(filepath)}` in the
previous fix round, leaving the PDF frame without an accessible name
while its siblings keep theirs. Restore it (WCAG frame titles), assert
it in the DOM test, and anchor the e2e on `iframe[title="report.pdf"]`
instead of `iframe:not([title])`.
* fix(projects): round-3 review — report the sweep's late finish, not a phantom cancel
`Task.cancel()` returns False when the sweep already finished inside the
window between the deadline firing and the cancel, so the shutdown log
claimed a cancellation that never happened. Branch on that outcome: the
warning stays for a real cancel, a late finish is logged at info, and both
paths still reap the task before worker exit.
* fix(projects): round-4 review — make Empty trash delete what it confirms
`POST /api/trash/purge` only ran the retention sweep, and the sweep's
candidate selection is age-gated, so a freshly trashed document survived
"Empty trash" even though the confirmation promises that every listed
document is permanently deleted. With one trashed row the route answered
`{"purged": 0}` and left it in place; `GET /api/trash/documents` sweeps
expired rows before listing, so the visible rows were normally ineligible
for the action by construction.
Empty trash now drives `purge_all_trashed`: the caller's trashed rows
(`list_all_trashed`, no age filter) each go through the same guarded,
row-locked `purge` as the single-document delete — bytes first, then the
row, in one transaction — so a row restored mid-flight is skipped instead of
force-deleted, and an unlink failure rolls that row back and answers 500 with
a retryable message. Retention expiry stays where it was: the sweep's
`purge_candidates` is now the only age-gated selection, and the lazy
retention sweep still runs on the listing and at startup.
Tests: the router suite replaces the retention-gated expectation with the
reviewer's repro (fresh row purged, bytes unlinked, shelf and other users'
trash untouched, a failing unlink stays retryable and 500); a blocking-I/O
anchor drives the new entry point through the offload; the mocked e2e covers
the action end to end; a new real-backend spec performs it against the real
gateway and re-reads `GET /api/trash/documents`. README, API, ARCHITECTURE
and the phase-2 design docs (en+zh) state the age-independent contract.
* feat(settings): persist account preferences across browsers
* docs(settings): scope preference guidance to user persistence
* fix(settings): preserve SSR and fence custom-agent defaults
* test: include user persistence in scoped guidance inventory
* fix(settings): sync explicit edits and preserve local tab updates
* feat(authz): gate thread-delete and run-cancel UI on effective permissions (Phase 4, #4063)
Consume the effective route permissions surfaced by #5228 so the UI hides
actions the caller's role cannot perform:
- threads:delete hides the sidebar thread-row Delete menu item and the
sidecar panel delete button (every useDeleteThread consumer)
- runs:cancel disables the composer stop affordance; all three stop entry
points converge on one check inside handleStopStreaming
hasPermission treats an absent/null/unresolved permission list as
permissive, so a mixed old-backend/new-frontend deploy never hides actions
the caller can still perform. The Gateway @require_permission guards
remain the single enforcement point.
* fix(authz): review follow-ups for stop gating (comment accuracy, a11y, tests)
- Correct the defense-in-depth comment: the submit-button click is the
only live entry into handleStopStreaming (handleSubmit returns early
with the pleaseWaitStreaming toast while streaming, so the kind==="stop"
branch is unreachable); the handler gate stays as future-proofing.
- Explain the disabled stop affordance with aria-label + title (Radix
tooltips don't fire on disabled buttons), with en-US/zh-CN strings.
- Add the composer stop-gating DOM tests (disabled + onStop never fires +
permissive default) and the sidebar delete-menu gating tests, so all
gated surfaces carry wiring tests.
* fix(authz): stop conditional aria-label from stripping the submit name
The stop-gating follow-up (1612855b) explained the disabled stop
affordance with aria-label/title but passed explicitly-undefined
values in the non-denied case. PromptInputSubmit declares its default
aria-label="Submit" before {...props}, so the undefined key landed in
the spread and clobbered the default: React omits the attribute
entirely and the submit control lost its accessible name in every
state, which broke the sidecar e2e layout helper (it locates the
button by its "Submit" label).
Spread the attributes conditionally so they only attach when
stopDenied, and lock the invariant with a DOM test asserting the base
"Submit" name survives when stop is not denied (mutation-verified:
reverting the conditional spread turns the new test red).
* test(authz): drop unused rerenderWith helper, guard accessible name by role query
Address the review nit on the stop-gating DOM tests: the
rerenderWith helper was never called, and a second render() would
append a composer instead of updating the first one anyway — drop it
(the sidecar-delete-gating tests already demonstrate the correct
rerender pattern if a granted->denied flip test is ever needed).
Also resolve the accessible-name regression guard through
getByRole("button", { name: "Submit" }) so it fails exactly the way
e2e and assistive tech consume the control (mutation-verified: the
explicitly-undefined aria-label form turns it red).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): add interval schedule type
Allow scheduled tasks to fire every N seconds from last dispatch, not
only wall-clock cron or a single run_at. Cadence is UTC now+N with no
missed-beat catch-up, bounded by min_once_delay_seconds and 30 days.
* fix(scheduler): let interval tasks create, edit, and keep next run
Create/edit now keep every_seconds. Unchanged interval spec no longer
resets next_run_at, including timezone-only PATCH.
* fix(scheduler): keep non-minute intervals on edit
Stop rounding every_seconds to whole minutes in the form. Values that
are not whole minutes or hours now use a seconds unit so edit/duplicate
round-trips the stored cadence instead of rewriting it and resetting
next_run_at. Document that min_once_delay_seconds is also the interval
floor.
* fix(scheduler): clamp interval seconds to the default 60s floor
The new seconds unit allowed 1–59, which the API rejects under the
default min_once_delay_seconds. Clamp the form to >= 60 and show the
floor next to the preview. Also mention interval in the scheduler
field_doc, matching config.example.yaml.
* fix(scheduler): do not clamp interval amount while typing
Keystroke clamp made 90 become 9 -> 60, then 600, and backspace could
not leave 60. Keep the raw field text and apply the 60s floor on blur
and emit only.
* test(scheduler): cover interval input editing
* fix(frontend): preserve saved interval cadence until edited
* style(tests): format scheduled task router tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(scheduler): let scheduled tasks pin a custom agent
Create and update accept optional assistant_id, defaulting to lead_agent.
Custom names are normalized and must already exist for the task owner.
The workspace form exposes the same choice, and duplicate copies it.
Fixes#5286
* fix(scheduler): keep assistant-id PR free of interval tests
Drop the six interval tests that belonged to the interval schedule PR
and fail here because this tree still only accepts once/cron.
Treat lead_agent case-insensitively so LEAD_AGENT / lead-agent store
as the default. Omit unchanged assistant_id on edit so a deleted custom
agent does not 422 unrelated PATCH (rename, reschedule).
* fix(scheduler): format task page and browser tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): support standalone demo APIs and runtime GitHub stars
* Update API origin URL to use environment variables
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* test(frontend): align static demo tests with runtime origin
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* feat(projects): project workspaces with scoped chats and thread membership
Backend:
- projects table model and migration; fail-closed ProjectRepository with
ownership checks, CRUD/archive/restore/delete router, and atomic thread
move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
metadata; project-aware thread create/search with pagination bounds and
membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
at write time and dropped when invalid; serialized against project
deletion and thread assignment
- branch creation inherits the source thread's project membership (an
archived/deleted project degrades the branch to unassigned instead of
failing the request)
Frontend:
- projects data layer, thread move API, and sidebar projects section with
flat/grouped modes, archived-project threads, and stable virtual-list
offsets
- project detail page with project-scoped new chat
(/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
continuation is dropped instead of saving the goal or launching the
abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
an open project page never keeps stale titles, recency, or pagination
* fix(chats): keep archive undo toast when the sidebar row unmounts
The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.
Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.
* fix(projects): pin project thread listing contract and exclude archived chats
GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.
The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.
Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.
* docs(migrations): record the 0019/0020 chain against the bootstrap reservation
The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.
Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.
* fix(chats): invalidate project thread lists on archive/restore
useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.
An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.
Regression test asserts the project-list prefix is invalidated on success.
* fix(projects): fetch project discovery only in grouped sidebar mode
RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.
Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.
* fix(threads): fail closed on project-scoped create in memory mode
MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).
Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.
Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.
* fix(projects): window the project page thread list
ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.
Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.
* fix(projects): restore row dividers and pin them with a render test
The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.
The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.
* fix(projects): validate forward schemas and fence membership reads
* feat(frontend): render markdown artifacts in the new window
The artifacts panel's "open in new window" action handed the browser the
raw Gateway response. For markdown that is a `text/markdown` body the
browser can only show as source, so the new window was a text dump rather
than a reader.
Route markdown artifacts to a new `/artifacts/view` page that renders them
with the same components the panel uses (SafeStreamdown + the artifact
rehype chain + citation links/panel), including the truncated-preview
banner and its "load full file" action. Everything else keeps the raw
Gateway URL — notably HTML/SVG, which the Gateway deliberately serves as a
download so active content never executes in the application origin.
- `core/artifacts/viewer.ts` centralizes which stored artifacts are
markdown (`.skill` archives included, since they hold a SKILL.md), so
the panel and the viewer route cannot drift.
- `ArtifactFilePreview` and its siblings move out of
`artifact-file-detail.tsx` into `artifact-file-preview.tsx`; otherwise
the standalone route would pull the CodeMirror editor into its bundle.
- The window title comes from the route's `generateMetadata`, not
`document.title`, which the App Router overwrites after hydration.
- The viewer reads content through `useStandaloneArtifactContent`, which
shares `useArtifactContent`'s query key but not its `useThread`
dependency, since a detached window has no thread context.
Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC
* fix(frontend): keep the artifact target across re-authentication
Review found the standalone viewer unrecoverable from an expired session.
The window's target lives entirely in `?path=...&thread_id=...`, and both
auth paths dropped it:
- The layout guard redirected to `/login` with no `next` at all. A layout
cannot read `searchParams`, so the guard moves into the page, which can
— and rebuilds the full viewer address for `next`. The layout loses its
AuthProvider along the way: nothing under this route reads `useAuth`,
and the guard now makes a single `getServerSideUser` call per request.
- The shared fetch wrapper built `next` from `window.location.pathname`,
which silently truncated the query string. It now carries `search` too,
so any route holding state in the query survives a 401, not just this
one. `validateAuthNextPath` already accepts a query string.
`buildArtifactViewerURL` is split out of `resolveArtifactOpenURL`: the
guard needs the route itself, never the Gateway fallback that the latter
takes for non-markdown targets.
Tests: the login round trip (unit — the rebuilt URL survives
`validateAuthNextPath` and parses back to the same target), the fetch
wrapper preserving the query on 401 (unit), and the expired-session
window reaching `/login` with the artifact intact (E2E). The E2E asserts
on the popup's navigation *requests*, since `(auth)/layout` answers
`/login` with a server redirect under DEER_FLOW_AUTH_DISABLED and no
navigation commits.
`tests/unit/core/models/api.test.ts` stubbed `window.location` without
`search`; a real Location always has it.
Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC
* fix(frontend): keep public showcase artifacts out of the auth gate
Review found that the viewer's access check regressed `/showcase`. Those
pages render with `isMock`, their artifacts are served by the
unauthenticated demo route, and the raw artifact URL this window replaced
stayed public — so gating the window unconditionally bounced every
logged-out showcase visitor to /login for a document that is already
public.
`requiresAuthenticatedViewer` exempts a mock target only when
`resolveStaticDemoArtifact` would actually serve it. The allowlist is the
authority rather than the flag: `mock=true` is caller-supplied, so a
target the demo route answers with 404 — a non-allowlisted path, or a
thread that is not a demo thread — still needs a session.
Covered in `tests/e2e-auth/`, since the default E2E config disables auth
and cannot see this: a public showcase artifact renders without a
session, while a non-allowlisted path and a missing mock flag both land
on /login. Verified the positive case goes red without the exemption.
Claude-Session: https://claude.ai/code/session_013AiCrC5SBc3HdFYNxsp1EC
* feat: show real-time context window usage in chat UI (#3125)
Adds a `context_usage` block to `GET /api/threads/{id}/token-usage`
(token count from the live checkpoint, the thread model's
`context_window`, and a percentage), introduces a new
`ModelConfig.context_window` distinct from the per-call `max_tokens`
output cap, and surfaces the percentage in the chat header — inside
`TokenUsageIndicator` when token-usage tracking is on, or as a
standalone badge when it's off so context capacity stays visible
independent of cost tracking.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat: per-category breakdown for context window usage
Replace the single-number context_usage payload with a Claude-Code-style
breakdown — messages, system prompt, skills, system/MCP tools (active +
deferred), custom agents, memory injection, autocompact buffer, and free
space — and surface it in the chat UI with a segmented progress bar and
per-row table.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(config): document context_window across model examples
Add `context_window` to every example model in config.example.yaml so the
new chat-UI "% context used" indicator works out of the box for whichever
example a user adopts. Each value is the published default at the time of
writing; users are pointed at the official model spec to verify. Bumps
config_version to 11 so `make config-upgrade` flags outdated user configs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* style: ruff format (line-length 240)
No behavior change — collapses two multi-line expressions that fit on
one line under the project's 240-char limit. Picked up by `make format`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* review: address Copilot bot comments on #3183
- token-usage-indicator: switch `{contextPercentage && (...)}` to an
explicit `!= null` check. (The string `"0"` is actually truthy in JS so
the original code wasn't buggy, but the explicit check is clearer.)
- context-usage-breakdown: drop the `useMemo` around segments/totals — the
computation is O(n) over a handful of rows and the previous memo deps
omitted `t.contextUsage.categories`, so the bar's tooltips/aria-labels
could stay in the old language after a locale switch.
- context_usage._split_tools: snapshot MCP names from
`get_cached_mcp_tools()` directly instead of re-reading
`extensions_config.json` after `get_available_tools()` already loaded
it. Removes redundant file I/O on every `/token-usage` poll.
(`get_available_tools()` still emits its own INFO logs — silencing
those is out of scope here.)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* style(frontend): prettier --write context-usage-breakdown
CI's `pnpm format` (prettier --check) caught two lines previously
formatted by hand. Collapses one comma to fit on one line; no behavior
change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(gateway): correct context-usage breakdown + add exact token counting
The context-usage indicator shipped two bugs that silently zeroed whole
breakdown rows (both caught by try/except, so the feature looked alive but
produced wrong numbers):
1. _count_system_prompt passed app_config= to get_deferred_tools_prompt_section,
which only accepts deferred_names -> TypeError swallowed -> system_prompt
row always 0, and used_tokens/percentage undercounted by the full prompt.
Also subtracted the deferred section twice (the rendered prompt already
excluded it). Fix: derive deferred names deterministically and pass them to
apply_prompt_template; drop the redundant subtraction.
2. _split_tools imported a non-existent get_deferred_registry -> ImportError
swallowed -> all four tool-category rows always 0. Fix: classify via the
public is_mcp_tool predicate + tool_search.enabled (mirrors
build_deferred_tool_setup); the MCP tag is set by get_available_tools.
Added token_usage.counting (approximate|exact). 'exact' routes text/schema/
message counting through the model tokenizer (tiktoken cl100k_base) via the
existing memory-module machinery (lazy load + cache + cooldown + CJK-aware
fallback), so CJK-heavy threads stop being undercounted by chars//4.
Regression + e2e tests added; 6621 backend tests pass.
* fix(gateway): harden context usage accounting
* fix(gateway): count promoted MCP tools as active in context usage
Promoted tools (deferred MCP tools the thread has fetched via tool_search)
have their full schema bound on every subsequent turn by
DeferredToolFilterMiddleware, so they consume context like any active tool.
The breakdown previously left them in the reserved *_deferred rows, under-
counting the thread's used_tokens.
Classification now treats a tool as deferred only when tool_search is enabled,
it is MCP-sourced, AND it has not been promoted. The promoted set is read from
the checkpoint's channel_values and scoped by catalog hash — matching the
runtime middleware, so a stale promotion from MCP-config drift cannot inflate
the active count.
The static system prompt still lists all deferred tool names (promotions only
affect schema binding, not the prompt), so _count_system_prompt's deferred
rendering is intentionally left unchanged.
8 new tests cover classification, catalog-hash scoping (match / drift /
compute-failure / malformed), and checkpoint extraction.
* fix(context): address review feedback
* fix(context): count structured message payloads
* fix(context): harden usage accounting
* fix(config): bump schema for context usage fields
* refactor: narrow context usage to core indicator
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(frontend): allow chat replies during clarification
* fix(frontend): unlock input polish during clarification
Remove hasOpenHumanInputCard from inputPolishDisabled so the polish
button stays available when a clarification card is open, matching
the composer unlock behavior. Clean up the now-unused useMemo and
import.
* fix(auth): recover from setup status timeouts
* test(auth): cover setup status recovery flows
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: add lark cli integration
* fix: polish lark integration actions
* feat: support lark incremental permissions
* fix: detect lark authorization completion
* fix: harden lark integration install
* feat: expand lark auth scopes and reuse host auth in sandbox
Default lark auth to least-privilege (recommend=false, base sign-in only)
and expose the full set of lark-cli --domain business domains as native
--domain grants instead of a 4-domain read-only mapping. Resolve the
skill pack from the latest larksuite/cli GitHub release at install time
with content-hash integrity, and surface version/runtime drift in status.
Share the per-user lark-cli config/data profile between the Gateway
Settings auth flow and agent conversations by mounting the integration
dirs into the AIO sandbox and injecting the matching env for lark-cli
commands, with an allowlisted extra_mounts path in the provisioner/K8s
backend and traversal guards on integration paths.
* style: fix lint issues from ruff and prettier
Sort imports in the provisioner PVC test and re-wrap two long i18n
description strings to satisfy backend ruff and frontend prettier CI.
* fix(lark): address managed integration review feedback
* fix(frontend): stabilize integrations settings e2e
* test(sandbox): isolate remote backend legacy visibility check
* test: fix backend unit failures after merge
* Harden Lark integration review fixes
* Format Lark integration E2E test
* fix(lark): harden sandbox credential exposure and status disclosure
Address willem_bd's security review on PR #3971:
- Mount the per-user lark-cli config dir (long-lived appSecret) read-only
into the AIO sandbox; only the refreshable-token data dir stays writable.
- Redact host filesystem paths (install_path, cli.path) from
GET /lark/status and the config/auth complete responses for non-admin
callers, fail-closed on any auth error.
- Document the npm postinstall trade-off (--ignore-scripts is not viable
because @larksuite/cli fetches its platform binary in postinstall).
- Document the sandbox credential trust boundary in AGENTS.md and README,
pointing at the sidecar-broker follow-up (#4338).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(agents): per-agent model and generation settings
Let each custom agent choose its own model and sampling settings
(temperature, max_tokens) plus thinking / reasoning_effort defaults,
so agents sharing a model profile are no longer stuck with one shared
temperature and output length (#4336).
AgentConfig gains optional model_settings / thinking_enabled /
reasoning_effort (None = inherit). create_chat_model applies per-caller
model_overrides on top of the profile before the thinking/Codex
transforms; the lead agent resolves each knob with precedence
request > agent config > profile/default. The /api/agents create/update
routes persist the fields and reject an unknown model. The default lead
agent path is unchanged (no agent config -> overrides None). The agent
chat composer also stops force-overriding an agent's configured default
model with models[0].
* fix(agents): tri-state thinking control and default-model capability gating
The model-settings dialog seeded the thinking switch to false, so opening
it to tweak temperature and saving silently disabled thinking (the runtime
default is on) with no way back to inherit. It also hid the thinking /
reasoning controls whenever the agent inherited the global default model,
since `__default__` never resolved through `models.find`.
Give thinking an explicit Inherit / On / Off tri-state so an untouched save
is a no-op, and resolve `__default__` to the effective default (models[0])
for the capability check. Logic lives in the tested helpers module.
* feat(browser): add agentic browser control
* fix(frontend): format browser view changes
* fix(browser): keep browser optional and isolate sidecar layout
* fix(browser): address PR review security and IME findings
- Nginx: add a browser-stream WebSocket location before the generic
/api/threads regex so Live upgrades instead of downgrading to HTTP
(both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
navigate, and tear down the browser session on thread deletion so a
later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
context-level route guard covering redirects, popups, iframes, and
subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
CJK candidate with Enter no longer submits the remote page form.
Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.
* fix(frontend): smooth streaming in long tool threads
* Revert "fix(frontend): smooth streaming in long tool threads"
This reverts commit f0462516eabe77f138d4027ea1c714fb226683cf.
* fix(browser): address review security and lifecycle findings
- Reject cross-origin WebSocket upgrades on the live browser stream
(Origin allow-list reuse of CORS/same-origin helpers) to close a
WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
(browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
can't permanently stall after the cumulative attempt cap.
* fix(frontend): reduce long tool thread render stalls
Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.
* fix(browser): keep live control responsive during continuous input
Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").
What:
- Frontend forwards one `click` per physical click instead of also emitting
`down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
start a rate-limited background refresh loop (leading frame + bounded cadence)
that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
inputs coalesce, and continuous input keeps refreshing before it stops.
Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.
* fix(browser): harden worker and session lifecycle
* fix(browser): address latest review feedback
* fix(frontend): preserve optimistic new-chat message
* test(e2e): preserve mocked message run ids
* fix(browser): address capability review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): harden artifact and markdown rendering
* fix(#4117): restore allow-scripts for scroll-restore, fix citation XSS bypass
willem-bd identified these issues:
1. [REGRESSION] sandbox removed allow-scripts which broke HTML artifact
scroll-restoration — the injected postMessage script could not run.
The prior config (allow-scripts allow-forms without allow-same-origin,
i.e. opaque origin) was already safe. Restored with corrected comment.
2. [XSS bypass] CitationLink rendered <a href={href}> directly before
isSafeHref ran, so prompt-injected [citation:x](javascript:...)
bypassed the safety check. Moved citation block after the guard.
Also:
- Added mailto: and tel: to SAFE_HREF_PROTOCOLS
- Kept anchor-only attributes off the <span> fallback (React DOM warning)
Note: the loadMessages re-throw originally in this commit was dropped
during the rebase — upstream #4065 rewrote useThreadHistory as a
TanStack useInfiniteQuery that already surfaces fetch failures.
* test(e2e): update sandbox assertion to match allow-scripts allow-forms
The artifact preview iframe's sandbox was restored to 'allow-scripts allow-forms'
for scroll-restoration. Update the E2E test to expect the corrected value.
* fix(#4117): address review — ArtifactLink XSS guard, urlOfArtifact sandbox e2e
- Apply the isSafeHref guard to ArtifactLink (artifact-link.tsx) so a
prompt-injected javascript:/data: href in a .md artifact preview cannot
reach a real anchor in the main document, matching the guard in
createMarkdownLinkComponent.
- Add an e2e asserting the urlOfArtifact iframe (non-code
browser-previewable files like PDF) keeps its empty sandbox.
Note: the loadMessages error-surfacing changes originally in this commit
were dropped during the rebase — upstream #4065 rewrote useThreadHistory
as a TanStack useInfiniteQuery whose queryFn already throws on
!response.ok and surfaces the failure via a toast.
* fix(frontend): let write_file non-code previewable artifacts render in sandboxed iframe
When a write_file artifact such as PDF is clicked in chat, the component receives a path that forced isCodeFile=true, hiding the sandboxed iframe behind the code editor. Now non-code browser-previewable files are detected early so the sandboxed iframe renders correctly.
Fixes the E2E test: renders sandboxed iframe for a browser-previewable non-code file.
* fix(#4117): align PDF artifact route pattern with convention (drop /mock/ prefix)
The test used /mock/api/threads/... for the PDF artifact content route,
but the urlOfArtifact helper generates /api/threads/... (without /mock/)
when isMock=false. The other tests (e.g. presented artifacts) already use
the correct /api/threads/... pattern.
* test(frontend): add render-level coverage for unsafe markdown/artifact links
- Render MarkdownLink and ArtifactLink via renderToStaticMarkup and
assert an unsafe javascript: href produces a disabled <span> (never an
<a>), including through the citation-labelled branch, and that safe
https hrefs render hardened anchors (target=_blank, rel=noopener).
- The new ArtifactLink render test caught the unsafe href leaking onto
the fallback <span> through the {...rest} spread; drop the spread in
both span fallbacks so anchor-only attributes (href/target/rel) and
react-markdown's node prop never reach the span.
- Cover mailto:/tel: in the isSafeHref unit cases and fix the stale
SAFE_HREF_PROTOCOLS docstring that still claimed http/https-only.
* fix(frontend): route the agent save-hint through the safe localStorage facade
Export safeLocalStorage from core/settings/local and use it for the
agent-create save-hint read/write so blocked browser storage (Safari
private mode, strict containers, embedded WebViews) cannot throw from
the effect. core/agents/feature-cache.ts already guards its
localStorage access with try/catch, so settings + agent pages are now
consistently best-effort.
* fix(frontend): allow scheme-less relative markdown links in isSafeHref
* feat(frontend): add structured human input cards for ask_clarification
Implement a reusable Human Input Card flow for ask_clarification while keeping
the existing text fallback for older clients and IM channels.
Backend:
- Add structured ToolMessage.artifact.human_input payloads for clarification requests.
- Preserve ToolMessage.content as the readable Markdown/text fallback.
- Normalize clarification options from native lists, JSON strings, plain strings,
mixed scalar values, None, and missing options.
- Derive input_mode as choice_with_other when options exist, otherwise free_text.
- Keep disable_clarification non-interactive behavior as a plain ToolMessage with
no human_input artifact.
- Cover artifact persistence and Gateway message metadata preservation in tests.
Frontend:
- Add human input protocol types, runtime guards, extractors, response builders,
and thread-state helpers.
- Add reusable HumanInputCard with option buttons, free-text input, pending,
read-only, disabled, and answered states.
- Render structured clarification cards from artifact.human_input, with Markdown
fallback for malformed or legacy tool messages.
- Preserve line breaks in structured question/context/option text.
- Hide submitted clarification bridge messages from the chat UI via
additional_kwargs.hide_from_ui.
- Send structured human_input_response metadata through the fourth sendMessage
options argument, preserving run context in the third argument.
- Wire submissions for normal chats, custom agent chats, agent bootstrap chats,
and sidecar chats.
- Derive answered state from raw thread.messages so hidden replies still update
the original card.
- Clear pending state when the hidden reply arrives, dispatch is dropped, or a
later async stream failure appears on thread.error.
* perf(frontend): optimize HumanInputCard UI interactions
- Support Enter key to submit text input (Shift+Enter for newline)
- Render question and context fields as Markdown instead of plain text
- Replace deprecated FormEventHandler type with structural typing
* test(frontend): add unit test cover optimize HumanInputCard UI interactions
* feat(frontend): disabled chatbox when has new human-input-card
* fix(style): lint error fix
* fix: sanitize hidden human input replies
- Preserve IME composition safety for human input card Enter submits
- Treat hidden human input responses as genuine user messages for sanitization
- Keep hidden card replies in memory filtering while excluding malformed/internal hidden messages
- Add regression coverage for card IME handling and hidden reply sanitization
* fix: tighten human input response validation
- Reject empty hidden human input response values
- Remove invalid list ARIA role from human input card options
- Add backend coverage for empty response payloads
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
