* feat(clarification): structured form fields for human-input cards
Add a request-side v2 `form` mode to the ask_clarification protocol so
business flows (e.g. expense reimbursement) can collect several values
in one card instead of sequential free-text questions:
- `ask_clarification` gains a restricted `fields` parameter (text /
textarea / number / select / multi_select / checkbox / date)
- ClarificationMiddleware validates and normalizes fields explicitly
(whitelisted types, unknown -> text, select-likes without options ->
text, duplicate/invalid entries dropped, all-invalid falls back to
the legacy modes) since the middleware short-circuits before tool
execution; the plain-text fallback lists fields for IM channels
- Form payloads carry `version: 2` so older frontends degrade to the
text fallback; replies stay on the v1 response protocol — the card
submits a readable summary as `response_kind: "text"`, so journal
persistence and answered-card recovery are unchanged
- Frontend renders typed field controls with required-field validation
and compact multi-select chips
Part of #4400 (scope narrowed per maintainer feedback: request-side
only, no new response kinds, no top-level multi_choice).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(clarification): harden form protocol per review feedback
Address the five review points on #4406:
- Reject field names colliding with JS Object.prototype members on both
sides; frontend reads form values via own-property access only, so
`constructor`/`toString`-style names can no longer leak inherited
members into required validation or the submitted summary
- Close open requests answered through the legacy text fallback: a
visible plain human reply (no response metadata) now marks every
previously-opened request as answered, so upgrading to a v2-aware
frontend cannot leave the composer locked on an already-answered card
- Give checkbox fields deterministic boolean semantics: values are
seeded to an explicit false ("no" in the summary) and `required` means
must-agree/consent; documented in the tool schema
- Make middleware field validation atomic: structurally broken entries
(bad/duplicate/reserved names, over-cap field/option counts or text
lengths) degrade the whole form instead of silently dropping fields;
options are trimmed/deduped with blanks removed so the backend never
emits payloads the frontend parser rejects
- Associate form labels/controls (htmlFor/id), aria-required,
aria-invalid, and error descriptions for accessibility
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(clarification): type the fields item schema via TypedDict
Replace `fields: list[dict[str, Any]]` with `list[ClarificationFormField]`
(a TypedDict with `name` required and the type whitelist as a Literal) so
the provider-facing tool schema documents the item shape instead of an
opaque object relying on the docstring. Runtime validation is unchanged
and stays in ClarificationMiddleware, which intercepts the call before
tool execution. Addresses the non-blocking review suggestion on #4406.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(frontend): drop unsupported aria-invalid from multi-select group
jsx-a11y: role=group does not support aria-invalid; the error linkage
stays via aria-describedby.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(clarification): coerce numeric required flags and normalize fields once
- `_normalize_bool` now coerces 1/0 (some providers serialize booleans
as integers), so `required: 1` no longer silently flips to optional
- `_handle_clarification` normalizes `fields` once and passes the result
to both the text fallback and the payload builder
Addresses the non-blocking review nits on #4406.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(clarification): harden form protocol per contract review round 2
Backend:
- Guard unhashable JSON in the intercept path: `type: []`/`{}` degrades
the field to text and `clarification_type: []` coerces to str instead
of raising TypeError (which, with return_direct, ended the turn with
an error and no card or fallback)
- Add a total budget over the serialized normalized fields (16KB UTF-8
bytes): per-item caps alone admitted forms whose IM text fallback
exceeded channel delivery limits (Slack 40k chars, Feishu ~30KB card),
silently truncating trailing fields; a boundary test proves any
accepted form's fallback stays deliverable
Frontend:
- Submission value now appends a JSON block keyed by stable field names
(readable summary alone is delimiter-ambiguous), with a collision
regression test
- Parser boundary tightened to match backend constraints: empty option
values (Radix SelectItem crash), duplicate option ids/values,
duplicate field names, and the form<->version-2 binding are rejected
- Keep the error node mounted while any field is still invalid so
aria-describedby never points at a removed element (happy-dom
interaction test)
- Required semantics are now accessible: native checkbox control (no
HTML required attribute — it would intercept the custom submit path),
visually-hidden localized "required" markers next to the aria-hidden
asterisks
- Legacy-fallback closure narrowed to the latest unanswered request:
nothing guarantees a single outstanding clarification across runs, and
closing all would silently swallow older decisions; an older request
left open becomes the active card again
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(frontend): keep clarification selects controlled
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Opening the dev stack on a LAN address or a proxied hostname serves the
SSR HTML but never hydrates: Next.js answers /_next/*, /__nextjs_font/*,
and HMR with 403 for any host it was not started on. The page renders, so
it looks up — but no client handler is attached, and the login form's
onSubmit never fires. It reads as "login is broken" rather than as an
asset problem, and the only clue is a warning in the dev-server log.
Wire Next's allowedDevOrigins to a new DEER_FLOW_DEV_ALLOWED_ORIGINS env
var. Unset by default, so the localhost-only default is unchanged; it is
also dev-only, as Next ignores allowedDevOrigins in production builds.
Entries are reduced to the bare host that allowedDevOrigins matches
against, since an entry pasted from the address bar as
"http://192.168.1.10:2026" would otherwise match nothing and leave the
operator with the same 403 they were trying to fix.
Reported in #54 and #203.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): restore resizing for the artifacts and sidecar panels
#3934 replaced the right panel's ResizablePanelGroup with a fixed CSS grid to
animate open/close, which removed the drag handle; #4187 then reintroduced a
resizable group for the browser panel only. The artifacts and sidecar panels
have had no way to resize since, while the browser divider still drags.
All three right panels now share one panel group, so there is no per-panel-kind
layout fork. Open/close goes through the side panel's collapse()/resize() so the
width still animates, and a dragged width survives closing and reopening.
Three library-specific constraints, each found by a failing test:
- the size transition is applied from the group as
[&>[data-panel]]:transition-[flex-grow], because <ResizablePanel className>
lands on an inner wrapper while the element the library sizes is its own
[data-panel] div;
- reopening uses resize(remembered) rather than expand(), which falls back to
minSize until the library has recorded a size, and the remembered width is
read before collapse() because the closing animation reports shrinking sizes;
- during the animation the content is held at its final width in cqw and
clipped, as the previous grid layout did — letting it reflow every frame makes
the message list re-run its scroll-to-bottom and re-wraps the sidecar
composer.
Fixes#4465
* fix(frontend): remove unreachable panel max size
* feat(frontend): pin recent chats
* fix(threads): address pin-chat review feedback
- Stop bumping updated_at on metadata-only PATCH (pin/unpin) via a new
update_metadata(touch=False) path so unpinning no longer jumps a chat
to the top of the updated_at-sorted recent list.
- Narrow patchThreadMetadata to a ThreadMetadataPatchResponse matching
the Gateway's actual response (no values/context).
- Namespace the pinned metadata key as deerflow_pinned for consistency
with deerflow_sidecar / deerflow_branch.
- Cover touch/touch=False behavior in repo + router tests; document the
e2e mock's updated_at preservation now mirrors production.
* style(frontend): format thread utils test
* fix(threads): make pinned ordering server-side
* test(frontend): keep infinite-scroll fixture order stable
* test(frontend): stabilize lark reconnect e2e
* docs: clarify thread pin metadata contract
* fix(auth): recover from setup status timeouts
* test(auth): cover setup status recovery flows
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* perf(frontend): coalesce streaming renders to a frame budget instead of per chunk
While a run streams, the merge/group/render pipeline consumed every SSE chunk
as its own React update (~60/s), re-rendering the whole thread tree per token.
Enable the SDK's same-tick batching (throttle: true) and publish the
render-facing messages snapshot at most once per 80 ms with a leading edge and
a trailing flush, keyed through a memoized merge so identities stay stable
between flushes. Lifecycle consumers (optimistic clearing, summarization
capture, usage baselines) keep reading the per-chunk array.
* perf(frontend): keep the transient bridge order array identity stable
mergeTransientHistoryBridgeOrder cloned unconditionally, so the render-time
call handed the coalesced merge memo a fresh array identity on every render
while the transient history bridge was open, re-running mergeMessages between
flushes. Clone lazily and return the input order when nothing is appended; the
merge only ever appends, so an unchanged length means unchanged content.
Consumers only read the returned order, so reusing the input is safe.
* perf(frontend): drive the render coalescer from a monotonic clock
The coalescing interval was measured with Date.now(). A backward wall-clock
step (NTP correction, sleep/wake) turns the elapsed term negative, so the
scheduled delay becomes interval + jump and the rendered snapshot stalls for
the length of the jump. Read performance.now() once per effect invocation
instead; the timer callback re-reads it because timers fire late and the next
interval must start from the real flush.
Seed the last-flush marker with -Infinity so the first update of a stream
still takes the leading edge under a page-load-relative clock.
* perf(frontend): reset the coalescer flush baseline when a stream ends
The leading-edge flush was scoped to the hook instance rather than to each
stream: a run starting within one interval of the previous one found a recent
flush baseline and deferred its first frame. Drop the baseline when leaving
the streaming state so every stream opens on the leading edge.
* perf(frontend): disarm the trailing flush when the leading edge wins
decideCoalesce checks the elapsed interval before the pending-timer flag, so
an update arriving past the interval takes the leading edge while a trailing
timer is still armed. Timers fire late under main-thread load -- exactly the
regime this coalescer targets -- so that timer then publishes a second time
and slips the flush baseline forward, breaking the at-most-one-flush-per-
interval property when it matters most.
Disarm the pending timer in the flush-now branch, and cover the previously
untested elapsed >= interval && hasPendingTimer quadrant.
* perf(frontend): stop syncing the render snapshot while idle
The snapshot is only read while streaming, so keeping it current on every
idle messages change costs one wasted render per history refetch or thread
navigation. Dropping that publish outright is not safe either: the leading
edge runs in a passive effect, but the render where isStreaming flips true
paints first and returns the snapshot, so a stale one would be painted --
after a thread switch, another thread's messages, since the chat page
deliberately avoids re-mounting on navigation.
Make the snapshot nullable, where null means no snapshot belongs to the
current stream, and return the live array while it is null. The idle branch
then writes state once per stream end instead of once per idle update, and
the stale-frame window does not exist rather than being short.
* feat: add lark cli integration
* fix: polish lark integration actions
* feat: support lark incremental permissions
* fix: detect lark authorization completion
* fix: harden lark integration install
* feat: expand lark auth scopes and reuse host auth in sandbox
Default lark auth to least-privilege (recommend=false, base sign-in only)
and expose the full set of lark-cli --domain business domains as native
--domain grants instead of a 4-domain read-only mapping. Resolve the
skill pack from the latest larksuite/cli GitHub release at install time
with content-hash integrity, and surface version/runtime drift in status.
Share the per-user lark-cli config/data profile between the Gateway
Settings auth flow and agent conversations by mounting the integration
dirs into the AIO sandbox and injecting the matching env for lark-cli
commands, with an allowlisted extra_mounts path in the provisioner/K8s
backend and traversal guards on integration paths.
* style: fix lint issues from ruff and prettier
Sort imports in the provisioner PVC test and re-wrap two long i18n
description strings to satisfy backend ruff and frontend prettier CI.
* fix(lark): address managed integration review feedback
* fix(frontend): stabilize integrations settings e2e
* test(sandbox): isolate remote backend legacy visibility check
* test: fix backend unit failures after merge
* Harden Lark integration review fixes
* Format Lark integration E2E test
* fix(lark): harden sandbox credential exposure and status disclosure
Address willem_bd's security review on PR #3971:
- Mount the per-user lark-cli config dir (long-lived appSecret) read-only
into the AIO sandbox; only the refreshable-token data dir stays writable.
- Redact host filesystem paths (install_path, cli.path) from
GET /lark/status and the config/auth complete responses for non-admin
callers, fail-closed on any auth error.
- Document the npm postinstall trade-off (--ignore-scripts is not viable
because @larksuite/cli fetches its platform binary in postinstall).
- Document the sandbox credential trust boundary in AGENTS.md and README,
pointing at the sidecar-broker follow-up (#4338).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Frontend unit tests run in a plain node environment, so nothing that
renders can be covered and no hook can be tested under real React. The
workaround that forces is already in the tree: the use-global-shortcuts
test mocks out react itself, so its useEffect never re-runs on a
dependency change and never goes through React's commit/cleanup
ordering -- it pins the stub rather than the hook.
Split the suite into two rstest projects: *.test.ts(x) keeps running on
node, *.dom.test.ts(x) runs on happy-dom. A global DOM environment was
measured first and rejected -- it takes the same 743 tests from 3.3s to
9.5s -- and rstest has no per-file environment docblock, so projects is
the only way to charge that cost to the tests that need it.
useIsMobile is the first consumer: it had no tests and cannot have any
without a document, since it reads window.matchMedia.
* fix(frontend): strip and parse the <current_uploads> upload context tag
#4174 renamed the upload context block that UploadsMiddleware prepends to
the user message from <uploaded_files> to <current_uploads>, but the
frontend tag vocabulary was not updated, so the raw block (file list plus
usage guidance) rendered inside the user's chat bubble and the file-chip
content fallback stopped matching. Add the new tag to
stripUploadedFilesTag, INTERNAL_MARKER_TAGS (covers export and streamdown
preprocessing), parseUploadedFiles, and the chip fallback detection.
<uploaded_files> stays supported for history and IM-channel messages.
* fix(frontend): parse upload-context sizes to bytes for the file chip
parseUploadedFiles stored the raw leading number of the human-readable
size (e.g. parseInt("177.6 KB") -> 177) into FileInMessage.size, which
is documented as bytes. On the content-fallback chip path (history/IM
messages without additional_kwargs.files) formatBytes then rendered a
177.6 KB file as "0.2 KB".
Convert the backend's "<n> KB" / "<n> MB" form back to bytes so the chip
re-renders at the original magnitude; update the parse tests to assert
byte values.
* fix(frontend): keep leading orphan tool messages visible
#3880 stopped dropping orphan tool messages that arrive after a terminal
group, but left the leading-orphan branch dropping the message with a
console.error on every render. The case is reachable: history pagination
cuts by event seq, not turn boundaries, so the first loaded page can
begin mid-turn with tool results whose AI tool-call message sits on an
unloaded older page — in dev mode the diagnostic surfaces as a
full-screen Next.js error overlay (#4399). Open a processing group for
the leading orphan instead; loading the older page re-groups it under
its real turn.
* test(frontend): lock leading-orphan accumulation invariant; clarify self-heal comment
Address review on #4408:
- The self-heal note only holds for the pagination path. The
hidden-only-precedent case and a truly orphaned tool have no older page to
load, so the processing group persists and renders as an empty
ChainOfThought shell (convertToSteps emits steps only for type === ai).
Document that as an accepted degradation instead of implying it always
self-heals.
- Add a test locking the accumulation invariant: a leading orphan followed by
a real tool-call turn folds into one processing group, not a stranded empty
group beside the real turn.
* feat(agents): per-agent model and generation settings
Let each custom agent choose its own model and sampling settings
(temperature, max_tokens) plus thinking / reasoning_effort defaults,
so agents sharing a model profile are no longer stuck with one shared
temperature and output length (#4336).
AgentConfig gains optional model_settings / thinking_enabled /
reasoning_effort (None = inherit). create_chat_model applies per-caller
model_overrides on top of the profile before the thinking/Codex
transforms; the lead agent resolves each knob with precedence
request > agent config > profile/default. The /api/agents create/update
routes persist the fields and reject an unknown model. The default lead
agent path is unchanged (no agent config -> overrides None). The agent
chat composer also stops force-overriding an agent's configured default
model with models[0].
* fix(agents): tri-state thinking control and default-model capability gating
The model-settings dialog seeded the thinking switch to false, so opening
it to tweak temperature and saving silently disabled thinking (the runtime
default is on) with no way back to inherit. It also hid the thinking /
reasoning controls whenever the agent inherited the global default model,
since `__default__` never resolved through `models.find`.
Give thinking an explicit Inherit / On / Off tri-state so an untouched save
is a no-op, and resolve `__default__` to the effective default (models[0])
for the capability check. Logic lives in the tested helpers module.
The composer sent `/goal <objective>` of any length, so an objective past
the backend limit (`MAX_GOAL_OBJECTIVE_CHARS = 4000`) only failed after a
round-trip with a raw HTTP 422. Mirror the limit client-side: reject an
over-length objective before the PUT with a friendly toast, and reject the
submit so PromptInput keeps the user's text for editing instead of clearing
it. Add a live footer counter that surfaces near the limit and turns
destructive when exceeded. Length is measured with the same whitespace
normalization the backend applies, so the counts match exactly.
* feat(browser): add agentic browser control
* fix(frontend): format browser view changes
* fix(browser): keep browser optional and isolate sidecar layout
* fix(browser): address PR review security and IME findings
- Nginx: add a browser-stream WebSocket location before the generic
/api/threads regex so Live upgrades instead of downgrading to HTTP
(both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
navigate, and tear down the browser session on thread deletion so a
later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
context-level route guard covering redirects, popups, iframes, and
subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
CJK candidate with Enter no longer submits the remote page form.
Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.
* fix(frontend): smooth streaming in long tool threads
* Revert "fix(frontend): smooth streaming in long tool threads"
This reverts commit f0462516eabe77f138d4027ea1c714fb226683cf.
* fix(browser): address review security and lifecycle findings
- Reject cross-origin WebSocket upgrades on the live browser stream
(Origin allow-list reuse of CORS/same-origin helpers) to close a
WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
(browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
can't permanently stall after the cumulative attempt cap.
* fix(frontend): reduce long tool thread render stalls
Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.
* fix(browser): keep live control responsive during continuous input
Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").
What:
- Frontend forwards one `click` per physical click instead of also emitting
`down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
start a rate-limited background refresh loop (leading frame + bounded cadence)
that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
inputs coalesce, and continuous input keeps refreshing before it stops.
Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.
* fix(browser): harden worker and session lifecycle
* fix(browser): address latest review feedback
* fix(frontend): preserve optimistic new-chat message
* test(e2e): preserve mocked message run ids
* fix(browser): address capability review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): harden artifact and markdown rendering
* fix(#4117): restore allow-scripts for scroll-restore, fix citation XSS bypass
willem-bd identified these issues:
1. [REGRESSION] sandbox removed allow-scripts which broke HTML artifact
scroll-restoration — the injected postMessage script could not run.
The prior config (allow-scripts allow-forms without allow-same-origin,
i.e. opaque origin) was already safe. Restored with corrected comment.
2. [XSS bypass] CitationLink rendered <a href={href}> directly before
isSafeHref ran, so prompt-injected [citation:x](javascript:...)
bypassed the safety check. Moved citation block after the guard.
Also:
- Added mailto: and tel: to SAFE_HREF_PROTOCOLS
- Kept anchor-only attributes off the <span> fallback (React DOM warning)
Note: the loadMessages re-throw originally in this commit was dropped
during the rebase — upstream #4065 rewrote useThreadHistory as a
TanStack useInfiniteQuery that already surfaces fetch failures.
* test(e2e): update sandbox assertion to match allow-scripts allow-forms
The artifact preview iframe's sandbox was restored to 'allow-scripts allow-forms'
for scroll-restoration. Update the E2E test to expect the corrected value.
* fix(#4117): address review — ArtifactLink XSS guard, urlOfArtifact sandbox e2e
- Apply the isSafeHref guard to ArtifactLink (artifact-link.tsx) so a
prompt-injected javascript:/data: href in a .md artifact preview cannot
reach a real anchor in the main document, matching the guard in
createMarkdownLinkComponent.
- Add an e2e asserting the urlOfArtifact iframe (non-code
browser-previewable files like PDF) keeps its empty sandbox.
Note: the loadMessages error-surfacing changes originally in this commit
were dropped during the rebase — upstream #4065 rewrote useThreadHistory
as a TanStack useInfiniteQuery whose queryFn already throws on
!response.ok and surfaces the failure via a toast.
* fix(frontend): let write_file non-code previewable artifacts render in sandboxed iframe
When a write_file artifact such as PDF is clicked in chat, the component receives a path that forced isCodeFile=true, hiding the sandboxed iframe behind the code editor. Now non-code browser-previewable files are detected early so the sandboxed iframe renders correctly.
Fixes the E2E test: renders sandboxed iframe for a browser-previewable non-code file.
* fix(#4117): align PDF artifact route pattern with convention (drop /mock/ prefix)
The test used /mock/api/threads/... for the PDF artifact content route,
but the urlOfArtifact helper generates /api/threads/... (without /mock/)
when isMock=false. The other tests (e.g. presented artifacts) already use
the correct /api/threads/... pattern.
* test(frontend): add render-level coverage for unsafe markdown/artifact links
- Render MarkdownLink and ArtifactLink via renderToStaticMarkup and
assert an unsafe javascript: href produces a disabled <span> (never an
<a>), including through the citation-labelled branch, and that safe
https hrefs render hardened anchors (target=_blank, rel=noopener).
- The new ArtifactLink render test caught the unsafe href leaking onto
the fallback <span> through the {...rest} spread; drop the spread in
both span fallbacks so anchor-only attributes (href/target/rel) and
react-markdown's node prop never reach the span.
- Cover mailto:/tel: in the isSafeHref unit cases and fix the stale
SAFE_HREF_PROTOCOLS docstring that still claimed http/https-only.
* fix(frontend): route the agent save-hint through the safe localStorage facade
Export safeLocalStorage from core/settings/local and use it for the
agent-create save-hint read/write so blocked browser storage (Safari
private mode, strict containers, embedded WebViews) cannot throw from
the effect. core/agents/feature-cache.ts already guards its
localStorage access with try/catch, so settings + agent pages are now
consistently best-effort.
* fix(frontend): allow scheme-less relative markdown links in isSafeHref
* fix(workspace-changes): classify a symlink replacing a file distinctly from deleted
scan_workspace_roots() skipped every symlinked path entirely
(host_file.is_symlink() -> continue), so the path was completely absent
from a snapshot instead of being recorded as a metadata-only stub the
way binary/large/sensitive-looking files already are. When an agent run
replaces a tracked file with a symlink (e.g. rm config.txt && ln -s
/some/other/path config.txt), the after-snapshot never contained that
path at all, so compare_snapshots()'s _status() saw after_file=None and
reported plain "deleted" -- silently hiding that the path is still alive
on disk, now as a symlink that can point anywhere on the host, including
outside the workspace root.
Add a "symlink" classification mirroring the existing binary/large/
sensitive pattern: scan_workspace_roots() now records a symlink as a
metadata-only FileSnapshot stub (symlink=True, symlink_target from
os.readlink(), lstat'd without ever following the link) instead of
omitting it. _status() reports "symlink_created" whenever a symlink
newly occupies a path that was not already a symlink (brand new or
replacing a prior file), so the security-relevant fact surfaces
distinctly instead of collapsing into "deleted". A symlink genuinely
removed with nothing replacing it is unchanged: still "deleted".
Verified against a real POSIX symlink (WSL; native Windows symlink
creation needs elevated privilege) driving the unmodified
scan_workspace_roots()/compare_snapshots() functions, and via a
patch-file revert/reapply cycle on this same fix to confirm the added
regression tests fail before and pass after.
* fix(workspace-changes): count symlink_created in the changed-file badge
getChangedFileCount summed only created + modified + deleted, so a run
whose only change was a symlink replacing a file (reported as the new
symlink_created status, not deleted) produced a count of 0 and
WorkspaceChangeBadge hid the badge entirely -- the exact scenario this
PR targets, with the opposite of the intended result.
Add symlink_created to the frontend WorkspaceChangeSummary interface
(already emitted by the backend) and include it in the count. The
per-file StatusIcon/statusLabel "modified" fallthrough is unaffected
and left as a follow-up, per review.
Regression test reverts cleanly to reproduce a count of 0 pre-fix.
* fix(workspace-changes): rank symlink_created in file sort and complete the type contract
sortWorkspaceChanges's statusRank had no entry for the new
symlink_created status, so statusRank[left.status] - statusRank[right.status]
evaluated to NaN for any comparison involving a symlink-created file,
violating Array#sort's ordering contract instead of producing a
deterministic order. The frontend WorkspaceChangeStatus and
DiffUnavailableReason unions, and the WorkspaceFileChange symlink
fields, also stayed narrower than what the backend now emits, so
TypeScript's satisfies Record<...> guard on statusRank could not catch
the gap.
Widen WorkspaceChangeStatus to include "symlink_created" and
DiffUnavailableReason to include "symlink", add the matching
symlink/symlink_target_before/symlink_target_after fields to
WorkspaceFileChange, and give symlink_created a rank alongside
modified in statusRank -- restoring the satisfies guard's ability to
catch a future unranked status. unavailableLabel now has an explicit
"symlink" branch (new symlinkUnavailable i18n string, en-US + zh-CN)
instead of falling through to the generic label.
Also fixes the failing e2e-tests CI check: the existing
workspace-changes.spec.ts mock summary predates the symlink_created
field, so getChangedFileCount computed 1 + 1 + 0 + undefined = NaN and
the badge rendered "Edited NaN files" instead of "Edited 2 files".
Added symlink_created: 0 to the mock to match the real backend
contract.
New sortWorkspaceChanges unit tests revert cleanly against the
unranked statusRank to reproduce the NaN-driven misordering. pnpm test
(626 tests), pnpm check, and pnpm format are clean, and the
previously-failing e2e spec plus the full e2e suite (94 tests) pass.
* fix(auth): let deployments close local self-registration
The OIDC provisioning policy (allowed_email_domains, require_verified_email,
auto_create_users) is enforced only in the SSO callback via
get_or_provision_oidc_user. POST /api/v1/auth/register creates a local account
without consulting any of it, and nothing can turn that path off, so a
deployment declaring an email-domain allowlist can still be joined by any
address through local registration.
Add auth.local.allow_registration (default true, so existing deployments are
unchanged) and gate /register on it before the account is created. Report the
flag from /setup-status so the login page stops offering a signup entry the
Gateway will reject.
/initialize is deliberately not gated: it is the bootstrap path, guarded by
admin_count == 0, and closing it would leave a fresh install unable to create
its first admin.
An unreadable config.yaml falls back to the pre-gate default (open) rather than
making these two endpoints a hard dependency on the file.
* docs(auth): align registration-gate fallback wording with the FileNotFoundError catch
* fix(frontend): use code-aware stripLeakedSystemTags for all internal marker tags
Replace the ad-hoc /<\/?memory>/g in ClipboardSafeStreamdown with a
dedicated stripLeakedSystemTags() function in preprocess.ts that:
- Covers all INTERNAL_MARKER_TAGS (<memory>, <system-reminder>,
<current_date>, <uploaded_files>, <slash_skill_activation>) instead
of only <memory>
- Is code-aware: skips fenced code blocks (```) and indented code
blocks (4-space indent), so user-written meta-discussions about the
memory system are not silently stripped
- Handles opening tags, closing tags, self-closing tags, and tags with
attributes
- Adds 15 unit tests covering all the above cases
* fix(frontend): use marker-aware fence tracking in stripLeakedSystemTags
The boolean insideFence toggle incorrectly closes a fence on any line
matching 3+ backticks or tildes, regardless of the actual delimiter.
This causes tags inside a tilde-fenced block containing a backtick
sub-fence, or a 4-backtick block containing a 3-backtick sub-fence, to
be silently stripped.
Track the opening fence marker (character and run length) so that only
a matching marker with at least the same length closes the fence.
Adds 4 new test cases:
- Tilde fence with inner backtick fence
- 4-backtick fence with inner 3-backtick fence
- Shorter tilde closing inside longer tilde fence
- Tags stripped after real closing fence
* fix(frontend): fix TypeScript strict errors in fence marker tracking
- Use non-null assertion (!) on fenceMatch[1] (guaranteed by regex)
- Use charAt(0) instead of [0] to avoid string | undefined type
* fix(runtime): persist original human input outside model sanitization
* refactor(history): load thread messages by global event sequence
* fix(frontend): make summarization rescue a transient history bridge
* fix(frontend): old message not append tail
1. add identity anchor
2. add bridgeOrder
* fix(frontend): lint error fix
* fix: address review feedback and harden pagination coverage
- defer transient history ref writes until after render commit
- cover large middleware-only history scans
- verify infinite-query refetch recalculates page cursors
- document AI event types and anchor-weaving differences
* fix: harden message pagination and enrichment
- append unmatched live tails after canonical history
- warn and stop when pagination has_more lacks a cursor
- deep-copy restored UI messages to isolate model-facing content
- log invalid event sequence and non-advancing cursor errors
- pass user_id explicitly through event-store history queries
- cover middleware-only AI runs across memory, JSONL, and DB stores
* fix: address pagination review feedback
* fix(frontend): checkpoint has unknow redener content, optimize the anchor policy
* fix(frontend): unit test issue missed previously, remove the TanStack cache trimming
* fix(gateway): harden message history queries and provenance
- reject externally forged original_user_content metadata
- validate provenance metadata in upload and sanitization middleware
- make run lookups fail closed by default
- batch feedback queries by run ID
- align memory message filtering with persistent stores
* feat(frontend): show real project version on About page
The About page heading was hardcoded "About DeerFlow 2.0" while the real
version is 2.1.0, kept in lockstep across backend/pyproject.toml,
frontend/package.json, and Chart.yaml by scripts/verify_versions.sh.
With the nightly workflow shipping unreleased main daily, a hardcoded
version drifts further from reality every day.
Wire the heading to the real version, nightly-aware:
- src/version.ts reads NEXT_PUBLIC_APP_VERSION (nightly override) with a
package.json fallback for release/local builds.
- about-content.ts interpolates APP_VERSION into the heading; the
"DeerFlow 1.0 and 2.0" milestone copy in acknowledgments is left as-is.
- Dockerfile adds ARG APP_VERSION / ENV NEXT_PUBLIC_APP_VERSION in the
builder stage so nightly CI can stamp the version.
- nightly.yaml computes the nightly string once in prepare
(<base>-nightly.<date>-<sha>, mirroring the chart scheme) and feeds it
to both the frontend image build-arg and the chart publish (refactored
to consume the shared output, so the displayed version and chart
version can't drift). Release builds (container.yaml) are untouched
and fall back to package.json.
Tests cover the version branches and the heading interpolation;
RELEASING.md notes the About page as a derived version consumer.
* test(frontend): assert positive About-page version value
Address review feedback on #4126: the unset-env assertion was negative-only
(checked the stale "2.0" literal was gone), so it would still pass if
APP_VERSION interpolated to "" or undefined. Mirror version.test.ts by
asserting the heading carries the real resolved APP_VERSION.
* ci(nightly): fail loudly on empty base version
Address review feedback on #4126: the prepare step's BASE (parsed from
Chart.yaml) had no empty-guard. A malformed/missing version would yield
`-nightly.<date>-<sha>` (invalid semver) that now flows into both the
chart publish and the frontend About-page version. Add `set -euo pipefail`
plus a `test -n "$BASE"` guard mirroring publish-chart. `pipefail` matters
- without it the pipeline's exit code is awk's, so `set -e` alone wouldn't
catch a grep miss.
* feat(subagents): show runtime metadata on task cards
* fix(subagents): stop task-card render loop and dedupe model fetches
Address code review on the runtime-metadata cards:
- P1 render loop: the terminal ToolMessage is re-parsed on every
MessageList render and always carries modelName/usage, so the
presence-based setTasks condition fired a fresh state object each
render -> "Maximum update depth exceeded". computeNextSubtask now
returns a value-compared `changed` flag and a pure subtaskNotification()
routes terminal transitions through the deferred after-render path
while skipping no-op re-parses.
- Per-card useModels refetch: add staleTime: Infinity to the ["models"]
query so every subtask card shares one /api/models fetch instead of
refetching on each mount.
* make format
* refactor(subagents): dedupe token-usage validators + tidy event narrowing
Address PR review follow-ups:
- DRY: extract one shared token-usage validator per side. Backend
status_contract.normalize_token_usage() now backs both the terminal
ToolMessage metadata and the subagent.step/.end run events
(step_events.py), and frontend messages/usage.normalizeTokenUsage()
backs both the live task_running event (lifecycle.ts) and the terminal
ToolMessage metadata (subtask-result.ts). Prevents the input/output/
total_tokens validation from drifting across the four former copies.
- Nit: onCustomEvent narrows event.type once instead of re-checking the
object shape per branch; the redundant task_started early-return
(already validated by taskEventToSubtaskUpdate) is dropped.
Phase 2 of #3875. Two guardrail axes can end a subagent run early — the turn
budget (GraphRecursionError) and the token budget (TokenBudgetMiddleware) —
and both now surface *why* through one additive `subagent_stop_reason` field
instead of a status enum.
This completes and course-corrects Phase 1 (#3949), which shipped the
turn-budget cap as a `max_turns_reached` status enum. The agreed Phase 2
design replaces that enum with an optional `stop_reason` field
(token_capped | turn_capped | loop_capped): a new enum value would break v1
consumers, while an additive field is ignored by older frontends and ledger
readers. `max_turns_reached` and SubagentStatus.MAX_TURNS_REACHED are removed.
- subagents.token_budget config (default enabled, 2,000,000 tokens, warn 0.7)
with per-agent override; TokenBudgetMiddleware is now attached in
build_subagent_runtime_middlewares so the cost-ceiling backstop engages for
every subagent. The hard-stop does not raise — it strips tool_calls and
lets the run finish with a final answer, recording the cap on a per-run
consume_stop_reason() accessor.
- executor.py: on normal completion it reads consume_stop_reason() and stamps
completed + token_capped when the budget fired; on GraphRecursionError it
recovers the last AIMessage partial (completed + turn_capped) or, if nothing
usable survived, failed + turn_capped. SubagentResult gains stop_reason.
- status_contract.py / contracts/subagent_status_contract.json (v2) /
frontend subtask-result.ts: additive subagent_stop_reason field, pinned by
test_status_values_match_contract / test_stop_reason_values_match_contract.
- task_tool.py + delegation_ledger.py: drop the max_turns_reached paths; the
ledger captures stop_reason and renders model-facing "capped" guidance so
the lead reuses a capped completion knowingly.
The 2,000,000-token default is deliberately loose (tighten to taste) — it
would have roughly halved the reported 4.4M burn while leaving legitimate
deep-research runs (max_turns=150) room. Subagent summarization is a follow-up.
* feat(frontend): render slash-skill activations as inline chips
Show an explicit `/skill` activation as a compact inline chip in both the
composer and the chat transcript instead of raw slash text.
- Composer: selecting a skill suggestion stores it as a removable chip
aligned inline with the textarea; the leading `/skill ` prefix is
reattached only at submit time, so the backend activation protocol is
unchanged. Backspace on an empty input or the chip's close button clears
it; history navigation is disabled while a chip is active.
- Transcript: human messages that begin with `/skill` render the skill as a
read-only chip followed by the task text.
- Add a shared `core/skills/slash.ts` (`parseSlashSkillReference` +
`resolveSlashSkillDisplay`) mirroring the backend `slash.py` gate, so the
transcript only shows a chip when the skill actually exists and is enabled.
This removes a duplicated regex/reserved-name list and keeps display
semantics consistent with backend activation.
Add unit tests for the shared slash parser and extend the chat e2e to assert
the composer still submits `/skill <task>` after showing a chip.
* chore(frontend): format chat e2e test
* refactor(skills): address slash-skill chip review feedback
Follow-up to the inline slash-skill chip PR, resolving three second-order
review findings:
- Drive the reserved-command set and skill-name grammar from a shared
contracts/slash_skill_contract.json instead of a hand-copied
"keep in sync" pair. slash.ts and slash.py now reference the fixture, and
contract tests on both sides fail CI if either drifts.
- Extract a shared SlashSkillChip so the composer and transcript chips stay
in lockstep, and normalize the off-scale /8 and /12 opacity steps to the
standard /10 and /20 tokens.
- Split HumanMessageText into a pure parse gate plus a slash-only subtree
that owns the useSkills() lookup, so a skill-enabled toggle no longer
re-renders every plain-text human turn.
Verified: frontend eslint + tsc clean, pnpm test 572 pass (incl. new
slash-contract test); backend slash contract + slash-skills tests 31 pass.
* style(tests): sort slash skill contract imports
* fix(composer): inline the slash-skill text so the chip aligns with input
Address the "composer body layout change" review on #3981 by rendering the
active skill as an inline chip in the same text flow as the prompt, rather
than a separate flex row that drifted the box model across states.
- Render the chip + prompt inside one leading-6 wrapper and edit the prompt
through a `contentEditable` span, so the chip sits inline with the first
line and long/multi-line input wraps naturally back to the container edge.
- Align the chip with `align-top`: its h-6 (24px) height matches the text
line height, so chip and first-line centers coincide exactly (measured
delta 0), fixing the chip being raised above the baseline.
- Restore the placeholder in chip mode via a `data-empty` CSS `::before`,
which also gives the empty editable span width so it is no longer treated
as hidden.
- Widen the IME helper to `HTMLElement` and route the span's keydown/paste
through the shared skill-suggestion, prompt-history, backspace-to-clear,
and IME-composition handlers so contentEditable behaves like the textarea.
- Extend chat.spec.ts to drive the inline skill editor instead of the
textarea after a chip is shown.
* style(frontend): fix composer class order formatting
* fix(composer): break long unbroken input inside the slash-skill row
The inline slash-skill editor wrapped with `break-words`
(overflow-wrap: break-word), which only moves an over-long token to the
next line before breaking it. A long unbroken string therefore started
on the line below the chip, and when the string contained a break
opportunity such as a hyphen the browser wrapped there and pushed the
remaining run to the next line, leaving a wide gap on the right.
Switch to `break-all` (word-break: break-all) so the text fills each
line from the chip and packs tightly regardless of hyphens or CJK.
* feat: add composer input polishing
* Revert "Merge branch 'main' into feat/input-polish"
This reverts commit 5b6ceccf0db3092bc62fde3b05e7816829601756, reversing
changes made to 45fbc57fef5fa5fd878cf0176c37f3e3bc7ebef6.
* Merge main into feat/input-polish
* style(frontend): format input helper polish guard
* fix(input-polish): address composer polish review findings
Frontend
- Add a cancel affordance to the in-flight polish status pill that calls
abortInputPolishRequest(), so a slow/hung provider no longer hard-locks the
composer for up to stream_chunk_timeout with a page reload (and draft loss)
as the only escape.
- Reset promptHistoryIndexRef/promptHistoryDraftRef when a rewrite is applied
(and on undo), so a stale history-browse index can no longer let the next
ArrowDown silently overwrite the polished draft.
- Disable polishing while an open human-input card is present, matching the
frontend/AGENTS.md rule that composer entry points defer to the card so
card-reply metadata is preserved.
- canPolishInput now reuses parseGoalCommand/parseCompactCommand instead of a
third hardcoded reserved-command regex, and drops the phantom /help entry
(no /help parser exists in the composer), so future builtins only need to be
taught to the existing parsers.
Backend
- Extract the non-graph one-shot LLM path (build model + inject Langfuse
metadata + system/user invoke + text extract) into
deerflow.utils.oneshot_llm.run_oneshot_llm, shared by the input-polish and
suggestions routers so tracing-metadata and invocation shape cannot drift
between the two copies.
- strip_think_blocks gains truncate_unclosed (default True, preserving the
suggestions/goal JSON-prep behavior); input polish passes False so a draft
that legitimately contains a literal <think> substring is no longer
truncated into a partial rewrite or a spurious 503.
- Validate the empty-check and max_chars boundary against the same stripped
view of the draft that is sent to the model, so the user-facing length
boundary and the model input can no longer disagree.
Tests / docs
- Backend: literal-<think> preservation, whitespace-only rejection, and
normalized-length/model-input agreement cases; suggestions tests repoint the
create_chat_model patch to the shared helper module.
- Frontend: helper unit tests updated for the /help/reserved-command change; a
new Playwright case covers cancelling an in-flight polish request.
- backend/AGENTS.md documents the shared one-shot helper and the polish
normalization/think-tag behavior.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(frontend): add structured human input cards for ask_clarification
Implement a reusable Human Input Card flow for ask_clarification while keeping
the existing text fallback for older clients and IM channels.
Backend:
- Add structured ToolMessage.artifact.human_input payloads for clarification requests.
- Preserve ToolMessage.content as the readable Markdown/text fallback.
- Normalize clarification options from native lists, JSON strings, plain strings,
mixed scalar values, None, and missing options.
- Derive input_mode as choice_with_other when options exist, otherwise free_text.
- Keep disable_clarification non-interactive behavior as a plain ToolMessage with
no human_input artifact.
- Cover artifact persistence and Gateway message metadata preservation in tests.
Frontend:
- Add human input protocol types, runtime guards, extractors, response builders,
and thread-state helpers.
- Add reusable HumanInputCard with option buttons, free-text input, pending,
read-only, disabled, and answered states.
- Render structured clarification cards from artifact.human_input, with Markdown
fallback for malformed or legacy tool messages.
- Preserve line breaks in structured question/context/option text.
- Hide submitted clarification bridge messages from the chat UI via
additional_kwargs.hide_from_ui.
- Send structured human_input_response metadata through the fourth sendMessage
options argument, preserving run context in the third argument.
- Wire submissions for normal chats, custom agent chats, agent bootstrap chats,
and sidecar chats.
- Derive answered state from raw thread.messages so hidden replies still update
the original card.
- Clear pending state when the hidden reply arrives, dispatch is dropped, or a
later async stream failure appears on thread.error.
* perf(frontend): optimize HumanInputCard UI interactions
- Support Enter key to submit text input (Shift+Enter for newline)
- Render question and context fields as Markdown instead of plain text
- Replace deprecated FormEventHandler type with structural typing
* test(frontend): add unit test cover optimize HumanInputCard UI interactions
* feat(frontend): disabled chatbox when has new human-input-card
* fix(style): lint error fix
* fix: sanitize hidden human input replies
- Preserve IME composition safety for human input card Enter submits
- Treat hidden human input responses as genuine user messages for sanitization
- Keep hidden card replies in memory filtering while excluding malformed/internal hidden messages
- Add regression coverage for card IME handling and hidden reply sanitization
* fix: tighten human input response validation
- Reject empty hidden human input response values
- Remove invalid list ARIA role from human input card options
- Add backend coverage for empty response payloads
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sidecar): make panel button delete the side chat instead of hiding it
The side chat panel's top-right button called `sidecar.close()`, which only
hid the panel — duplicating the header `SidecarTrigger` toggle. Repurpose the
button so the panel owns deletion and the header toggle owns hide/show.
- When a conversation exists, the button shows a trash icon and opens a
confirmation dialog before deleting via `useDeleteThread` (backend cascade).
- In the draft state (references only, no thread yet) the header trigger is not
rendered, so the button falls back to a plain close (X) that discards the
draft without a confirm — there is nothing persisted to delete.
Also fix a latent double-delete bug surfaced by the new dialog:
`useDeleteThread` deletes through the LangGraph route (which drops the
thread_meta row) and then calls `deleteLocalThreadData`, which hits the same
gateway handler whose `require_existing=True` guard now 404s. Treat that 404 as
idempotent success. The recent-chat delete used fire-and-forget `mutate`, so it
swallowed this error; the sidecar's `mutateAsync` surfaced it as a toast.
The e2e mock now mirrors the gateway ownership guard (second DELETE 404s once
the thread is gone) so the regression stays covered. Adds tests for delete,
draft-state close, and header-owned hide/show.
* fix(sidecar): lock delete dialog while deletion is in flight
The delete confirmation dialog disabled Cancel during an in-flight
delete but still allowed dismissal via the overlay, Esc, and the
built-in close (X). Those paths could imply the delete was cancelled
when it was actually still running. Ignore dismissals and drop the
close button while `isDeleting` so only the (disabled) Cancel path
controls the dialog.
