`shell-curl-pipe-shell` hard-codes `bash|zsh|dash|fish|sh` in the download-pipe
matcher while the structural `env` fallback reads `_SHELL_NAMES`. The two lists
drift: `curl ... | ksh` (and csh/tcsh/ash/mksh) pipes a fetched script in
silence, even though sandbox_audit_middleware already treats ksh as a shell.
Derive the alternation from `_SHELL_NAMES` and widen both to the same ten
names, so a shell added once is visible to both detectors.
* fix(extensions): keep atomic-write cleanup from replacing the save error
Extension snapshot rollback, the managed plugins rewrite, and skill-archive
upload copies now use cleanup_failed_atomic_write so a failed unlink or
warning log cannot mask the original publication error.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
* fix(skills): keep upload and storage cleanup from replacing the save error
Route the skill-archive upload discard, custom-skill file writes, and
per-user skill-state writes through cleanup_failed_atomic_write. Close the
extension rollback descriptor on fdopen failure, and record the plugins
temporary path before the write so a short write is still removed.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
* fix(skills): keep form close and snapshot delete from masking the save error
Discard the uploaded skill archive before closing the multipart form, and
suppress close errors so a flush failure cannot replace the install outcome.
Snapshot rollback of a missing original now uses cleanup_failed_atomic_write,
and skill-state writes clean up on BaseException including KeyboardInterrupt.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
* test(extensions): assert logging cleanup cannot replace the delete-branch error
The log-handler check only ran for the unlink parameter, where the handler
is never patched. Move it under the logging failure so a handler exception
must stay out of the surfaced install error.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
---------
Co-authored-by: wannT-1 <200890097+wannT-1@users.noreply.github.com>
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: honor checkpoint object in thread state updates
Prefer the checkpoint object over the deprecated checkpoint_id, validate
its thread_id, and preserve checkpoint namespace when updating state.
Add regression coverage for checkpoint-object forks and thread mismatch.
* fix: preserve client context handling in checkpoint PR
* fix: support state updates in checkpoint namespaces
* fix: reject unknown checkpoint namespaces
* fix(gateway): isolate subgraph titles from thread metadata
Only synchronize state titles into conversation metadata for the root checkpoint namespace. Cover persisted child updates with and without checkpoint IDs in full and delta modes, and retain root title synchronization and branch sequence cleanup.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): return from quarantine teardown when a peer already did the work
retire() and retire_replaced_generations() guard their final rmdir() but list
the fence directory and unlink each record unguarded, while contains() and
_exists() in the same class already report a directory removed underneath them
as "no fence". The teardown exclusion is a TTL lease, not a lock, so a peer can
finish the same teardown in those two windows and the resulting FileNotFoundError
escapes the unwrapped _retire_absent_quarantine() call at the top of creation.
Both passes now tolerate a peer that removed the directory or the record first.
* test(sandbox): correct quarantine docstring reference
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): resolve relative links in messages to thread artifacts
An assistant message that links a file it wrote with a relative
destination, e.g. [Report](report.md), rendered a plain relative href.
The browser resolved it against the chat route to
/workspace/chats/report.md, so clicking it opened a "thread" whose id is
the file name and the Gateway rejected it (422). Only /mnt/ links were
rewritten, and the clarification and present_files message text did not
get the thread-aware link renderer at all.
- A relative link (no scheme, no leading /, no .. segment) that
suffix-matches exactly one thread artifact now opens that artifact in
a new tab, using the rule message images already use. When zero or
several artifacts match it renders as text instead of a link to a
broken chat URL. Links outside the message list are unchanged.
- The artifact list reaches the link renderer through a context:
Streamdown memoizes its root without comparing `components`, so a
renderer built before the list arrives would keep the stale list.
- /mnt/ artifact links get the normal link styling (they rendered as
unstyled text before).
- A reopened thread rebuilds its values from the POST /history head,
which carries messages but not `artifacts`. The message list and the
artifact panel now add the /mnt/ paths from present_files calls in the
messages, so links resolve and the panel lists the files after reload.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(frontend): address artifact link review
---------
Co-authored-by: huangzz <110116@nd.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
* fix(sandbox): retain process group id for timeout cleanup
* test(sandbox): cover exited leader during group cleanup
* fix(sandbox): capture process group id before drain threads
* style(sandbox): separate timeout regression test
* test(sandbox): cover missing process-group fallback
* style(tests): format sandbox timeout regressions
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(acp): fail fast when an ACP agent exits and drain its stderr
An ACP agent that exited before answering (missing credentials, a wrong
argument) left invoke_acp_agent waiting for the whole timeout_seconds
budget (30 minutes by default) and then suggested raising the timeout.
The pinned SDK never fails pending requests when the agent's stdout
reaches EOF, so nothing noticed the process was gone.
The SDK also opens the agent's stderr as a pipe and never reads it. An
agent that logged more than asyncio's pipe buffering absorbs blocked on
its next stderr write and stalled the invocation the same way.
Race the ACP exchange against the agent process: when it exits (or a
newer SDK reports the connection closed), allow a short grace for
responses still buffered in the pipe, then report the exit code. Drain
stderr continuously into a bounded tail that is logged on failure and
kept out of model-visible text.
* fix(acp): log the full stderr tail on timeout and pin tail retention
Log the timeout tail after the SDK context has terminated and reaped the
agent, so lines still in flight when it stalled are included. Make the
exit test write more stderr than the retained tail, so keeping the start
of the stream instead of its end fails the assertion on the fatal line.
* feat(agent): expose read-only run status tool
* refactor(tools): name the subagent dispatch tool set
Hoist the hardcoded subagent dispatch tool-name literal into _SUBAGENT_DISPATCH_TOOL_NAMES so a future dispatch-tool rename has one place to update. No behavior change: the folded-subagent context_fill guard compares the same two names.
* test(tools): account for run status in collision assembly
* docs: keep script guidance within size budget
* fix(subagents): report model_length_capped when a subagent hits the output limit
ModelLengthFinishReasonMiddleware only ran on the lead chain. A subagent
whose response was cut at the provider output limit (finish_reason=length,
stop_reason=max_tokens, MAX_TOKENS, Responses max_output_tokens) reached
the lead as a clean `completed`, and a tool call truncated at that
boundary still executed with partial arguments.
Register the same middleware on the subagent chain, in the lead's
relative position (after configured extensions, before Safety). It now
also records each detected cap per context run_id and exposes
`consume_stop_reason`, so the executor's existing duck-typed guard
collection reports `stop_reason=model_length_capped`. Add the value to
the subagent status contract (Python enum, contract fixture, frontend
whitelist) with the label "model output length limit", and update the
subagent docs.
Tests cover the middleware channel, chain placement, the contract, and
an end-to-end executor run with a fake model.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* test(tasks): cover cap reasons on completed and failed runs
Extend the existing shared-contract round-trip matrix to preserve both terminal statuses alongside every supported stop reason, including model_length_capped and loop_capped.
---------
Co-authored-by: huangzz <110116@nd.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): search skills across category mounts
* fix(sandbox): address virtual search mount review
Search explicitly configured category mounts regardless of ignored names,
while retaining descendant filters and host-parent exclusion. Warn with the
virtual mount path before re-raising broken-mount errors, defer ordinary
path resolution, and cover lookahead across mount boundaries.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): treat a fetched process substitution as a program operand
`bash <(curl url)` hands the fetched file to bash as an *operand*, not as a
redirect: bash runs it. The audit only recognised the stdin spelling
(`bash < <(curl url)`), so the operand spelling executable-fetched content
through the sandbox audit unimpeded — the miss documented in the middlewares
AGENTS.md known-gaps list.
An interpreter in command position whose first operand is a risky process
substitution now blocks. The two exceptions mirror the stdin rule: a script
operand in front (`bash script.sh <(curl url)`) keeps the substitution an
ordinary argument, and a shell noexec flag (`bash -n <(curl url)`) only
parses it.
Verified in Git Bash:
bash <(echo "echo FETCHED_AS_PROGRAM") -> FETCHED_AS_PROGRAM
bash <(echo "echo FETCHED_WITH_ARG") consumer.sh -> FETCHED_WITH_ARG
bash -n <(echo "echo NOEXEC") -> exit 0, no output
* docs(sandbox): clarify substitution audit limitations
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(tui): let quit skip an interrupted run's unfinished step
After Ctrl+C the synchronous agent stream keeps running until its next
event. The run was a Textual thread worker, i.e. a thread on asyncio's
default executor, which App.run() joins on shutdown (up to 300 s on
3.12) and interpreter exit joins again with no timeout. Quitting while
a long step was still running therefore hung silently after the
terminal had been restored, and further Ctrl+C presses only printed a
traceback.
Run workers on a daemon thread instead. Quit now waits visibly for
running workers (an interrupted step still checkpoints, a finished run
still saves its title) and refuses sends meanwhile; a second quit
request exits at once. Report unexpected worker exceptions as an error
row and end the run, since Textual no longer catches them.
* docs(changelog): reference #7326 for the TUI quit-after-interrupt fix
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(goal): use spare evidence budget for long user input
* fix(goal): retry one malformed evaluator verdict
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
The unsuccessful-upload branch called result.message.trim() directly. A
gateway or proxy that omits message turns a handled upload failure into a
TypeError while handling it, losing the clean "Upload failed" path.
Coerce a non-string message to an empty string before trimming.
The regression test sends success:false without a message and expects
"Upload failed"; it throws a TypeError on main.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(tui): append streamed assistant deltas verbatim
The reducer merged each delta by comparing it with the row text: an
equal delta was a no-op, a delta starting with the row replaced it, and
a delta the row started with was dropped. Those rules absorbed re-sends,
replayed history and cumulative snapshots, none of which
DeerFlowClient.stream emits any more, so they only deleted real tokens
("ha" + "ha" -> "ha", "1" + "10" -> "10") and every standalone Markdown
marker an answer opened with (closing **, list bullets, table pipes,
closing code fences).
Append every delta on both the id-keyed and anonymous paths, and drop
_merge_stream_text. Rewrite the tests that pinned the re-send premise
as append contracts, run the merge cases through reduce() on both
paths, and drive the real client stream into the reducer.
* docs(changelog): note TUI streamed-delta fix (#7325)
* docs(changelog): drop trailing spaces after the #7325 entries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(artifacts): keep a delimited sandbox path whole
The free-text artifact scan matched `/mnt/user-data/\S+`, so a tool that
quoted a filename with spaces registered a handle for the prefix: `Created
"/mnt/user-data/outputs/quarterly report.pdf" successfully.` became
`/mnt/user-data/outputs/quarterly`, which resolves to nothing after compact or
checkpoint reload. Quotes and inline backticks had the same failure.
The scan now classifies each candidate by how it is delimited. A span opened by
a matching quote or backtick on the same line is one value and keeps its spaces
and punctuation; a span whose delimiter never closes on that line reports
nothing, because a truncated prefix names a file that does not exist; a bare
path keeps the conservative whitespace boundary it always had. Ordering is the
text order, and the shared `seen` set still deduplicates against structured
references.
Pins the three delimiters, both unchanged behaviours (bare path boundary,
punctuation outside the delimiter), the unterminated and next-line cases, two
delimited paths in one result, a mixed delimited/bare result, and the extracted
handle end to end.
* fix(artifacts): reject line breaks and match escaped quotes
* fix(artifacts): validate quoted path boundaries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(community): migrate You.com tools to async HTTP
The You.com provider sent its search request through a sync httpx.Client. A
sync LangChain tool invoked through ainvoke runs on the default executor, so
the request held one of that pool's threads for the whole network round-trip,
and several providers queried in parallel queued behind each other.
Make the tool a coroutine over httpx.AsyncClient, resolve the API key and
the configured max_results in a single config read, and move both disk reads
off the event loop: the tool config, and the User-Agent, which reads the
installed package metadata the first time it is needed.
* refactor(community): drop the dead You.com API key wrapper
The async migration resolved the key and max_results through
_resolve_tool_settings, which left _get_api_key with no production
caller: only TestGetApiKey still exercised it. Keeping a second seam
around the same config read lets it drift from the real call path, so
drop the wrapper and point the tests at _resolve_tool_settings.
Also keep the hot path off the executor: _user_agent is cached, so only
the first call reads the installed package metadata. Later calls are
in-memory hits and now skip the to_thread round-trip.
Finally, record which httpx client factory the provider reaches for in
the runtime anchor. MockTransport runs its handler on the calling thread
without real blocking I/O, so a provider that stayed async but sent the
request through a sync client would satisfy the thread assertion; the
recorded factory is what catches that partial regression.
* test(community): align You.com URL safety stub with async tool
The URL safety matrix still patched the removed _get_api_key seam and invoked the You.com tool synchronously. Mock the awaited search with AsyncMock and call the tool through ainvoke so URL screening remains covered after the async migration.
---------
Co-authored-by: treasuresyq <309743290+treasuresyq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Exa web_search and web_fetch accept a per-tool base_url, falling back to
EXA_BASE_URL and then the SDK default. Overrides are validated like the
Serper base URL. web_fetch screens a configured backend with the shared
delegated-backend guard.
Co-authored-by: lachy3433 <lachlanallen.co@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(models): honor HTTP-date Retry-After hints in Claude retries
* refactor(models): share bounded HTTP-date retry parsing
* test(agents): move retry date regression to middleware suite
get_soul was the only read/write method that skipped validate_agent_name,
so a name like "../../other-user/agents/planner" reached
resolve_agent_dir and read another user's SOUL.md. Validate it like the
other methods do.
The regression test asserts ValueError for a traversal name; it reads the
other user's soul on main.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): release the E2B ownership lease when acquire finds the VM gone
The in-process reuse path (dead or unreachable cached client) and the
warm-pool reclaim path (reconnect reports "sandbox not found") dropped
their local bookkeeping but kept the VM's ownership lease. Nothing later
sees that ID: the warm sweep only walks the warm pool, reconciliation
only walks the remote inventory, and the lease renewer keeps renewing
(and re-claims a lapsed lease) until shutdown. Every reaped VM left one
entry in _owned_sandbox_ids and one renewed ownership key behind.
Call _release_ownership in both branches. Both act only on E2B's trusted
not-found signal, the store's release is a no-op for a peer's lease, and
a backend error still drops the local ID so the shared lease expires on
its TTL.
* docs(changelog): reference #7277 for the E2B dead-VM lease fix
* test(sandbox): pin that transient E2B errors keep the ownership lease
The dead-VM release is only safe because the gone signatures match
narrowly. Pin the other side: a non-gone ping error keeps reusing the
cached VM, and a non-gone warm reconnect error leaves the lease to
eviction.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(startup): detect extras for browser-configured IM channels
* docs(startup): clarify channel extra provider source
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(mcp): keep host paths out of the MCP config mutation 500 details
The five MCP config mutation routes answered an unexpected failure with
`f"Failed to …: {str(e)}"`. Each one runs a read-modify-write of
`extensions_config.json` (`_apply_mcp_config_update`,
`_apply_mcp_servers_create`, `_apply_mcp_server_config_update`,
`_apply_mcp_server_delete`, `_apply_mcp_server_state_update`), so the error the
handler catches is usually an `OSError` from `atomic_write_extensions_config`
that names the absolute host path — the `DEER_FLOW_HOME` layout of the server.
The custom-agent, user-profile and skills-read routes already return fixed text
for this reason (#7255; `routers/AGENTS.md`: "agent config parse errors and
`OSError`s carry host paths and stay in the server log"). These five were the
remaining MCP mutation 500s on the same file, so they now return the fixed
detail and keep the exception in the existing log line.
Red first, on the unmodified `main` — all five put the planted host path in
`detail`:
$ python -m pytest tests/test_mcp_config_error_details.py -q
5 failed in 6.37s
After the change, 5 passed; the wider MCP suites (246 tests across
`blocking_io/test_mcp_router.py`, `test_mcp_config_secrets.py`,
`test_mcp_lifecycle*.py`) stay green.
* fix(mcp): normalize personal config mutation errors
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skills): treat only top-level scripts/ paths as executable in skill_manage
write_file classified a support file as executable content with a substring
test ("scripts/" in path), so any support path merely containing "scripts/"
(e.g. references/scripts/overview.md or templates/my-scripts/setup.md) was
moderated as executable: that scan requires an exact "allow" decision and
hard-blocks when the moderation model is unavailable. Executable content is
now exactly a file under the skill's top-level scripts/ directory, matching
the scripts/ member rule in skills/package_files.py and the same Path
semantics the storage layer already used to validate the support path.
* refactor(skills): share the scripts/ member rule with skill_manage
The write_file executable gate re-derived the top-level scripts/ member rule
locally. Use package_files.is_scripts_member instead so the gate, installer,
and SkillScan stay in lockstep.
Extend the regression test to pin scripts/sub/run.sh and ./scripts/run.sh as
executable content, plus the shared helper's normalization.
* fix(skills): moderate nested code files as executable content in skill_manage
The write_file executable gate classified only top-level <scripts/> members as
code, so runnable support files elsewhere in the package
(references/scripts/helper.py, templates/my-scripts/run.sh,
assets/my-scripts/tool.js) and extensionless shebang files fell back to static
moderation: a warn decision, or an unavailable moderation model with
security_fail_closed=false, wrote them to disk. The deterministic package scan
does not enforce the allow-only model verdict, and this write path publishes
through write_custom_skill without running the mutations CandidateScanner, so no
later check restores executable moderation.
Classify with the complete shared code-file rule from
deerflow/skills/package_files.py (scripts members, code suffixes, and
extensionless shebang files) instead of the scripts-member rule alone, and keep
Markdown/data support files on the static tier. Extend the regression suite with
the code-suffix/ shebang cases plus the warn and unavailable-provider outcomes.
* refactor(helm): derive the postgres Secret name from one helper
`databaseUrlSecret` re-encoded the `postgresql.external.existingSecret` then
`postgresql.existingSecret` chain in its own `if`/`else if`. Three consumers
read it — the gateway's `DATABASE_URL`, the postgres StatefulSet's
`POSTGRES_PASSWORD`, and `postgres-secret.yaml`'s own `metadata.name` — so a
source added to one helper and not the other would make the gateway and the
StatefulSet resolve different Secrets. That is the class of bug #7149 fixed
for the redis pair.
Extract `databaseUserSecret` as the single encoding of the chain and let
`databaseUrlSecret` fall back to the chart-managed name when it is empty,
mirroring `redisUserSecret` / `redisUrlSecret`.
Pure refactor: the rendered manifest is byte-identical before and after across
the four configurations (chart-managed, `existingSecret`,
`external.existingSecret`, and both set), compared with `helm template`.
Adds `test_helm_database_secret_precedence.py`, which pins that both consumers
name the same Secret and that only the chart-managed branch renders a Secret.
Dropping either precedence source fails the two cases that depend on it.
Suggested in review of #7161.
* fix(helm): keep the postgres Secret helper's truthiness
Review follow-up on the previous commit. Two things were still wrong:
`postgres-secret.yaml` kept its own copy of the precedence chain for the
render gate, so "the only place the chain is encoded" was not yet true — a
source added to `databaseUserSecret` alone would have left that gate empty
and rendered an unreferenced `<release>-postgres` Secret. It now reads the
helper, like `redis-secret.yaml` already did.
`or` returns its last operand even when it is falsy, so
`postgresql.existingSecret: false` (how people spell "not set") named the
Secret `false` instead of falling back to the chart-managed one. Chaining
`default` keeps the old if/else-if truthiness and yields an empty string
for any falsy value; the gateway's `$pgConfigured` now reads the same
helper rather than re-listing the two keys.
Two falsy cases join the parametrize list: `false` in the lower-precedence
slot and in both slots. Both fail against the previous `or` version with
`assert [] == [False]` — the Secret literally named `false` — so they are
regressions this test can actually catch. The other four configurations
render byte-identically before and after.
* fix(helm): keep the redis user-Secret helper's truthiness
Same defect the postgres helper had, one release later: `redisUserSecret`
used `or`, which returns its last operand even when falsy, so
`redis.existingSecret: false` named both the gateway's `redis-url` ref and
the StatefulSet's `redis-password` ref after a Secret called `false`. The
render gate in `redis-secret.yaml` saw the string "false" as truthy, so it
skipped creating the chart-managed Secret — leaving two dangling refs and
a StatefulSet that cannot start.
`default` chaining restores the pre-`redisUserSecret` if/else-if behaviour:
any falsy value yields an empty string, so the chart-managed branch is
taken. `redisConfigured` now reads the helper instead of re-listing the two
keys, matching the gateway's `$pgConfigured`.
Verified against the four string-valued configurations (chart-managed,
`existingSecret`, `external.existingSecret`, both): renders are
byte-identical before and after. The two new falsy cases fail on the
previous version, and `--basetemp` makes all 14 cases in this file pass.
* docs(helm): drop a stray article in the helper comment
