3834 Commits

Author SHA1 Message Date
Nan Gao
063383d995
fix(deploy): wire the multi-replica prerequisites into the Helm chart and compose (#6347)
* fix(deploy): wire the multi-replica prerequisites into the Helm chart and compose

The chart shipped gateway.replicas: 1 with a README citing the long-closed
issue #3948, while its rendered config lacked run_ownership heartbeats and
db run events: raising replicas made every starting Pod write its peers'
lease-less runs off as orphans, and the Gateway's startup gate could not
notice because its worker count only sees one Pod.

Enable heartbeats and db run events in the default config, export
DEER_FLOW_MULTI_INSTANCE from the replica count for the startup gate,
generate and inject a shared AUTH_JWT_SECRET (per-Pod .jwt_secret files
logged users out across replicas), declare a surge-then-drain rollout
strategy, render a PodDisruptionBudget when replicas > 1, and bound
uvicorn's graceful shutdown inside a 90s termination grace period.
docker-compose gets the same uvicorn bound and a 60s stop_grace_period so
the 30s memory flush is no longer SIGKILLed. Docs and comments now list
what actually remains single-instance: IM channels, WeChat QR login and
browser tools.

* fix(deploy): honor existingAppSecret, declare multi-instance explicitly, cover the dev stack's shutdown

Follow-up to the multi-replica prerequisites wiring, addressing review:

- `deer-flow.appSecret` returns `existingAppSecret` when it is set. The
  gateway and frontend kept referencing the generated `<release>-app`
  Secret, which the chart stopped rendering, so every Pod sat in
  CreateContainerConfigError. The value is now declared in values.yaml.
- `gateway.multiInstance` (boolean, default null) overrides the
  replica-derived DEER_FLOW_MULTI_INSTANCE, PodDisruptionBudget and
  AUTH_JWT_SECRET requirement for deployments scaled with kubectl or an
  HPA, which change the replica count without a Helm upgrade.
- AUTH_JWT_SECRET is required, not optional, whenever the gateway is
  multi-instance: concurrently booting Pods otherwise race to write
  different `.jwt_secret` files on the home volume and sign sessions with
  different keys.
- `gateway.uvicornGracefulShutdownSeconds: 0` is honored instead of being
  swallowed by `default 10`.
- docker-compose-dev.yaml and dev-entrypoint.sh get the same uvicorn
  graceful-shutdown bound and `stop_grace_period: 60s`.
- The chart README documents the one-time sign-out when upgrading a
  release that predates AUTH_JWT_SECRET and how to seed the previous key
  into the Secret first.
- Tests read the shutdown budget from the Gateway through a shared helper
  instead of hard-coding 5/5/30, the raw-template string matches give way
  to `helm template` renders, and new renders cover multiInstance,
  existingAppSecret, a disabled PDB and a zero graceful-shutdown bound.

* docs(changelog): link the multi-replica deploy entry to #6347

* fix(deploy): budget every bounded teardown hook and flag the multiInstance PDB window

Review follow-ups on #6347:

- The shutdown budget modeled only channel stop + run drain + memory flush
  (40s). The lifespan also runs the startup trash sweep, the notification
  delivery worker stop, browser-session close and the MCP session-pool
  close, each bounded at _SHUTDOWN_HOOK_TIMEOUT_SECONDS, plus a 1s
  retrieval-warm wait: about 61s worst case. _gateway_shutdown_budget.py
  now reads all of them from the Gateway, a drift guard pins the hook
  count against app.py, and both compose files move stop_grace_period
  from 60s to 90s (Helm's 90s already had room: 5 + 10 + 61). Comments
  and the chart README cite the full arithmetic.
- gateway.multiInstance: true renders the PodDisruptionBudget immediately,
  which blocks drains of the single Pod until the scale-up happens; the
  README, values comment and both deployment guides say so and name
  podDisruptionBudget.enabled: false as the interim escape hatch.
2026-10-06 11:21:51 +08:00
asts
b88d8db5fc
fix(mcp): share rotating OAuth state with durable task calls (#6342)
* fix(mcp): share rotating OAuth state with durable task calls

* refactor(mcp): centralize task server selection

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 10:27:44 +08:00
Willem Jiang
7c6fb15f8e
chore(doc):update the changelog for new merged the PRs (#6345) 2026-10-06 10:22:51 +08:00
PeaceMaker-best
3606c4a098
feat(agents): add portable custom agent packages (#6127)
* feat(agents): add portable agent packages

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>

* fix(agents): show readable package import validation errors

---------

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 10:20:38 +08:00
Hyeonsang Cho
67d30f84ac
fix(sandbox): stop pinning a stale agent skill view once host bash is allowed (#6344)
* fix(sandbox): stop pinning a stale agent skill view once host bash is allowed

A restricted Agent run leaves a thread skill view behind. SandboxMiddleware
refreshes that view only while the provider can enforce skill isolation;
with host bash allowed LocalSandboxProvider cannot, so the middleware skips
it and restricted runs are rejected. The provider still mounted the view
whenever it existed, so later unrestricted runs on the thread kept the old
allowlist and could not read other enabled or newly added skills.

Look up the thread view only while isolation is supported, so those runs
get the shared projection the middleware expects. The view stays on disk
and is enforced again once host bash is turned off.

* docs(changelog): reference #6344 for the stale skill view fix

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 10:03:56 +08:00
Hyeonsang Cho
ab126d419c
fix(sandbox): keep line endings in local sandbox reads, writes and str_replace (#6343)
* fix(sandbox): keep line endings in local sandbox reads, writes and str_replace

LocalSandbox opened files in text mode with the default newline=None.
Reads translated CRLF to LF, so str_replace on a CRLF file rewrote every
line as LF and the read-before-write hash missed newline-only changes;
on Windows, writes turned LF content into CRLF (breaking `bash run.sh`).

Read with newline="\n" and write with newline="", keeping line endings
as stored like the remote providers; reads still end lines only at "\n",
the rule count_file_lines and the truncation marker number lines by.
str_replace spells an LF-written old_str/new_str with CRLF when the file
uses it, so multi-line edits still match CRLF files on any provider.

The tool-output budget middleware wrote externalized output the same
way: on Windows the file no longer matched its stamped blob ref and the
next model call deleted it when no blob store was configured. It now
writes with newline="" too.

* docs(changelog): reference #6343 for the line-ending fix

* fix(sandbox): end local grep lines where read_file does

find_grep_matches still opened files with universal newlines, so a bare
"\r" split a line for grep but not for LocalSandbox.read_file, and a hit's
line_number stopped matching the start_line that reads it back. Read with
newline="\n" like read_file, and fold a CRLF terminator back to "\n" so
$-anchored patterns keep matching CRLF lines as before.
2026-10-06 09:45:47 +08:00
spud
3f8f91d064
fix(mcp): isolate pooled sessions by ownership domain (#6133)
* fix(mcp): isolate pooled sessions by ownership domain

Separate deployment and personal stdio sessions even when they share the same runtime server name and thread scope. Preserve deployment call compatibility while carrying personal ownership through Agent wrappers, durable task calls, and disconnect eviction.

Add focused regression coverage for personal discovery wiring and domain-scoped disconnect cleanup.

* test(mcp): address ownership-domain review feedback

Accept the new wrapper keyword in the prefix-overlap test double and reuse MCPPoolDomain for internal task-caller ownership parameters.

* style(mcp): format ownership-domain changes

* style(mcp): apply repository ruff formatting
2026-10-06 09:08:25 +08:00
IronMurphy
9dd8e3e94b
fix(middleware): derive the externalized filename from the tool call id (#6107)
* fix(middleware): derive the externalized filename from the tool call id

_build_externalized_filename documents that the host-disk and sandbox paths
share it so both produce the identical naming scheme, but it used a random uuid
suffix and ignored tool_call_id: the same call got a different name every time,
so the two paths could not agree and externalizing one output twice left two
files behind. Derive the name from the call id instead, sanitised the same way
tool names are, and drop the now unused uuid import.

* fix(middleware): hash externalized call IDs and content

* fix(middleware): catch sandbox filename encoding failures

* fix(middleware): handle host output encoding failures

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 09:06:33 +08:00
Guo-Yixin
52edef4f12
fix(models): exclude thinking blocks from Claude cache breakpoints (#6339)
* fix(models): exclude thinking blocks from Claude cache breakpoints

* fix(models): declare Claude test SDK and clarify caching docs

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 09:05:55 +08:00
Guo-Yixin
d8bdd7552c
fix(models): tolerate null Codex response usage (#6335) 2026-10-06 08:11:05 +08:00
Daoyuan Li
2a81f913a5
fix(projects): synchronize document character-count cache (#6315) 2026-10-06 08:09:15 +08:00
Grapette.L
2523be658b
fix(discord): read allowed_guilds as IDs and deny every guild on an unreadable allowlist (#6338)
* test(discord): expose allowlist parsing failures instead of opening the bot

config.example.yaml documents channels.discord.allowed_guilds as a value that
"can also be a single guild ID", but the constructor iterates it: a scalar
string becomes one entry per character, and an unquoted YAML int raises
TypeError. Whatever remains is gated by truthiness, so an allowlist with no
parseable entry leaves the set empty and admits every guild.

* fix(discord): read allowed_guilds as IDs, and fail closed when unreadable

The allowlist was built by iterating the configured value, so the documented
single-guild shorthand ("can also be a single guild ID" in
config.example.yaml) became one entry per character: allowed_guilds: "321"
admitted guilds 1, 2 and 3 while blocking 321, and an unquoted YAML int raised
TypeError in the constructor. The empty result then read as "no allowlist", so
a restriction the operator configured could open the bot to every guild the bot
joined.

Parse both ID lists through helpers that treat a scalar as a one-entry list and
drop unreadable entries with a warning, following the telegram channel: an
allowlist with no parseable entry now denies every guild and logs at ERROR.
Mention exemptions keep their IDs verbatim, where an unreadable entry only
leaves mention_only in force.

* docs(discord): explain allowlist parsing and deny-all behavior

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 07:56:10 +08:00
XuRuuuy
a9c8f79dfe
fix(gateway): resolve the agent factory off the event loop (#6206)
* fix(gateway): resolve the agent factory off the event loop

start_run() called resolve_agent_factory() inline, which lazily imports
the full lead-agent assembly stack (middlewares, authz, MCP, jsonschema;
measured at ~8s cold on Windows, multi-second on slower filesystems).
start_run is the single choke point every run-creation path flows
through, so the first run after Gateway startup blocked the event loop
for the whole import, stalling every other request, stream and channel
on the same loop - the loop-stall family issue #5172 fixed for agent
construction itself (the checkpoint-state path already resolves through
the assembly pool).

Resolve the factory via asyncio.to_thread: the first call pays one
worker-thread hop for the import, later runs hit the cached module.
Regression test pins the contract with a probe module whose module-level
__getattr__ records the thread that performed the from-import.

* fix(gateway): isolate factory resolution in the assembly pool

Move import regressions into the strict blocking-I/O suite and cover default-executor isolation, request context, and failure-before-admission ordering.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 07:52:47 +08:00
haochen
8d5654363e
fix(wechat): enforce outbound limits while reading attachments (#6309)
* fix(wechat): enforce outbound limits while reading attachments

* fix(wechat): centralize outbound read overflow rejection

---------

Co-authored-by: Zhangchenhao-111 <285911435+Zhangchenhao-111@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 07:45:18 +08:00
Hyeonsang Cho
b9ee8af328
fix(skills): keep custom-skill edit and storage lookups off the event loop (#6332)
* fix(skills): keep custom-skill edit checks off the event loop

update_custom_skill built the user-scoped storage, ran the editability
probes, validated the frontmatter through a temporary directory and read
the content being replaced on the Gateway loop; only the write and history
append had been offloaded. Run the pre-scan checks in one worker and the
replaced-content read in another, matching rollback_custom_skill, and build
delete_custom_skill's storage off the loop as well.

* fix(skills): build skill storage off the loop in install and skill_manage

The same sweep that moved the custom-skill edit checks off the loop finds
two more storage lookups on it: the archive-install helper behind both
install routes, and the agent's skill_manage tool, whose lookup resolves
and stats config.yaml on every call. Offload both, and pin that the
caller's user context still reaches the storage lookup now that the edit
route resolves it inside a worker.

* docs(changelog): reference #6332 for the skill storage off-loop fix

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 07:20:04 +08:00
sxh
a72c6489da
fix(skills): windows interpreter scripts outside scripts/ escape security scanning (#6321)
* fix(skills): scan Windows interpreter scripts outside scripts/

CODE_SUFFIXES is the single definition of "this file is code" that
skills/AGENTS.md requires the installer and SkillScan to import, but it
listed .ps1 while omitting .bat/.cmd/.vbs/.wsf/.psm1. Those files
outside scripts/ therefore got is_code_file() == False, so SkillScan
skipped them as binaries (zero findings, not even
package-undecodable-script) and the installer never reached the
executable=True guardrail scan.

Add the five Windows interpreter extensions to the shared table so both
surfaces see them on every platform.

Refs #6320

* fix(skills): scan Windows script host sibling formats

---------

Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-06 07:17:22 +08:00
haotianqi666
aa7846823d
fix(uploads): accept uppercase markdown companions (#6308)
Co-authored-by: haotianqi666 <250384991+haotianqi666@users.noreply.github.com>
2026-10-06 07:10:55 +08:00
NanPan
c5b9964c25
fix(aio): keep renewal shutdown retryable (#6317)
* fix(aio): keep renewal shutdown retryable

* test(aio): cover renewal shutdown retry

* test(aio): stabilize renewal shutdown retry liveness

* fix(aio): preserve signal forwarding on shutdown timeout

* test(aio): match renewal shutdown timeout diagnostic
2026-10-06 07:08:33 +08:00
NanPan
9c5e5253ea
fix(telegram): retain polling worker on stop timeout (#6333)
* fix(telegram): retain polling ownership on stop timeout

* test(telegram): cover stop timeout ownership retry

* docs(channels): pin Telegram stop ownership
2026-10-06 07:00:09 +08:00
Nan Gao
5872d07e9a
fix(gateway): gate declared multi-instance deployments and require the redis stream bridge (#6328)
* fix(gateway): gate declared multi-instance deployments and require the redis stream bridge

GATEWAY_WORKERS / WEB_CONCURRENCY only count the uvicorn workers of one
process tree, so a Kubernetes Deployment with replicas > 1 (one worker
per Pod) passed every multi-worker startup check while each Pod's
startup orphan reconciliation wrote the other Pods' lease-less runs off
as crashed on every rolling update.

Add `deployment.multi_instance` as an explicit declaration that runs the
same gate, plus a DEER_FLOW_MULTI_INSTANCE env override so deploy tooling
can set the declaration from its replica count without editing
config.yaml (wiring the Helm chart to export it is a separate change).
Both the worker count and the declaration now also require a Redis
stream bridge and refuse an explicit memory sandbox ownership store,
since the memory bridge and ownership store are process-local. The
agent-storage divergence warning, the inbound webhook dedupe warning and
the WeChat QR-login guard honor the same declaration.

* docs(changelog): link the multi-instance gate entry to #6328

* fix(ci): bump chart config_version to 53 and keep AGENTS.md under the size limits

config.example.yaml moved to config_version 53 for the new deployment
section, so the Helm chart's embedded config and its README example
follow. The multi-instance notes in backend/AGENTS.md and
backend/app/gateway/AGENTS.md shrink to one line each: main already sits
within 55 bytes of the per-file hard limit for the gateway guide and
within 163 bytes of the chain hard limit under backend/, and the full
contract lives in backend/docs/CONFIGURATION.md.

* fix(gateway): name every active knob in the multi-process rollback

When GATEWAY_WORKERS > 1 and a multi-instance declaration are both
active, the refusal offered only "Set GATEWAY_WORKERS=1", so an operator
who followed it bounced through a second refusal on
DEER_FLOW_MULTI_INSTANCE / deployment.multi_instance at the next start.
The rollback now withdraws both knobs, and the browser-tools refusal
takes the same path instead of the worker-count-only message.

multi_instance_declaration() returns a frozen MultiInstanceDeclaration
(source, value) with knob/rollback properties instead of a display
string, so deps.py no longer branches on message prose. Tests pin the
combined rollback text and that following it actually clears the gate.

* docs(gateway): keep the gateway AGENTS.md under the size hard limit after merging main

main grew backend/app/gateway/AGENTS.md by 30 bytes, which left it 25 bytes
under the 49152-byte hard limit; the one-sentence multi-instance note from
this branch pushed the merged file to 49164. Shorten that sentence by 18
bytes so scripts/check_agent_guidance.py passes on the merge result.

* ci: re-run checks after test_stop_cancels_a_hung_driver_poll flaked on shard 4

tests/test_mcp_task_service.py::test_stop_cancels_a_hung_driver_poll timed
out once on backend-unit-tests (4) for 2c16206ac. The MCP task service and
its test are identical to main, main's run for cfe2023df passed, and 15
consecutive local runs pass; the test gives service.stop() a 1s budget
while cancellation drain allows 5s, so a loaded runner can exceed it.
Fork contributors cannot re-run a job, hence this empty commit.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 22:34:57 +08:00
Hyeonsang Cho
c2f31e440e
fix(knowledge): load the catalog's agent config off the event loop (#6313)
* fix(knowledge): load the catalog's agent config off the event loop

Both retrieval-catalog handlers called load_agent_config() directly from
async code. The agent store is synchronous: file IO on the file backend,
a sync SQLAlchemy round trip on the db backend, so a slow disk or
database stalled every other Gateway request. Offload the load with
asyncio.to_thread, matching agents.py, scheduled_tasks.py and the
run-admission path in services.py.

The strict Blockbuster anchors drive both handlers end to end against
real file and SQLite agent stores, plus the unknown-agent 404 path.

* docs(changelog): reference #6313 for the knowledge catalog fix

* Added the Chinese changelog entry back

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 22:17:09 +08:00
subhash7476
a558884c3a
perf(summarization): skip the summarizer LLM call for an input already proven no-op (#6318)
* perf(summarization): skip the summarizer LLM call for an input already proven no-op

When the exact rendered summarizer prompt (plus candidate models) previously
produced output byte-identical to the current summary_text, reuse that summary
instead of calling the LLM. Compaction proceeds through the normal path, so
messages, hooks and state transitions are unchanged.

- Only proven no-op inputs are cached (bounded LRU, thread-safe); failed
  attempts are never cached and any cache error falls back to the LLM call.
- The middleware:summarize event gains changes.llm_call_skipped.
- Applies to both compact_state and acompact_state.

Builds on the per-call telemetry in the P0 branch.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(summarization): keep fallback no-ops retryable

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 22:04:36 +08:00
Totoro
cdbcc0d9e9
feat(scheduled-tasks): show goal outcomes and end conditions in run history (#6326)
* docs(scheduler): document tool_enabled, goal notices and goal-evaluation change

Follow-up to #6229:
- CONFIGURATION.md: add scheduler.tool_enabled to the example and notes.
- CONFIGURATION.md, README (en/zh/pt): unmet goal runs send a goal-unmet
  notice instead of a completion notice, and three unmet runs add an
  auto-pause notice; the docs still said only success/failed notify.
- CHANGELOG (en/zh): the #6229 entry now states the default-on goal
  evaluation change for scheduled/webhook/autonomous goal runs and that
  evaluator usage counts toward run usage.

* feat(scheduled-tasks): show goal outcomes and end conditions in run history

Follow-up to #6229. The API already returns these fields; the tasks page
did not show them, so a met goal read as plain "Success", an unmet
reason code rendered as a red execution error, and the goal-unmet IM
notice pointed to a workspace showing neither objective nor verdict.

- Task detail shows the per-run goal, max automatic runs and end time
  when a conversation-created task has them.
- Goal runs show "Goal met" (or met relying on stated assumptions);
  unmet runs show a localized reason plus the host code in neutral
  styling. The run whose agent asked to stop the schedule is marked.
- Runs without a goal, and failed runs, render as before.

Read-only and frontend-only: no API change.

* docs(changelog): add #6326 scheduled-task goal outcome entry

* fix(scheduled-tasks): show readable unmet reasons instead of raw codes

Unmet runs now show only the localized reason; the host code moves to the
tooltip so it can still be matched with the IM notice. Codes without a
label are still shown verbatim.

* fix(scheduler): show readable goal reasons instead of internal codes

Task detail now localizes the host-written last_error values from goal and
stop finalization (agent stop, automatic pause, unmet reason codes) under a
matching label, and Chinese copy is tightened. Goal-unmet and auto-pause IM
notices state the reason in words instead of printing the raw code; unknown
codes are still not forwarded.

* docs(changelog): describe readable goal reasons in #6326 entry

* test(scheduler): pin host goal strings with a cross-language contract

Address review on #6326:
- Add contracts/scheduled_goal_notes_contract.json listing the task
  last_error strings and unmet reason codes the tasks page and IM notices
  translate. A backend test checks finalization constants, blocker and
  stand-down codes against it; a frontend test checks every listed value
  has a label.
- Move the two last_error strings in finalization.py into constants.
- Frontend reason lookup strips the blocked: prefix like the IM map and
  now covers goal_not_met_yet, with en/zh labels.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 21:56:04 +08:00
NanPan
78150e9b30
fix(subagents): keep isolated-loop shutdown ownership retryable (#6316)
* fix(subagents): retain isolated-loop shutdown ownership

* fix(subagents): reap stopped isolated loop before replacement

* test(subagents): cover pending-loop recovery after worker exit

* docs(subagents): document pending-loop recovery

* fix(subagents): revalidate pending loop recovery
2026-10-05 21:55:01 +08:00
siye566
d8ca2c9358
test(community): make URL offload regressions DNS-independent (#6314)
* test(community): make URL offload regressions DNS-independent

* test(community): reject unsupported DNS fixture families

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 21:53:55 +08:00
Willem Jiang
cfe2023df3
test(backend): run make test in four parallel shards (#6324) 2026-10-05 21:28:08 +08:00
Hyeonsang Cho
0223d949bf
fix(threads): remove a deleted thread's files off the event loop (#6319)
* fix(threads): remove a deleted thread's files off the event loop

DELETE /api/threads/{id} called shutil.rmtree over the thread directory
directly on the event loop, so deleting a thread with a large workspace
stalled every other Gateway request and live SSE stream until the whole
tree was gone. Branch copying in the same router already offloads its
filesystem work through run_file_io; deletion did not.

The removal now runs on the file-IO pool. It is wrapped in await_drained
rather than bare-awaited: cancelling the await abandons the worker, not
the removal, so the durable delete reservation would otherwise release
while files are still being deleted, and a newly admitted run could
write into a directory that is about to disappear.

* docs(changelog): reference #6319 in the thread delete off-loop entry

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 21:27:46 +08:00
Guo-Yixin
9f8a59da80
fix(models): skip undecodable Claude descriptor handoffs (#6323) 2026-10-05 20:21:31 +08:00
jay
7abb973f72
fix(harness): detect shell pipe variants with sudo, full paths and other shells (#6312)
* fix(harness): detect shell pipe variants with sudo, full paths and other shells

The shell-curl-pipe-shell rule only matched the bare 'curl ... | bash'
spelling. The variants that real install scripts actually ship —
'sudo bash', 'sudo -E bash', '/bin/bash', 'zsh'/'dash'/'fish', and
line-continued pipes — produced no finding at all, so a HIGH-severity
remote-code-execution pattern went unreported (#6310 lists the table).

The pattern now tolerates an optional sudo/doas-style flag prefix, full
interpreter paths, and line continuations, and covers the common shell
names. Non-shell pipes (jq, tee) stay unreported.

Fixes #6310

* test(skillscan): restore lost parametrize decorator in merge resolution

* test(skillscan): fix indentation of the http_host parametrize block

* style: ruff-format the skillscan test file

* fix(skillscan): bound pipe matching and handle real continuations

* fix(skillscan): tolerate continuations between sudo, its flags and the shell

Verified undetected until now: `curl ... | sudo \
  bash` and the -E
variant (LF and CRLF). Whitespace runs inside the sudo segment use the same
disjoint (?:\s|\\r?\n)+ shape as the pipe side; value-bearing flags such as
sudo -u root stay intentionally out of scope and are documented in the skills
AGENTS.md note. 17 positive / 6 negative variants verified red-to-green.
2026-10-05 19:53:01 +08:00
Grapette.L
d67a9e4d74
fix(extensions): measure the team response size gate in UTF-8 bytes (#6274)
* fix(extensions): measure the team response size gate in UTF-8 bytes

json.dumps escapes non-ASCII by default, so a CJK clarification answer
counted ~6 escaped bytes per character and an emoji 12, while the plugin
documents 4,000 characters / 8,000 UTF-8 bytes and its own UI validates
exactly those two bounds. A 漢 answer of 2,000 characters (6,000 UTF-8
bytes, inside the bound) was refused by the gate.

Serialize with ensure_ascii=False so the gate measures the UTF-8 bytes it
names, matching the budget walkers in the same module.

* fix(agent-teams): honor the raw UTF-8 response boundary

Measure string answers without JSON quotes or escapes, retain the structured JSON byte limit, and reject malformed Unicode with the normal response validation error. Add boundary and state-preservation regressions and document the response budgets.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 19:45:54 +08:00
wgwfn3
7d50f3fc7b
fix(sandbox): keep the exit marker last when appending audit warnings (#6307)
* fix(sandbox): keep the exit marker last when appending audit warnings

The medium-risk warning was appended after the bash exit marker and the ToolMessage was rebuilt from four fields. The subagent executor anchors its exit-status regex on the trailing 'Exit Code: N' (and fullmatches 'Command exited with code N'), so a failed command was harvested as success; the rebuild also dropped deerflow_tool_meta, artifact and id. Insert the warning before a trailing marker, leave a bare remote marker untouched, and copy the message instead of rebuilding it.

Document the warning placement in agents/middlewares/AGENTS.md.

* docs(changelog): reference #6307 in the sandbox audit entry

* fix(sandbox): keep a trailing remote exit marker last as well

Match the full sandbox.tools._BASH_EXIT_MARKER_TAIL_RE alternation when
choosing where to insert the medium-risk warning, so output that ends in
`Command exited with code N` after other text keeps the marker last too.

* test(sandbox): import the middleware tail regex inside its test

A module-level import of the new name made the whole file fail to collect
against main, hiding which tests the fix turns green.
2026-10-05 19:36:05 +08:00
PeaceMaker-best
fcee85b822
feat(storage): persist viewed images as shared blobs (#6160)
Store validated image bytes in the configured content-addressed blob backend and resolve checkpointed refs before local compatibility paths.

Refs: #4189

Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
2026-10-05 19:06:16 +08:00
JasonH
22623d1b00
fix(projects): validate explicit document names before staging (#6287) 2026-10-05 18:38:22 +08:00
LingYi
3a2f768d4c
fix(tui): isolate callbacks from interrupted runs (#6302)
* fix(tui): isolate callbacks from interrupted runs

* fix(tui): recover when agent worker creation fails

---------

Co-authored-by: LingYi-Liang <270316896+LingYi-Liang@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 16:47:55 +08:00
Daoyuan Li
4d9f1d7aa0
fix(extensions): exclude unadmitted team waits from active budget (#6298)
* fix(extensions): exclude unadmitted team waits from active budget

* docs(extensions): explain frozen team input reservation
2026-10-05 16:10:07 +08:00
Guo-Yixin
2970aa4323
fix(scripts): reject invalid Claude Keychain credential containers (#6281)
* fix(scripts): validate Claude Keychain export credentials

* fix(scripts): reuse validated Keychain access token
2026-10-05 15:39:32 +08:00
NanPan
6cdc600895
fix(e2b): keep shutdown retryable while maintenance workers are live (#6244)
* fix(e2b): retain teardown ownership across maintenance timeout

* fix(e2b): fence acquisition and preserve signal forwarding

* test(e2b): cover shutdown admission and signal forwarding

* docs(e2b): pin shutdown admission and signal invariants

* fix(e2b): scope deferred-cleanup signal handling

* test(e2b): exercise dedicated deferred-cleanup signal error

* style(tests): format E2B shutdown regression tests

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 15:37:33 +08:00
Hyeonsang Cho
7f2c0fbe0c
fix(persistence): wait out a peer's Postgres bootstrap past command_timeout (#6306)
* fix(persistence): wait out a peer's Postgres bootstrap past command_timeout

The bootstrap advisory lock was taken with a blocking pg_advisory_lock on
the app engine, whose asyncpg command_timeout (30s by default) also bounds
that statement. A second Gateway instance therefore failed startup with
TimeoutError whenever another instance's migration outlasted the timeout.

Poll the non-blocking pg_try_advisory_lock instead: each attempt returns
immediately and stays bounded by command_timeout, while the wait lasts as
long as the holder's migration. The wait is logged once.

* docs(changelog): reference #6306 in the bootstrap lock entry

* docs(persistence): describe polled bootstrap lock in the migrations guide

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 15:33:29 +08:00
jay
fff9d185c5
fix(harness): normalize local HTTP host case in skillscan (#6280)
* fix(harness): normalize local HTTP host case in skillscan

_LOCAL_HTTP_HOSTS holds lowercase names and every membership test
compared the extracted host verbatim, so an endpoint written with any
uppercase letter (e.g. http://LOCALHOST:8080) was classified as an
external endpoint: non-root deployments failed with EACCES while root
deployments silently wrote plans outside the workspace.

Normalize the host to lowercase in _http_host and in the
network-cleartext/local scan branch so all three call sites agree.

Fixes #6278

* test(skillscan): cover local host case and external endpoint controls

* fix(skillscan): parse IPv6 hosts and detect metadata case variants

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 15:17:03 +08:00
Guo-Yixin
57a2d7fa70
fix(models): decode CLI credential files as UTF-8 (#6277)
* fix(models): decode CLI credential files as UTF-8

* fix(doctor): accept BOM CLI credentials consistently
2026-10-05 14:47:14 +08:00
quifox
420b39ed90
fix(acp): apply invocation timeout to initialization and session creation (#6292)
* fix(acp): apply invocation timeout to initialization and session creation

* fix(acp): preserve SDK timeout errors before the invocation deadline
2026-10-05 14:39:59 +08:00
Shxiao
b85e4ae74b
fix(channels): serialize lifecycle mutations under per-channel lock (#6304)
* fix(channels): serialize lifecycle mutations under per-channel lock

* fix(channels): invalidate stale config snapshots across lifecycle mutations under per-channel lock
2026-10-05 14:35:51 +08:00
Magno Gouveia
c0895d295b
docs(i18n): add Brazilian Portuguese README translation (#6286)
* docs(i18n): add Brazilian Portuguese README translation

* docs(i18n): disambiguate one-line agent setup heading in pt readme

* docs: keep inherited agent guidance within budget

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 13:38:46 +08:00
NanPan
8237dd75b3
fix(boxlite): keep private-loop shutdown retryable (#6283)
* fix(boxlite): keep private-loop shutdown retryable

* test(boxlite): cover private-loop shutdown retry

* docs(sandbox): pin BoxLite loop shutdown ownership

* fix(ci): format BoxLite shutdown changes and trim guidance

* fix(boxlite): drain acquisitions before shutdown teardown

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 13:38:17 +08:00
Hyeonsang Cho
52046aa82e
fix(projects): convert shelf documents outside the SQLite write lock (#6305)
* fix(projects): convert shelf documents outside the SQLite write lock

Lazy conversion ran pymupdf/markitdown inside the BEGIN IMMEDIATE
transaction that serializes the derived/converted.md publish against
trash and purge. On SQLite that lock is database-wide, so every unrelated
writer (run status, thread metadata, the scheduler) waited for the whole
conversion and failed with "database is locked" past busy_timeout (30s).

Conversion now writes into .staging/ outside any transaction. The
document row lock is taken only to revalidate the live row and atomically
rename the output into place (publish_under_live_lock, renamed from
convert_under_live_lock); a row trashed or purged meanwhile discards the
staged output, so nothing is published after purge. A read-only probe
skips converting rows that are already gone, a failed conversion is
still revalidated so a purged row reports content_missing, and the
conversion is drained on cancellation so its staging file is always
removed.

* docs(changelog): reference #6305 in the shelf conversion lock entry

* fix(projects): share one conversion across concurrent first reads

A first-read conversion holds a bounded file-IO pool thread for its
whole duration. With the conversion moved out of the row lock,
concurrent first reads of one document each converted on their own
thread, so a burst of reads could starve unrelated offloaded file work.

Route the conversion through a single-flight keyed by (user, document,
project): the first reader leads and the rest wait on its result
without holding a pool thread. The map holds concurrent.futures.Future
values under a threading.Lock because the read tool also runs on the
isolated subagent loop. Followers wait through asyncio.shield, so a
cancelled follower cannot cancel the shared future, and a leader that
is cancelled or raises hands the work to a follower instead of failing
it.
2026-10-05 13:34:19 +08:00
siye566
e8f1e269e5
fix(uploads): exclude punctuated numeric columns from document outlines (#6266)
* fix(uploads): exclude punctuated numeric columns from outlines

* fix(uploads): filter currency and later numeric outline columns

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 13:20:00 +08:00
sxh
ef76f085a3
fix(skills): stop reporting an env path, shebang or flag as an env dump (#6297)
* fix(skills): stop reporting an env path, shebang or flag as an env dump

* fix(skills): report shell env dumps only at a command position

`shell-env-dump` matched the word `env` anywhere it was not preceded by a path, dot, dash or word character, so ordinary argument text (`echo env`) and comment text (`# env`, `# export -p`) were still reported as environment dumps. Anchor the match to a shell command position -- the start of a line or right after a `;`, `&`, `|`, `(`, `)` or `{` separator, optionally behind `NAME=value` assignment prefixes -- and add regression cases for argument and comment text.

* fix(skills): ignore an env dump inside a comment or heredoc body

`shell-env-dump` matches `env`, `printenv` and `export -p` at a command
position, but the text it sees is still raw shell. A `;` inside a comment
(`# documentation; env is only an example`) and a bare `env` line inside
heredoc data both sit at a command position on paper while the shell runs
neither, so both were reported as environment dumps.

Reduce the text to shell code before matching: a quote-aware pass strips
comments (only a `#` at a word start opens one) and blanks heredoc bodies,
covering `<<-` tab stripping and quoted delimiters, while keeping one output
line per input line so reported line numbers stay correct. Real commands
after a separator (`true; env`) and on the line that opens a heredoc
(`env <<EOF`) still report.

* fix(skills): report env dumps after a keyword and stop masking ${#...}

The command-position anchor for `shell-env-dump` accepted only a line start or a
`;`, `&`, `|`, `(`, `)` or `{` separator, so real dumps were dropped: after a
reserved word that introduces a command (`if true; then env; fi`,
`while true; do printenv; done`), after `exec` (`exec env`) and inside a backtick
substitution (``V=`env` ``). Accept those positions too, with a guard so a path
component (`./then env`) does not arm the anchor, and add the backtick to the
separator set.

A `{` directly before a `#` is not a word start -- bash reads `${#HOME}` as the
length operator and `}#` as part of a word, while a brace group needs the space
of `{ # ...`. The comment heuristic no longer treats `{` and `}` as comment
openers, so `n=${#HOME}; env` keeps its `; env` and is reported again, while
`${env}` (a variable read, not a command) stays silent.

---------

Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
2026-10-05 12:25:41 +08:00
Daoyuan Li
49dcd49f91
fix(agents): keep malformed todo statuses recoverable (#6299)
* fix(agents): keep malformed todo statuses recoverable

* docs(agents): fit todo guidance within inherited size limit

* docs(agents): clarify todo attribution guidance

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-05 12:21:39 +08:00
Jun
57d946b730
fix(browser): drain live cleanup before releasing session (#6284)
* fix(browser): drain live cleanup before releasing session

* test(browser): cover cancellation precedence during cleanup

* style(browser): separate cancellation regressions
2026-10-05 12:15:21 +08:00
siye566
335c383147
fix(paths): reject Windows console device aliases (#6279) 2026-10-05 10:25:51 +08:00