* test(discord): expose allowlist parsing failures instead of opening the bot
config.example.yaml documents channels.discord.allowed_guilds as a value that
"can also be a single guild ID", but the constructor iterates it: a scalar
string becomes one entry per character, and an unquoted YAML int raises
TypeError. Whatever remains is gated by truthiness, so an allowlist with no
parseable entry leaves the set empty and admits every guild.
* fix(discord): read allowed_guilds as IDs, and fail closed when unreadable
The allowlist was built by iterating the configured value, so the documented
single-guild shorthand ("can also be a single guild ID" in
config.example.yaml) became one entry per character: allowed_guilds: "321"
admitted guilds 1, 2 and 3 while blocking 321, and an unquoted YAML int raised
TypeError in the constructor. The empty result then read as "no allowlist", so
a restriction the operator configured could open the bot to every guild the bot
joined.
Parse both ID lists through helpers that treat a scalar as a one-entry list and
drop unreadable entries with a warning, following the telegram channel: an
allowlist with no parseable entry now denies every guild and logs at ERROR.
Mention exemptions keep their IDs verbatim, where an unreadable entry only
leaves mention_only in force.
* docs(discord): explain allowlist parsing and deny-all behavior
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): resolve the agent factory off the event loop
start_run() called resolve_agent_factory() inline, which lazily imports
the full lead-agent assembly stack (middlewares, authz, MCP, jsonschema;
measured at ~8s cold on Windows, multi-second on slower filesystems).
start_run is the single choke point every run-creation path flows
through, so the first run after Gateway startup blocked the event loop
for the whole import, stalling every other request, stream and channel
on the same loop - the loop-stall family issue #5172 fixed for agent
construction itself (the checkpoint-state path already resolves through
the assembly pool).
Resolve the factory via asyncio.to_thread: the first call pays one
worker-thread hop for the import, later runs hit the cached module.
Regression test pins the contract with a probe module whose module-level
__getattr__ records the thread that performed the from-import.
* fix(gateway): isolate factory resolution in the assembly pool
Move import regressions into the strict blocking-I/O suite and cover default-executor isolation, request context, and failure-before-admission ordering.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skills): scan Windows interpreter scripts outside scripts/
CODE_SUFFIXES is the single definition of "this file is code" that
skills/AGENTS.md requires the installer and SkillScan to import, but it
listed .ps1 while omitting .bat/.cmd/.vbs/.wsf/.psm1. Those files
outside scripts/ therefore got is_code_file() == False, so SkillScan
skipped them as binaries (zero findings, not even
package-undecodable-script) and the installer never reached the
executable=True guardrail scan.
Add the five Windows interpreter extensions to the shared table so both
surfaces see them on every platform.
Refs #6320
* fix(skills): scan Windows script host sibling formats
---------
Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): gate declared multi-instance deployments and require the redis stream bridge
GATEWAY_WORKERS / WEB_CONCURRENCY only count the uvicorn workers of one
process tree, so a Kubernetes Deployment with replicas > 1 (one worker
per Pod) passed every multi-worker startup check while each Pod's
startup orphan reconciliation wrote the other Pods' lease-less runs off
as crashed on every rolling update.
Add `deployment.multi_instance` as an explicit declaration that runs the
same gate, plus a DEER_FLOW_MULTI_INSTANCE env override so deploy tooling
can set the declaration from its replica count without editing
config.yaml (wiring the Helm chart to export it is a separate change).
Both the worker count and the declaration now also require a Redis
stream bridge and refuse an explicit memory sandbox ownership store,
since the memory bridge and ownership store are process-local. The
agent-storage divergence warning, the inbound webhook dedupe warning and
the WeChat QR-login guard honor the same declaration.
* docs(changelog): link the multi-instance gate entry to #6328
* fix(ci): bump chart config_version to 53 and keep AGENTS.md under the size limits
config.example.yaml moved to config_version 53 for the new deployment
section, so the Helm chart's embedded config and its README example
follow. The multi-instance notes in backend/AGENTS.md and
backend/app/gateway/AGENTS.md shrink to one line each: main already sits
within 55 bytes of the per-file hard limit for the gateway guide and
within 163 bytes of the chain hard limit under backend/, and the full
contract lives in backend/docs/CONFIGURATION.md.
* fix(gateway): name every active knob in the multi-process rollback
When GATEWAY_WORKERS > 1 and a multi-instance declaration are both
active, the refusal offered only "Set GATEWAY_WORKERS=1", so an operator
who followed it bounced through a second refusal on
DEER_FLOW_MULTI_INSTANCE / deployment.multi_instance at the next start.
The rollback now withdraws both knobs, and the browser-tools refusal
takes the same path instead of the worker-count-only message.
multi_instance_declaration() returns a frozen MultiInstanceDeclaration
(source, value) with knob/rollback properties instead of a display
string, so deps.py no longer branches on message prose. Tests pin the
combined rollback text and that following it actually clears the gate.
* docs(gateway): keep the gateway AGENTS.md under the size hard limit after merging main
main grew backend/app/gateway/AGENTS.md by 30 bytes, which left it 25 bytes
under the 49152-byte hard limit; the one-sentence multi-instance note from
this branch pushed the merged file to 49164. Shorten that sentence by 18
bytes so scripts/check_agent_guidance.py passes on the merge result.
* ci: re-run checks after test_stop_cancels_a_hung_driver_poll flaked on shard 4
tests/test_mcp_task_service.py::test_stop_cancels_a_hung_driver_poll timed
out once on backend-unit-tests (4) for 2c16206ac. The MCP task service and
its test are identical to main, main's run for cfe2023df passed, and 15
consecutive local runs pass; the test gives service.stop() a 1s budget
while cancellation drain allows 5s, so a loaded runner can exceed it.
Fork contributors cannot re-run a job, hence this empty commit.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* perf(summarization): skip the summarizer LLM call for an input already proven no-op
When the exact rendered summarizer prompt (plus candidate models) previously
produced output byte-identical to the current summary_text, reuse that summary
instead of calling the LLM. Compaction proceeds through the normal path, so
messages, hooks and state transitions are unchanged.
- Only proven no-op inputs are cached (bounded LRU, thread-safe); failed
attempts are never cached and any cache error falls back to the LLM call.
- The middleware:summarize event gains changes.llm_call_skipped.
- Applies to both compact_state and acompact_state.
Builds on the per-call telemetry in the P0 branch.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(summarization): keep fallback no-ops retryable
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs(scheduler): document tool_enabled, goal notices and goal-evaluation change
Follow-up to #6229:
- CONFIGURATION.md: add scheduler.tool_enabled to the example and notes.
- CONFIGURATION.md, README (en/zh/pt): unmet goal runs send a goal-unmet
notice instead of a completion notice, and three unmet runs add an
auto-pause notice; the docs still said only success/failed notify.
- CHANGELOG (en/zh): the #6229 entry now states the default-on goal
evaluation change for scheduled/webhook/autonomous goal runs and that
evaluator usage counts toward run usage.
* feat(scheduled-tasks): show goal outcomes and end conditions in run history
Follow-up to #6229. The API already returns these fields; the tasks page
did not show them, so a met goal read as plain "Success", an unmet
reason code rendered as a red execution error, and the goal-unmet IM
notice pointed to a workspace showing neither objective nor verdict.
- Task detail shows the per-run goal, max automatic runs and end time
when a conversation-created task has them.
- Goal runs show "Goal met" (or met relying on stated assumptions);
unmet runs show a localized reason plus the host code in neutral
styling. The run whose agent asked to stop the schedule is marked.
- Runs without a goal, and failed runs, render as before.
Read-only and frontend-only: no API change.
* docs(changelog): add #6326 scheduled-task goal outcome entry
* fix(scheduled-tasks): show readable unmet reasons instead of raw codes
Unmet runs now show only the localized reason; the host code moves to the
tooltip so it can still be matched with the IM notice. Codes without a
label are still shown verbatim.
* fix(scheduler): show readable goal reasons instead of internal codes
Task detail now localizes the host-written last_error values from goal and
stop finalization (agent stop, automatic pause, unmet reason codes) under a
matching label, and Chinese copy is tightened. Goal-unmet and auto-pause IM
notices state the reason in words instead of printing the raw code; unknown
codes are still not forwarded.
* docs(changelog): describe readable goal reasons in #6326 entry
* test(scheduler): pin host goal strings with a cross-language contract
Address review on #6326:
- Add contracts/scheduled_goal_notes_contract.json listing the task
last_error strings and unmet reason codes the tasks page and IM notices
translate. A backend test checks finalization constants, blocker and
stand-down codes against it; a frontend test checks every listed value
has a label.
- Move the two last_error strings in finalization.py into constants.
- Frontend reason lookup strips the blocked: prefix like the IM map and
now covers goal_not_met_yet, with en/zh labels.
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(harness): detect shell pipe variants with sudo, full paths and other shells
The shell-curl-pipe-shell rule only matched the bare 'curl ... | bash'
spelling. The variants that real install scripts actually ship —
'sudo bash', 'sudo -E bash', '/bin/bash', 'zsh'/'dash'/'fish', and
line-continued pipes — produced no finding at all, so a HIGH-severity
remote-code-execution pattern went unreported (#6310 lists the table).
The pattern now tolerates an optional sudo/doas-style flag prefix, full
interpreter paths, and line continuations, and covers the common shell
names. Non-shell pipes (jq, tee) stay unreported.
Fixes#6310
* test(skillscan): restore lost parametrize decorator in merge resolution
* test(skillscan): fix indentation of the http_host parametrize block
* style: ruff-format the skillscan test file
* fix(skillscan): bound pipe matching and handle real continuations
* fix(skillscan): tolerate continuations between sudo, its flags and the shell
Verified undetected until now: `curl ... | sudo \
bash` and the -E
variant (LF and CRLF). Whitespace runs inside the sudo segment use the same
disjoint (?:\s|\\r?\n)+ shape as the pipe side; value-bearing flags such as
sudo -u root stay intentionally out of scope and are documented in the skills
AGENTS.md note. 17 positive / 6 negative variants verified red-to-green.
* fix(harness): normalize local HTTP host case in skillscan
_LOCAL_HTTP_HOSTS holds lowercase names and every membership test
compared the extracted host verbatim, so an endpoint written with any
uppercase letter (e.g. http://LOCALHOST:8080) was classified as an
external endpoint: non-root deployments failed with EACCES while root
deployments silently wrote plans outside the workspace.
Normalize the host to lowercase in _http_host and in the
network-cleartext/local scan branch so all three call sites agree.
Fixes#6278
* test(skillscan): cover local host case and external endpoint controls
* fix(skillscan): parse IPv6 hosts and detect metadata case variants
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(agents): keep malformed todo statuses recoverable
* docs(agents): fit todo guidance within inherited size limit
* docs(agents): clarify todo attribution guidance
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: include completed child usage in lead budget checks
* test: require completed usage middleware in lead budget regressions
* docs: fit middleware guidance within the size budget
Condense the subagent, loop, and token budget guard descriptions while retaining their limits, lifecycle rules, and completed child usage ordering.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(extensions): add full Custom Agent team collaboration plugin
* fix(extensions): register agent teams in the capability catalog
* fix(agent-teams): recover incomplete teams and preserve request capacity
* fix(agent-teams): preserve clarification waits and active handoff lineage
* fix(agent-teams): align mention and input text limits
* fix(gateway): read uvicorn's WEB_CONCURRENCY in the remaining worker-count readers
* fix(gateway): name the worker spelling an operator can act on
The browser refusal and the dedupe warning both took the count but printed
GATEWAY_WORKERS, so on the launches that pass uvicorn no --workers the advice
silences the gate while the extra processes keep starting. Resolve which
variable set the count and print that, and let an unparsable GATEWAY_WORKERS
fall through to WEB_CONCURRENCY instead of reporting a single worker.
Also stop the comments from claiming the readers are equivalent: they share
the two spellings and the blank-means-unset rule, but channel_connections.py
turns an unparsable value into 0 and refuses while these stay inert.
* docs: state the single-worker requirement under both spellings
The browser tool group and the inbound dedupe warning now refuse or warn on
WEB_CONCURRENCY as well, so guidance that names only GATEWAY_WORKERS=1 is
incomplete for the shipped Dockerfile and serve.sh launches, where uvicorn
takes the process count from WEB_CONCURRENCY.
* test(features): isolate the browser capability surface from the shell's worker count
browser_capability() asks for the Gateway worker count, which this change makes
readable from WEB_CONCURRENCY, so an exported count in the invoking shell turned
test_features_reports_browser_control_enabled_when_configured_and_runtime_available
into a failure that has nothing to do with the router.
* feat(runtime): add P0 per-LLM-call telemetry and summarization no-op counter
Measurement only; no runtime behavior changes.
- RunJournal enriches llm.ai.response / llm.error metadata with
langchain_run_id, caller_category, provider, model, stop_reason, status,
token counts and rendered request size (chars). Unavailable values are None.
- Summarization middleware emits a middleware:summarize event per summarizer
call and counts outputs identical to the prior summary_text. The call is
never skipped.
- All telemetry fails soft.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(runtime): register summarize middleware tag, document telemetry contract
- Add MIDDLEWARE_SUMMARIZE_TAG to the middleware event catalog and the
contract's known_tags; emit the middleware:summarize event with it.
- Document the optional per-call telemetry metadata on llm.ai.response and
llm.error in contracts/run_event_stream_contract.json.
- Measure request size by walking string leaves instead of json.dumps on the
event-loop thread.
- Initialise summarization call/no-op counters in __init__.
- Move RunJournal telemetry docs to the runtime guide.
- Add tests: llm.error without a start, replayed on_llm_end, and a DB-store
metadata round trip.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(runtime): refresh token counters on usage replay
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skillscan): report secrets bound by unpacking and container literals
The AST walk behind secret-env-assignment claims "precision, never less
coverage", but it only read a single assignment target and a bare string
value, so a credential moved into `host, api_key = endpoint, "..."` or
into `api_key = ["..."]` stopped being reported even though the pre-#5648
line sweep still reported the same bytes.
Pair unpacking targets with values positionally (fixed head and tail only,
so a runtime-sized slice names nothing) and assert on each literal a
container or conditional holds, in source order.
* fix(skillscan): distinguish mapping labels from credentials
---------
Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(telegram): deny all users when allowed_users has no valid ID
TelegramChannel parsed channels.telegram.allowed_users with int() and
dropped every entry that failed without logging, and _check_user treated
an empty result as "no allowlist". A list of @usernames therefore opened
the bot to everyone. A single string ID was iterated character by
character ("123456" allowed users 1-6 and blocked 123456), which is what
an allowed_users: $ENV reference resolves to, and null or a bare integer
crashed the channel at startup.
The allowlist is now parsed into frozenset[int] | None: None (unset,
null, empty list, blank string) still allows everyone, a single ID is a
one-entry list, entries that are not positive numeric IDs (@usernames,
floats, booleans, comma-joined strings) are dropped with a warning, and
a configured list with no valid ID denies every user and logs an error.
* docs(changelog): reference #6230 for the telegram allowed_users fix
* fix(telegram): say "positive" in the allowed_users drop warning
0 and negative integers are numeric but still dropped, so "expected one
numeric Telegram user ID" misdescribed them. The warning now asks for a
positive numeric ID, matching _parse_telegram_user_id, and the drop test
covers 0 and -5 alongside @alice.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): ignore literal base tags in HTML artifact previews
* fix(frontend): address HTML preview base review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs(readme): sync TOC with headings and fix deploy.sh paths
* docs(readme): move skills content stranded in the RAGFlow chapter back into Skills & Tools
* fix(browser): pin browser connections to the SSRF-vetted addresses
The navigate screen and the per-request guard resolve a hostname to vet
it, but Chromium resolved the name again to open the connection. A
rebinding DNS server could answer both checks with a public address and
the connection with a private or cloud-metadata one, and the browser
followed it.
Launch each private browser through a per-session loopback SOCKS5 proxy.
Over SOCKS5 Chromium sends the hostname itself, so the proxy resolves it
exactly once through the session's egress resolver and connects only to
the addresses that resolver vetted. resolve_browser_egress applies the
same allow_private_addresses policy as validate_browser_url, on top of a
new url_safety.resolve_public_addresses helper that
validate_public_http_url now shares. Loopback is proxied explicitly with
bypass "<-loopback>". The tools, navigate_and_capture and the Live
WebSocket all pass the resolver, because a thread's first caller fixes
its launch options.
CDP-attached Chrome is unchanged. Delegated fetch services (crawl4ai,
Browserless, fastcrw) still resolve on their own side, which the Gateway
cannot pin.
* docs(changelog): reference #6201 in the browser egress pinning entry
* fix(browser): bound egress proxy faults and close, and record the WebRTC gap
Review follow-ups for the pinned browser egress proxy:
- A resolver exception other than ValueError escaped the connection
handler as an unretrieved task error. Log it with its traceback and
reply SOCKS5 general failure (0x01), so a resolver fault stays
distinguishable from a policy refusal (0x02).
- Bound the resolve with its own timeout. The handshake timeout did not
cover it, so a wedged getaddrinfo held the half-open tunnel until
Chromium gave up. A timeout replies host unreachable.
- Bound close(): Server.wait_closed() also waits for a connection whose
handler registered after the writer sweep, which could stall session
close and eviction for the 15 s handshake. A handler that starts after
close now drops its connection without opening a tunnel.
- Docs said a launched browser connects "solely" through the proxy.
WebRTC UDP does not traverse a SOCKS proxy, so AGENTS.md, README (en/zh)
and both changelogs now scope the guarantee to TCP and name the gap.
* fix(tui): preserve multiline pasted input
The TUI composer used a single-line `Input`, so a bracketed paste
containing code, a stack trace, or multiple paragraphs kept only the
first line when submitted. The composer is now a multiline `TextArea`:
Enter still submits, Up/Down move inside the document before falling
back to input history at the first/last line, and the slash palette is
unchanged.
Originally #4630 by @felix-windsor, cherry-picked onto current main;
conflicts in app.py key gating (transcript paging keys), test imports,
and the relocated TUI docs were resolved by hand.
Fixes#4613
* fix(tui): submit multiline input unchanged and document the widgets layer
Address the review on #4630:
- Send non-command input to the agent and history exactly as typed or
pasted. Only the empty-input check uses strip(), so leading
indentation and trailing newlines in pasted code and stack traces
survive. Slash commands are still trimmed so they resolve as before.
- widgets/composer.py imports Textual, so the module-layout notes in
tui/AGENTS.md and docs/TUI.md now exempt widgets/ alongside app.py.
- Cover pressing Down on the last document line (history fallback),
whitespace-only input, and slash commands with surrounding whitespace.
Co-authored-by: Felix <felixwindsor3344@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(tui): treat soft-wrapped rows as composer navigation boundaries
TextArea soft-wraps long lines, and its Up/Down move between the wrapped
rows of one logical line. The history fallback only looked at logical
rows, so Up from the second visual row of a long prompt recalled history
instead of moving the cursor, and Down from its first visual row could
replace the draft with history's empty value.
Use the TextArea navigator's first/last wrapped-row checks for the
boundaries, and cover wrapped prompts in both directions. Also restore a
tui/AGENTS.md sentence about PageUp/PageDown to the app.py entry it
belongs to.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Felix <felixwindsor3344@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
