* fix(gateway): read uvicorn's WEB_CONCURRENCY in the remaining worker-count readers
* fix(gateway): name the worker spelling an operator can act on
The browser refusal and the dedupe warning both took the count but printed
GATEWAY_WORKERS, so on the launches that pass uvicorn no --workers the advice
silences the gate while the extra processes keep starting. Resolve which
variable set the count and print that, and let an unparsable GATEWAY_WORKERS
fall through to WEB_CONCURRENCY instead of reporting a single worker.
Also stop the comments from claiming the readers are equivalent: they share
the two spellings and the blank-means-unset rule, but channel_connections.py
turns an unparsable value into 0 and refuses while these stay inert.
* docs: state the single-worker requirement under both spellings
The browser tool group and the inbound dedupe warning now refuse or warn on
WEB_CONCURRENCY as well, so guidance that names only GATEWAY_WORKERS=1 is
incomplete for the shipped Dockerfile and serve.sh launches, where uvicorn
takes the process count from WEB_CONCURRENCY.
* test(features): isolate the browser capability surface from the shell's worker count
browser_capability() asks for the Gateway worker count, which this change makes
readable from WEB_CONCURRENCY, so an exported count in the invoking shell turned
test_features_reports_browser_control_enabled_when_configured_and_runtime_available
into a failure that has nothing to do with the router.
* fix(gateway): read uvicorn's WEB_CONCURRENCY in the multi-worker safety gates
The SQLite/scheduler gates in deps.py derived the worker count from
GATEWAY_WORKERS alone, but backend/Dockerfile and scripts/serve.sh launch
uvicorn without --workers, and uvicorn then takes the count from
WEB_CONCURRENCY. A Gateway scaled that way starts N processes with every
gate inert, which is exactly what the gates exist to prevent.
Read both variables, preferring GATEWAY_WORKERS, treat a blank value as
unset the way the compose command's ${GATEWAY_WORKERS:-1} does, and name
the variable that actually set the count in the refusal text. The channel
connection gate already reads both (channel_connections.py).
* fix(gateway): fall through to the fallback worker count
A non-numeric GATEWAY_WORKERS made the shared reader return one worker
immediately, so on the launches that pass no --workers (backend/Dockerfile,
scripts/serve.sh, backend/Makefile gateway) an operator's
GATEWAY_WORKERS=abc plus WEB_CONCURRENCY=4 started four processes while every
gate stayed inert. Keep looking at the remaining spellings and report a single
worker only after all of them have been tried.
The gate file also needs to be independent of the invoking shell: with
WEB_CONCURRENCY exported, the pre-existing
test_gate_treats_invalid_env_as_single_worker already failed on this branch,
because the reader now consults that variable too.
* feat(runtime): add P0 per-LLM-call telemetry and summarization no-op counter
Measurement only; no runtime behavior changes.
- RunJournal enriches llm.ai.response / llm.error metadata with
langchain_run_id, caller_category, provider, model, stop_reason, status,
token counts and rendered request size (chars). Unavailable values are None.
- Summarization middleware emits a middleware:summarize event per summarizer
call and counts outputs identical to the prior summary_text. The call is
never skipped.
- All telemetry fails soft.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(runtime): register summarize middleware tag, document telemetry contract
- Add MIDDLEWARE_SUMMARIZE_TAG to the middleware event catalog and the
contract's known_tags; emit the middleware:summarize event with it.
- Document the optional per-call telemetry metadata on llm.ai.response and
llm.error in contracts/run_event_stream_contract.json.
- Measure request size by walking string leaves instead of json.dumps on the
event-loop thread.
- Initialise summarization call/no-op counters in __init__.
- Move RunJournal telemetry docs to the runtime guide.
- Add tests: llm.error without a start, replayed on_llm_end, and a DB-store
metadata round trip.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(runtime): refresh token counters on usage replay
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
#6229 gave the scheduler its own integer knobs, but they accept YAML
booleans: recursion_limit: true loads as 1, so every scheduler-launched
run dies at its first LangGraph super-step - the same hole the
app-config guard already closes for Gateway runs (#6171), now on the
same-named field. max_concurrent_runs: true serializes the scheduler
and poll_interval_seconds: true polls five times a second.
Mirror the app-config boolean guard: a before-validator rejects booleans
with a field-specific error across all six integer fields, keeping
integers and numeric strings as-is. lease_seconds and
queue_timeout_seconds already reject true via their higher ge floors,
but with a misleading "should be greater than or equal to" message -
the guard gives every field the same boolean-specific error.
Signed-off-by: yetuge <2219677952@qq.com>
YAML booleans coerce into the ge=1 integer backstops that bound every
subagent and ACP invocation: timeout_seconds: true became 1 (the executor's
future.result(timeout=...) then killed every subagent run after one second)
and max_turns: true became 1 (the LangGraph recursion budget collapsed to a
single super-step). The ACP agent timeout mirrors subagents.timeout_seconds
and shared the hole. false was already rejected by ge=1, so only true
slipped through - silently. Mirror the app-config boolean guard: reject
bools before coercion in SubagentOverrideConfig, CustomSubagentConfig and
SubagentsAppConfig (timeout_seconds/max_turns/max_total_per_run) and
ACPAgentConfig (timeout_seconds), keeping None, integers, and numeric
strings as-is.
Signed-off-by: yetuge <2219677952@qq.com>
* fix(memory): treat facts deleted during unlocked DeerMem scans as removed
load(), reload(), and the full rebuild_index() scan list agent fact
files without the storage locks. A delete committed between the listing
and opening a file raised MemoryStorageCorruption for intact data, which
surfaced as HTTP 500 from the memory API and an empty memory block in
the prompt. A listed fact that no longer exists is now skipped; an entry
that is still present but unreadable is still corruption.
* docs(changelog): reference #6255 for the DeerMem vanished-fact fix
* docs(memory): note lock coverage and heuristic limits of the vanished-fact skip
* fix(skillscan): report secrets bound by unpacking and container literals
The AST walk behind secret-env-assignment claims "precision, never less
coverage", but it only read a single assignment target and a bare string
value, so a credential moved into `host, api_key = endpoint, "..."` or
into `api_key = ["..."]` stopped being reported even though the pre-#5648
line sweep still reported the same bytes.
Pair unpacking targets with values positionally (fixed head and tail only,
so a runtime-sized slice names nothing) and assert on each literal a
container or conditional holds, in source order.
* fix(skillscan): distinguish mapping labels from credentials
---------
Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The channel runtime config store's atomic 0o600-temp + replace + chmod
cannot land a POSIX mode on Windows, where chmod is a no-op
approximation. Skip the two POSIX-bound tests and guard the trailing
mode assertion of the chmod-failure test, keeping its platform-
independent log-behavior coverage everywhere.
Fixes#6256
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
Four of the eight packages/harness/deerflow citations in this file name modules
that are not in the tree: sandbox/local.py, mcp/manager.py, skills/loader.py,
and tools/__init__ (missing its extension). The LocalSandboxProvider class lives
under sandbox/local/, the MCP and skills subsystems are packages rather than
single manager/loader modules, and get_available_tools is re-exported by
tools/__init__.py.
* refactor(config): converge the boolean guards into one shared helper
Willem-bd's #6171 review follow-up: a tiny shared helper lets the next numeric field get the guard in one line and unifies the messages. Consolidates 14 validator copies across 9 config files into deerflow/config/_boolean_guards.reject_boolean(value, info, kind=...); the bare per-file messages gain their field name prefix, command_timeout keeps its distinct wording.
Signed-off-by: yetuge <2219677952@qq.com>
* refactor(config): converge stream_bridge_config and pin the guard wording
Review follow-ups on #6226: convert the last surviving per-file copy (stream_bridge_config) so deerflow/config is fully converged, make kind required keyword-only so a float field cannot silently inherit integer wording, and pin the unified field-prefixed messages with direct tests.
Signed-off-by: yetuge <2219677952@qq.com>
* test(config): cast the boolean-guard test probe to ValidationInfo
The _info probe was annotated -> object, so a type checker flags every call
site that passes its result to reject_boolean's info: ValidationInfo
parameter (runtime was fine since the helper only reads .field_name).
Return cast(ValidationInfo, _Probe(field_name)) to keep the probe honest
for both.
Signed-off-by: yetuge <2219677952@qq.com>
---------
Signed-off-by: yetuge <2219677952@qq.com>
* fix(memory): sign DeerMem reload before reading the document
reload() read the document before computing its cache signature, so a
write committed in between cached the old document under the new
signature and later load() calls served stale memory until the next
write. Compute the signature first, matching load().
* test(memory): cover markdown storage and agent scope in reload race test
* docs(changelog): reference #6238 for the DeerMem reload fix
* fix(telegram): deny all users when allowed_users has no valid ID
TelegramChannel parsed channels.telegram.allowed_users with int() and
dropped every entry that failed without logging, and _check_user treated
an empty result as "no allowlist". A list of @usernames therefore opened
the bot to everyone. A single string ID was iterated character by
character ("123456" allowed users 1-6 and blocked 123456), which is what
an allowed_users: $ENV reference resolves to, and null or a bare integer
crashed the channel at startup.
The allowlist is now parsed into frozenset[int] | None: None (unset,
null, empty list, blank string) still allows everyone, a single ID is a
one-entry list, entries that are not positive numeric IDs (@usernames,
floats, booleans, comma-joined strings) are dropped with a warning, and
a configured list with no valid ID denies every user and logs an error.
* docs(changelog): reference #6230 for the telegram allowed_users fix
* fix(telegram): say "positive" in the allowed_users drop warning
0 and negative integers are numeric but still dropped, so "expected one
numeric Telegram user ID" misdescribed them. The warning now asks for a
positive numeric ID, matching _parse_telegram_user_id, and the drop test
covers 0 and -5 alongside @alice.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(extensions): add host-bound Agent run control
* fix(extensions): preserve internal launches and scope run control
* docs(extensions): use a plugin-local run operation key
* feat(frontend): support extension providers in composer mentions
* fix(mentions): retain settled plugin suggestions during search
* fix(extensions): validate identifiers and skip empty mention searches
* test: stabilize history pagination after latest page refresh
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): run Discord connection-repository work on the Gateway loop
discord.py delivers messages on a private loop in the client thread, and
the Discord adapter awaited the channel-connection repository there even
though its SQLAlchemy engine and pool belong to the Gateway loop. On
PostgreSQL the first Discord message after the Gateway used the pool
failed with "got Future attached to a different loop" and was dropped; on
SQLite a pool-exhausting burst failed with "Queue is bound to a different
event loop" and left the pool's wait queue bound to the Discord loop, so
the Gateway's own queries then failed the same way.
Resolve the identity together with the intake commit, and run /connect
binding, on the Gateway loop through _submit_threadsafe_coroutine, as
Telegram, Feishu and DingTalk already do. Bind replies return through
_run_on_discord_loop, and start()/stop() now open and drain the
cross-thread submissions.
* docs(changelog): reference #6214 for the Discord connection-loop fix
* fix(channels): correct the unscheduled Discord bind comment and pin it
When _submit_threadsafe_coroutine refuses the bind, base.py closes the
coroutine, so consume_oauth_state never runs and the one-time code stays
valid; only the Discord message is swallowed. Say that instead of
claiming the code is consumed, and pin both halves with a test: the
attempt is handled (never a chat turn) and the code still binds later.
* fix(channels): undo Discord typing when the Gateway hand-off fails
Since identity resolution moved to the Gateway loop, both inbound paths
started typing and the ack reaction as soon as the hand-off was
scheduled. A lookup that then raised (database outage, pool-checkout
timeout) dropped the message but left an endless typing loop and a
success reaction behind.
The hand-off now returns its completion future (new
Channel._submit_threadsafe_coroutine_future, which the bool helper wraps)
and the Discord side waits on it through asyncio.shield. Typing still
registers before the hand-off settles, so a fast reply's _stop_typing
cannot run first; the ack reaction waits for the commit. On failure, or
when stop() drains the hand-off, the indicator this message started is
stopped (never one another in-flight message owns) and no ack is added.
* fix(channels): keep shared Discord typing alive for dependent messages
Two messages to the same target share one typing task: the first starts
it, a later one reuses it. If the creator's hand-off failed after a
follower had committed, or while the follower was still pending, the
creator stopped the indicator the follower relied on, and nothing
restored it while the follower's request ran.
Count the hand-offs that depend on each typing task. A pending or
committed hand-off holds the task (a commit keeps its hold until a reply
stops the indicator), a failed one releases its hold, and the indicator
is stopped only when the failed message started it and no hold remains.
The count is dropped when the task ends or the channel cancels typing.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): ignore literal base tags in HTML artifact previews
* fix(frontend): address HTML preview base review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs(readme): sync TOC with headings and fix deploy.sh paths
* docs(readme): move skills content stranded in the RAGFlow chapter back into Skills & Tools
* fix(lark): match broker subcommand denylist through separate option values
The opt-in DEERFLOW_LARK_BROKER_DENY_SUBCOMMANDS check dropped only
`-`-prefixed tokens and compared the rest from the start, so the value in
`--profile work config show` became the leading positional and a
`config show` rule never matched. Real lark-cli 1.0.65 runs `config show`
for that shape and for `config --profile work show`.
The broker cannot know which options take a value, so a rule now matches
when its tokens appear in order among the non-flag tokens. A contiguous
match would still miss a value placed between the path tokens. Argument
values that spell a denied path in order are refused too (fail-closed).
* docs(changelog): reference #6212 for the lark broker denylist fix
* docs(lark): call denylist entries command paths, not prefixes
Entries are matched as in-order subsequences now, so the remaining
'prefix' wording in the BrokerConfig comment, the parse_deny_subcommands
and run_lark_cli docstrings, and the broker README undersold the rule.
Also rewrap the comment and docstring lines this PR had pushed past ~80
columns.
* fix(sandbox): scope ignore patterns to the search root in glob/grep
Rebuilt on top of current main so the changelog entry lands in the
Unreleased (2.2.0) Fixed section rather than the now-released 2.1.0 block.
* test(sandbox): cover the include_dirs glob branch at an ignored search root
The file.list_path branch only ran against a non-ignored tree, so a
regression there would go unnoticed. Two cases now exercise it with an
ignored root: the requested root keeps its contents, and ignored
descendants stay hidden.
* fix(config): reject boolean integer settings in app_config
Follows the loop-detection and circuit-breaker guards: YAML booleans next
to integer knobs load cleanly, and Pydantic coerces booleans to 1/0.
- recursion_limit coerced from `true` to 1, so every Gateway run that does
not supply its own limit hit the LangGraph recursion ceiling at the
first super-step (max_recursion_limit likewise clamped the ceiling to 1).
- The five llm_call integers coerced similarly: retry_max_attempts as
`true` disabled retries, max_concurrent_calls as `true` serialized the
pipeline.
All seven integer fields now fail configuration loading with a
field-specific error, while valid integers and numeric strings keep their
existing behavior. Mirrors the loop-detection helper (#6017) and the
in-file circuit-breaker validator.
Signed-off-by: yetuge <2219677952@qq.com>
* docs(changelog): record the app-config boolean rejection
Changelog entries for #6171, mirroring the #6017 entry format in both
languages.
Signed-off-by: yetuge <2219677952@qq.com>
* test(config): pin the False-side coercion regression in the boolean guards
Per review: the tests only exercised `True`. On the `ge=1` fields `false`
already failed the constraint pre-fix, but on the `ge=0` fields
(`max_concurrent_calls`, the two delay fields) `false` coerced silently to
the harmless-looking `0`, disabling the concurrency cap with no error.
Parametrize both boolean tests over `[True, False]` to lock that regression
in as well.
Signed-off-by: yetuge <2219677952@qq.com>
---------
Signed-off-by: yetuge <2219677952@qq.com>
* fix(mcp): drain the config read-modify-write across cancellation
Route the five MCP config endpoints through the canonical
run_drained_write helper and fold the tools-cache reset into the drained
unit, so a cancelled caller cannot detach from the worker holding the
extensions-config locks or skip the cache invalidation behind a settled
write. Caller-facing HTTPException contracts re-raise unlogged.
Fixes#6209
* fix(mcp): type the drain helper against the worker signature and unlist drain adopters
- _run_drained_mcp_apply now mirrors the canonical helper's [**P, T]
Callable[P, T] typing so each call site's argument list is checked;
- AGENTS.md stops enumerating the adopted routers, so the line stops
needing an edit on every new adopter.
---------
Co-authored-by: xiaodu55 <xiaodu55@users.noreply.github.com>
* fix(community): refuse non-global addresses in the shared SSRF guard
is_blocked_address ORed is_private, is_loopback, is_link_local,
is_reserved, is_multicast and is_unspecified. None of them covers the
100.64.0.0/10 shared address space, which holds CGNAT and Tailscale
hosts and Alibaba Cloud's 100.100.100.200 instance metadata endpoint, so
web_fetch, web_capture, the agentic browser and personal MCP connections
could reach it, including through the IPv4-mapped ::ffff:100.100.100.200
form a DNS answer can carry.
Also refuse any address that is not is_global, the classification the
repo's other SSRF filters already use. Keep the existing flags alongside
it: some non-public forms still report is_global, such as the NAT64
spelling of a metadata address (64:ff9b::a9fe:a9fe) and 224.0.0.1, so an
is_global-only check would newly allow them.
* docs(changelog): reference #6202 in the shared address space entry
* test(community): pin the IPv4-mapped metadata spelling at the web_fetch and personal MCP entry points
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(browser): pin browser connections to the SSRF-vetted addresses
The navigate screen and the per-request guard resolve a hostname to vet
it, but Chromium resolved the name again to open the connection. A
rebinding DNS server could answer both checks with a public address and
the connection with a private or cloud-metadata one, and the browser
followed it.
Launch each private browser through a per-session loopback SOCKS5 proxy.
Over SOCKS5 Chromium sends the hostname itself, so the proxy resolves it
exactly once through the session's egress resolver and connects only to
the addresses that resolver vetted. resolve_browser_egress applies the
same allow_private_addresses policy as validate_browser_url, on top of a
new url_safety.resolve_public_addresses helper that
validate_public_http_url now shares. Loopback is proxied explicitly with
bypass "<-loopback>". The tools, navigate_and_capture and the Live
WebSocket all pass the resolver, because a thread's first caller fixes
its launch options.
CDP-attached Chrome is unchanged. Delegated fetch services (crawl4ai,
Browserless, fastcrw) still resolve on their own side, which the Gateway
cannot pin.
* docs(changelog): reference #6201 in the browser egress pinning entry
* fix(browser): bound egress proxy faults and close, and record the WebRTC gap
Review follow-ups for the pinned browser egress proxy:
- A resolver exception other than ValueError escaped the connection
handler as an unretrieved task error. Log it with its traceback and
reply SOCKS5 general failure (0x01), so a resolver fault stays
distinguishable from a policy refusal (0x02).
- Bound the resolve with its own timeout. The handshake timeout did not
cover it, so a wedged getaddrinfo held the half-open tunnel until
Chromium gave up. A timeout replies host unreachable.
- Bound close(): Server.wait_closed() also waits for a connection whose
handler registered after the writer sweep, which could stall session
close and eviction for the 15 s handshake. A handler that starts after
close now drops its connection without opening a tunnel.
- Docs said a launched browser connects "solely" through the proxy.
WebRTC UDP does not traverse a SOCKS proxy, so AGENTS.md, README (en/zh)
and both changelogs now scope the guarantee to TCP and name the gap.
The English and Chinese READMEs document the Projects feature
(shared instructions, document shelf, archive read semantics, trash),
but the Japanese, French, and Russian READMEs skip it: their heading
structure jumps straight from the embedded Python client to Scheduled
Tasks, and their tables of contents have no Projects entry.
Add the translated section at the same position between the embedded
Python client and Scheduled Tasks, and add the matching table of
contents entry, so all five language variants cover the same 21 H1/H2
sections.
Follow-up in spirit to #4803 and #6129, which synced the Sister
Projects and Startup Modes sections across the localized READMEs.
* fix(tui): preserve multiline pasted input
The TUI composer used a single-line `Input`, so a bracketed paste
containing code, a stack trace, or multiple paragraphs kept only the
first line when submitted. The composer is now a multiline `TextArea`:
Enter still submits, Up/Down move inside the document before falling
back to input history at the first/last line, and the slash palette is
unchanged.
Originally #4630 by @felix-windsor, cherry-picked onto current main;
conflicts in app.py key gating (transcript paging keys), test imports,
and the relocated TUI docs were resolved by hand.
Fixes#4613
* fix(tui): submit multiline input unchanged and document the widgets layer
Address the review on #4630:
- Send non-command input to the agent and history exactly as typed or
pasted. Only the empty-input check uses strip(), so leading
indentation and trailing newlines in pasted code and stack traces
survive. Slash commands are still trimmed so they resolve as before.
- widgets/composer.py imports Textual, so the module-layout notes in
tui/AGENTS.md and docs/TUI.md now exempt widgets/ alongside app.py.
- Cover pressing Down on the last document line (history fallback),
whitespace-only input, and slash commands with surrounding whitespace.
Co-authored-by: Felix <felixwindsor3344@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(tui): treat soft-wrapped rows as composer navigation boundaries
TextArea soft-wraps long lines, and its Up/Down move between the wrapped
rows of one logical line. The history fallback only looked at logical
rows, so Up from the second visual row of a long prompt recalled history
instead of moving the cursor, and Down from its first visual row could
replace the draft with history's empty value.
Use the TextArea navigator's first/last wrapped-row checks for the
boundaries, and cover wrapped prompts in both directions. Also restore a
tui/AGENTS.md sentence about PageUp/PageDown to the app.py entry it
belongs to.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Felix <felixwindsor3344@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(sandbox): pin utf-8 decoding on the local sandbox backend's subprocess calls
The local container backend shells out to docker / the container runtime
with text-mode subprocess calls that do not pass an encoding, so output is
decoded with the platform's preferred encoding. On hosts whose ANSI code
page is not UTF-8 (cp936 Chinese Windows, cp1252 Western European), any
non-ASCII byte in the CLI output raises inside subprocess's reader thread:
stdout/stderr come back None, the JSON-parsing call sites then crash on
None or silently mis-parse, and stderr diagnostics are dropped.
Pin encoding="utf-8" with errors="replace" on every text-mode call in
the module. The container CLIs write UTF-8 on every platform; replace
keeps the parseable portion of any non-conforming output alive instead of
discarding it wholesale.
* test(sandbox): validate UTF-8 subprocess settings per call
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
