* feat(sandbox): add boxd sandbox provider
Adds deerflow.community.boxd_sandbox:BoxdSandboxProvider. Each user/thread
scope runs in an isolated boxd machine. New machines get the thread's files
and skills before they are handed out, release mirrors outputs back to
thread storage, and released machines wait in the shared warm pool.
The boxd SDK is an optional extra (deerflow-harness[boxd]). Unit tests run
against a fake client; tests/test_boxd_sandbox_live.py runs against real
machines when DEER_FLOW_RUN_LIVE_TESTS=1 and BOXD_API_KEY are set.
* fix(boxd): retain failed cleanup and bound output mirror calls
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(search): add DeepSeek web search with configurable thinking
* fix(search): drain DeepSeek teardown and retain citation spans
Reuse the cancellation-safe context helper so repeated cancellation cannot interrupt HTTP transport cleanup. Preserve validated Unicode citation ranges on retained public sources so the agent can associate claims with their supporting pages. Add regressions for cancellation, timeout, normal completion, invalid ranges, URL screening, and result limits.
* fix(search): preserve DeepSeek citation excerpts through guardrails
* fix(search): share DeepSeek citation excerpts across sources
Group each validated answer range once and associate it with response-local source IDs after URL filtering and result limits. Preserve full-answer PII redaction and existing source-card fields, document the model-facing citation mapping, and repair live-test configuration for both thinking modes. Add HTTP transport, guardrail/budget, attribution, and browser reload regressions.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(opensandbox): configurable sandbox resource limits and requests
The OpenSandbox provider passes no resources to SandboxSync.create(), so
every sandbox gets the SDK default of 1 CPU / 2Gi and the server's default
requests. Workloads that need more (builds, PDF rendering, data work) can't
get it, and clusters with tight quotas can't ask for less.
Add two optional sandbox options, `resource` (limits) and
`resource_requests`, each a map of resource names to quantities, forwarded
to SandboxSync.create(). Numbers become strings; other value types fail at
provider startup. Unset, null or empty keeps today's defaults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(opensandbox): reject blank or non-positive resource quantities; document requests
Review follow-up for #8305:
- Reject blank or whitespace resource names and quantities, and numeric
quantities that are zero, negative or non-finite, at provider startup
instead of failing at acquire().
- Document that only Kubernetes runtimes apply resource_requests, that an
omitted request defaults to its limit, and that requests must fit within
the limits, including the SDK default 1 CPU / 2Gi when resource is unset.
- Bump config_version to 67 (example, chart values, chart README) and add a
CHANGELOG entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore: drop the CHANGELOG entries (generated in batch by maintainers)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web-fetch): stop sending .env.example placeholder keys to providers
make config and make setup copy .env.example into .env, and the template
set nine search/fetch provider keys to example values such as
JINA_API_KEY=your-jina-api-key. The default keyless Jina Reader sent
that text as a Bearer key, so every web_fetch on a fresh install failed
with 401 while make doctor reported web fetch as configured.
- .env.example comments those nine lines out, like every other credential
in the template already is.
- provider_credentials.resolve_api_key reads a your-...-key/-token/-secret
value as unset, so existing .env files that still hold the placeholders
recover without editing, for every provider that resolves keys there.
* chore: leave the changelog entry to merge time
Per CONTRIBUTING.md (#8202), maintainers add changelog entries when they
merge a pull request.
* fix(web-search): route every provider key and doctor through one placeholder rule
Review follow-up for #8206:
- provider_credentials exposes is_template_placeholder(), the single
definition of an .env.example example value. key/token/secret must be
a whole hyphen-separated word, so a real value such as your-monkey or
your-passkey is no longer dropped.
- Serper, Serply, Sofya, SpecStream, Unbrowse, Webz, Brave, Tencent WSA
and You.com resolve config and environment keys through
resolve_api_key instead of private strip-and-send copies, so an old
.env that still holds the template value no longer sends it.
- make doctor asks the same predicate: a placeholder key is reported as
still holding the .env.example value instead of "set", and a literal
placeholder in config.yaml falls back to the environment key as the
runtime does.
- A census test keeps credential environment reads under community/
wrapped in resolve_api_key.
* fix(web-search): match shipped placeholders exactly and mirror Browserless precedence in doctor
Second review follow-up for #8206:
- is_template_placeholder() matches exactly the your-... values that
.env.example ships instead of a pattern, so a token an operator chose
(a self-hosted Browserless TOKEN such as your-review-team-token) is
sent again. A test keeps the set equal to the template.
- make doctor mirrors Browserless: a configured token field is
authoritative, blank or not, and BROWSERLESS_TOKEN is read only when
the field is omitted. A blank or placeholder token now warns instead
of reporting the variable, cross-checked against the client.
- The keyless Jina and You.com providers stay ok but name JINA_API_KEY
or YDC_API_KEY when a placeholder there is ignored, and a literal
placeholder api_key in config.yaml is named when no env key backs it.
* fix(doctor): match each web provider's key precedence; drop the dead Jina api_key from docs
Found while re-checking #8206 against every provider's runtime:
- Browserless used as web_fetch had no doctor key check; it now gets the
same authoritative token handling as web_capture.
- InfoQuest reads only INFOQUEST_API_KEY, so doctor no longer counts an
api_key field in its tool config as configured.
- GroundRoute needs a key for every call but had no doctor entry, so a
missing key reported ok.
- You.com names a placeholder api_key in config that it ignores.
- The Jina web_fetch docs snippet carried api_key: $JINA_API_KEY, which
the tool never reads. With JINA_API_KEY now commented out in
.env.example, that line made config loading fail on a fresh install.
- The provider_credentials docstring now scopes doctor's use of the
predicate to web provider keys, and the scripts guide no longer says
.env.example ships live placeholder keys.
* fix(community): return an error for malformed URLs in the SSRF guard
validate_public_http_url and validate_delegated_backend_url document
returning an "Error: ..." string, but urlparse() raises ValueError for
malformed authorities such as http://[::1/api, so the exception escaped
to callers (web_fetch/web_capture, browser_navigate, personal MCP, and
the browser stream navigation check) instead of a clean validation error.
Parse through a small _parse_url helper that returns None for malformed
input (including non-string values), and treat a raw bracket in the
userinfo slot as malformed on every Python version, mirroring the
SkillScan host parser merged in #8017.
The only URLs newly rejected are userinfo values containing a raw
bracket, which RFC 3986 forbids; no valid host becomes unreachable and
fail-closed behavior is otherwise unchanged.
* test(community): share malformed URL cases and add changelog entry
Drive both validators' malformed-URL parametrization from one list, and
document the contract fix in the Unreleased Fixed section (#8079).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(opensandbox): add opt-in secure_access sandbox option
Forward a new `sandbox.secure_access` option (default false) to
SandboxSync.create so OpenSandbox can require a per-sandbox endpoint
token at the ingress gateway and server proxy. The server returns the
token as an endpoint header and SDK 0.1.15 already attaches endpoint
headers to execd and file requests, so DeerFlow needs no token handling.
Non-boolean values are rejected at provider startup so a quoted "false"
cannot silently enable it. Documented in config.example.yaml,
CONFIGURATION.md, the SandboxConfig docstring and the provider README.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(opensandbox): validate server proxy boolean
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Totoro <msy626836554@gmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(browserless): reject timeouts httpx cannot use
`_coerce_timeout` forwarded any int, float or numeric string to
`httpx.AsyncClient(timeout=...)` unchanged, so a configured `timeout_s: 0`
or a negative value failed every fetch and capture with `ConnectTimeout`
against a healthy backend, `nan` raised `ValueError: deadline must not be
NaN`, and `inf` left the request without a deadline so it never completed.
Each of those silently disabled the provider.
Fall back to the documented 30s default with a warning when the number is
not a usable deadline. This matches `crawl4ai._coerce_timeout` after #7481;
`jina_ai._coerce_timeout` still forwards non-positive values.
* fix(browserless): address timeout review and format tests
Explain how HTTPX handles NaN and positive infinity. Document that
Browserless accepts positive finite timeouts without an upper limit,
and pin that behavior with a 1e308 test case.
Crawl4AI and Jina timeout validation are tracked in #7481 and #7823.
Remove the Browserless docstring's claim that Crawl4AI shares its
coercion behavior. Format the test file for the backend lint check.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs: turn README into an index and move reference detail to docs/guide
The README had grown to 393 KB because every change appended its notes there. The text moves verbatim into 18 topic guides under docs/guide/; only headings are re-leveled and relative links re-pointed. The README keeps the introduction, a quick start and a guide index. Contributor test notes move to CONTRIBUTING.md.
* docs: send user docs to topic guides and budget the root README
The rule 'update README.md for user-facing changes' is what grew the README. Guidance now points to docs/guide/, and check_agent_guidance.py fails when the root README exceeds 16 KiB (AG003).
* chore: remove unreferenced screenshots and finished plan notes
pr-build/ holds two PR screenshots (3 MB) from #1986. plans/ holds the plan for the merged #4049. docs/SKILL_NAME_CONFLICT_FIX.md is a February change note. Nothing references them.
* docs: route always-loaded AGENTS.md files to scoped guides
* docs(agents): cut scoped guides to scope, verify, traps, decisions
Each AGENTS.md keeps four kinds of content: the scope of the directory, the
verification commands, traps a local test run does not show, and decisions the
code cannot state. Values, field lists, function inventories, test names, issue
numbers and history are deleted, because the code, the tests and git state them.
Search and fetch providers share one community guide. Four guides with no
remaining content are removed. The size check now allows 6 KiB per file (hard
limit 8 KiB) and 24 KiB per chain (hard limit 32 KiB).
* style: format frontend guide table and guidance test imports
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(browserless): resolve the token through the shared credential rule
Browserless read BROWSERLESS_TOKEN and the configured token raw, so a value
from a file kept its trailing newline and became the auth query parameter;
every request then failed authentication. A whitespace-only value was sent as
a token instead of counting as unset. Use provider_credentials.resolve_api_key
like the other search and fetch providers, with the blank-config fallback to
the environment.
* fix(browserless): keep an explicit blank token opt-out
An explicitly configured token ("" or null) is authoritative: a self-hosted
capture endpoint uses it to opt out of authentication, and falling back to
BROWSERLESS_TOKEN sent the cloud credential to that endpoint. Only an omitted
token key falls back to the environment. Regression tests cover the client
token and the outgoing capture query.
* docs(changelog): record the browserless token normalization
Match the sibling #7981 entry in both locales; the fix closes out the
credential-rule family for the bundled search and fetch providers.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(channels): keep Gateway calls off the environment proxy
* fix(channels): keep the aio_sandbox policy import lazy
Importing deerflow.community.aio_sandbox.backend at module scope executes
the package __init__, which eagerly imports AioSandbox, AioSandboxProvider
and both backends. That makes every ChannelManager import load the entire
sandbox provider stack even when a different sandbox is configured.
Mirror app/gateway/health.py::_make_provisioner_client: move the import into
a small _make_gateway_client helper and route both Gateway call sites through
it, so the URL policy stays lazy.
* fix(channels): keep the run client off the environment proxy
langgraph_sdk.get_client() builds httpx.AsyncHTTPTransport(retries=5) on an
httpx.AsyncClient that leaves trust_env at its default True and exposes no
transport/trust_env override, so ChannelManager._get_client() still sent
runs.wait submissions to a loopback Gateway through HTTP_PROXY. That is the hop
#7876 reports: the control-plane calls are fixed, the run submission is not.
Route the SDK client through _make_langgraph_client: loopback and private
Gateways get an explicit trust_env=False httpx.AsyncClient carrying the SDK's
own retries, timeout and User-Agent and handed to LangGraphClient, while
external Gateways keep the stock get_client() path.
Also move the URL policy out of the aio_sandbox package into
deerflow.utils.http_policy (control_plane_trust_env). Keeping the import lazy
did not help on its own: aio_sandbox/__init__ eagerly imports the provider
stack, so importing aio_sandbox.backend still loaded all of it. The neutral
module has no httpx or sandbox dependency, so manager.py and gateway/health.py
import it at module scope, and aio_sandbox.backend re-exports it as
sandbox_http_trust_env for the existing sandbox call sites.
* fix(channels): keep AGENTS.md under budget and align legacy CSRF test with new client
* style(sandbox): remove unused urlparse import
---------
Co-authored-by: jack12178 <jack12178@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* refactor(agent): ground runtime decisions in structured evidence
Use producer status and structured errors for tool recovery, preserve memory context without lexical confirmation, and remove predetermined receipt and skill conclusions.
Require explicit MCP discovery and authenticated schedule trial actions. Preserve authorization, ownership, isolation, and bounded execution; update documentation and regression coverage.
* test(sandbox): expect read_file initialization error callback
* fix(agent): address evidence review follow-ups
* fix(agent): preserve retry budgets for OpenAI transport errors
* fix(agent): expose tool execution status in model requests
Project validated execution metadata and exact tool text into request-only JSON so status survives provider serialization. Preserve stored results and receipt hashes, cover SDK and graph boundaries, and migrate replay input hashes without changing recorded responses.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(community): strip provider API keys before they reach a header
GroundRoute (from GROUNDROUTE_API_KEY), InfoQuest, Tavily, Exa, Jina,
Firecrawl, fastCRW and Crawl4AI put the configured or environment key into
an HTTP header as read. A key read from a file keeps its trailing newline
(a Kubernetes Secret created with --from-file and mounted through envFrom),
and the transport refuses it with an error that quotes the whole header:
h11's "Illegal header value b'Bearer <key>\n'" or requests' "... in header
value: '<key>\n'". The providers returned that message as the tool result,
so the key reached the model and the checkpointed conversation, and
GroundRoute, InfoQuest, Jina and Crawl4AI also wrote it to the error log.
Resolve each key through provider_credentials.resolve_api_key, as the other
bundled providers already strip theirs. Tavily, Exa and Firecrawl now read
their environment fallback themselves instead of leaving it to the SDK,
which would read the raw value again. A blank configured key now falls back
to the environment, as it already does for GroundRoute and the stripping
providers.
* docs(changelog): reference #7981 for the provider API key fix
* fix(community): keep fastCRW from sending the Firecrawl key
fastCRW builds the Firecrawl SDK client, and the SDK reads
FIRECRAWL_API_KEY whenever api_key is None. Once a blank configured key
resolved to None, a fastCRW config with api_key: "" and no CRW_API_KEY sent
the Firecrawl key to the fastCRW endpoint. With no api_key configured at
all, main already did the same.
Pass "" when fastCRW resolves no key of its own; the SDK falls back only on
None and sends no Authorization header for an empty key. A loopback test
pins the empty, blank and absent cases with FIRECRAWL_API_KEY set.
* fix(community): keep a blank environment key away from the SDKs
Tavily, Exa and Firecrawl resolved their environment fallback themselves,
but passed None when it was blank, and each SDK then re-read the same
variable unstripped. Tavily and Firecrawl now pass "", which the SDKs
treat as a missing key; Exa sends "" as is, so it fails before any
request instead.
The fastCRW comment records that only the Firecrawl SDK's v2 client
honours "", and the fastCRW regression test now covers web_fetch as well
as web_search, so a move to the v1 client would fail it.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): build the readiness client off the event loop
`wait_for_sandbox_ready_async` exists so sandbox startup waits do not block the
runtime event loop, yet it constructed `httpx.AsyncClient` inline: the
constructor resolves the system trust store synchronously, so the loop stalled
for the whole construction before the first poll was sent (~320 ms in a local
probe on this host).
Move the construction into a worker thread and own it as a task, the way
backend/AGENTS.md now requires for HTTPX clients: a cancellation that lands
while the worker is still inside the constructor waits for it and closes the
client it builds, so the pool is not abandoned. Requests, polling cadence,
timeouts and cleanup stay on the event loop.
The blocking-I/O regression pins both halves. Against the previous code the
trust-store assertion fires on the loop thread; with a plain `asyncio.to_thread`
instead of the owned task, the cancellation test reports the leaked client.
* fix(sandbox): preserve readiness cancellation during cleanup
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(community): initialize InfoQuest TLS clients off the event loop
InfoQuest's fetch, web search and image search constructed
httpx.AsyncClient inline in their coroutines, so the default transport
loaded the TLS trust store synchronously on the agent event loop
(~125 ms per call in a local probe). RAGFlow, LightRAG, You.com,
GroundRoute and Webz already construct their clients with
asyncio.to_thread; InfoQuest was the missed sibling.
Move the three constructions to asyncio.to_thread and cover all three
methods in the blocking-I/O TLS test. Requests, cleanup, timeouts and
responses stay on the event loop.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* docs: qualify the off-loop HTTPX provider list
The list is not exhaustive; several community clients still construct
their client inline. Use non-exhaustive wording until the follow-up
sweep lands.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* docs: correct the off-loop provider list after merging main
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(browserless): construct the fetch and screenshot clients off the event loop
* fix(browserless): close the client a cancelled construction still builds
Keeping the constructor off the loop is right, but awaiting
`asyncio.to_thread(httpx.AsyncClient, ...)` directly is not cancellation-safe:
the await can be cancelled while the worker thread is still inside the
constructor. The await is abandoned, the thread runs on and hands back a live
client, and execution never enters the `async with` that would have closed it,
so `is_closed` stays False and the connection pool leaks on that cancellation.
Own the construction as a task instead. When the caller is cancelled, wait for
the task with the shield loop this package already uses for cleanup, close
whatever client it produced, and only then let the cancellation propagate. A
failure while closing is logged rather than raised, so it cannot mask the
cancellation the caller asked for.
The new regression test gates the constructor on a thread event, cancels the
operation while it is parked there, and releases the worker afterwards. Against
the previous commit it fails for both fetch_html_with_status and
capture_screenshot with "the client built after the cancellation was leaked".
---------
Co-authored-by: inchang-ing <197932532+inchang-ing@users.noreply.github.com>
* fix(sandbox): defer fresh AIO skill-policy acquisition
Prepare the authorized thread skill view before the model runs without
acquiring an AIO container for a fresh lazy run. Keep existing sandbox
bindings and upload-based providers on the eager rebind/sync path.
Add sync/async regressions for first-tool authorization, strict rebinding,
filtered read-only AIO mounts, and policy-scoped warm reuse.
AI assistance: Codex; the account owner confirmed human review of the
complete seven-file patch and responsibility for the change.
* docs(sandbox): keep lazy-isolation guidance within inherited budget
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(crawl4ai): construct the fetch client off the event loop
Crawl4AiClient builds an httpx.AsyncClient per fetch, and AsyncClient
construction loads the TLS trust store synchronously: on the agent event
loop that blocks every other request for the CA bundle read. Construct it
through asyncio.to_thread, the same rule the RAGFlow/LightRAG and
You.com/GroundRoute/Webz clients already follow; the request and cleanup
stay on the loop.
* docs(agents): list Crawl4AI among the off-loop HTTPX clients
#7889 extended the same sentence for its providers; keep the named list in
the same change set as the client, and record the clients that still
construct on the loop so the next migration has a checklist.
* fix(community): resolve async tool config off the event loop
Agent runs execute on the Gateway event loop, and the async community
tools resolved their tool config through get_app_config() inline. That
call stats and hashes config.yaml and extensions_config.json on every
call and reparses them after an edit, so each tool call held the loop
for it, up to three times per call: Tavily, SearXNG and Firecrawl read
the config again to build their client, InfoQuest read it three times,
and the browser tools read the launch config for every action.
Resolve it through asyncio.to_thread once per call, as image_search and
groundroute already do, and build the client from that same read. This
covers jina_ai, crawl4ai, browserless, tavily, searxng, firecrawl,
infoquest, ragflow, lightrag and browser_automation.
* docs(changelog): reference #7580 for the off-loop community tool config reads
* docs(changelog): use the existing community-tools label in the zh entry for #7580
* fix(community): resolve the jsonify dataset_rows config off the event loop
dataset_rows_tool is async and called _load_settings() inline, so every
call ran get_app_config()'s stat + sha256 revalidation on the loop.
Read it through asyncio.to_thread like the other async community tools,
and add the tool to the off-loop anchors.
* docs(changelog): include Jsonify in #7580 and drop trailing whitespace
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): leave SIGTERM/SIGINT teardown to the Gateway lifespan
AIO and E2B providers built on the event-loop thread installed SIGTERM/SIGINT
handlers that force-destroyed every sandbox synchronously, before uvicorn
started the run drain. The Gateway lifespan now holds sandbox shutdown
ownership from its first step, so providers skip those handlers and the
existing hook closes the provider after run drain and MCP teardown. SIGHUP
keeps the provider fallback. AIO acquire now refuses once shutdown began.
* docs(sandbox): document Gateway-owned sandbox shutdown and AIO admission fence
* test(gateway): inspect the lifespan body now that lifespan wraps it
* docs(changelog): reference #7560 for the sandbox SIGTERM drain-order fix
* ci: retrigger sandbox image smoke after a registry pull timeout
* fix(sandbox): derive the AIO sandbox ID off the event loop in async acquire
_acquire_internal_async offloads every acquire layer except the sandbox ID
derivation, which ran inline. _sandbox_id_for_thread checks the thread's
skills_view directory with exists(), and resolving it goes through
Paths.base_dir, so each async thread acquire made filesystem syscalls on the
event loop. The strict blocking-I/O gate raises on it.
Run the derivation through the same owned-worker helper as the reuse and
warm-pool layers around it, and anchor it in the blocking-I/O suite for both
the default and the policy-scoped ID.
* docs(changelog): reference #7507 for the AIO async acquire sandbox ID fix
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): return from quarantine teardown when a peer already did the work
retire() and retire_replaced_generations() guard their final rmdir() but list
the fence directory and unlink each record unguarded, while contains() and
_exists() in the same class already report a directory removed underneath them
as "no fence". The teardown exclusion is a TTL lease, not a lock, so a peer can
finish the same teardown in those two windows and the resulting FileNotFoundError
escapes the unwrapped _retire_absent_quarantine() call at the top of creation.
Both passes now tolerate a peer that removed the directory or the record first.
* test(sandbox): correct quarantine docstring reference
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(community): migrate You.com tools to async HTTP
The You.com provider sent its search request through a sync httpx.Client. A
sync LangChain tool invoked through ainvoke runs on the default executor, so
the request held one of that pool's threads for the whole network round-trip,
and several providers queried in parallel queued behind each other.
Make the tool a coroutine over httpx.AsyncClient, resolve the API key and
the configured max_results in a single config read, and move both disk reads
off the event loop: the tool config, and the User-Agent, which reads the
installed package metadata the first time it is needed.
* refactor(community): drop the dead You.com API key wrapper
The async migration resolved the key and max_results through
_resolve_tool_settings, which left _get_api_key with no production
caller: only TestGetApiKey still exercised it. Keeping a second seam
around the same config read lets it drift from the real call path, so
drop the wrapper and point the tests at _resolve_tool_settings.
Also keep the hot path off the executor: _user_agent is cached, so only
the first call reads the installed package metadata. Later calls are
in-memory hits and now skip the to_thread round-trip.
Finally, record which httpx client factory the provider reaches for in
the runtime anchor. MockTransport runs its handler on the calling thread
without real blocking I/O, so a provider that stayed async but sent the
request through a sync client would satisfy the thread assertion; the
recorded factory is what catches that partial regression.
* test(community): align You.com URL safety stub with async tool
The URL safety matrix still patched the removed _get_api_key seam and invoked the You.com tool synchronously. Mock the awaited search with AsyncMock and call the tool through ainvoke so URL screening remains covered after the async migration.
---------
Co-authored-by: treasuresyq <309743290+treasuresyq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Exa web_search and web_fetch accept a per-tool base_url, falling back to
EXA_BASE_URL and then the SDK default. Overrides are validated like the
Serper base URL. web_fetch screens a configured backend with the shared
delegated-backend guard.
Co-authored-by: lachy3433 <lachlanallen.co@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(sandbox): release the E2B ownership lease when acquire finds the VM gone
The in-process reuse path (dead or unreachable cached client) and the
warm-pool reclaim path (reconnect reports "sandbox not found") dropped
their local bookkeeping but kept the VM's ownership lease. Nothing later
sees that ID: the warm sweep only walks the warm pool, reconciliation
only walks the remote inventory, and the lease renewer keeps renewing
(and re-claims a lapsed lease) until shutdown. Every reaped VM left one
entry in _owned_sandbox_ids and one renewed ownership key behind.
Call _release_ownership in both branches. Both act only on E2B's trusted
not-found signal, the store's release is a no-op for a peer's lease, and
a backend error still drops the local ID so the shared lease expires on
its TTL.
* docs(changelog): reference #7277 for the E2B dead-VM lease fix
* test(sandbox): pin that transient E2B errors keep the ownership lease
The dead-VM release is only safe because the gone signatures match
narrowly. Pin the other side: a non-gone ping error keeps reusing the
cached VM, and a non-gone warm reconnect error leaves the lease to
eviction.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(community): add Jsonify dataset rows tool
Adds deerflow.community.jsonify.tools:dataset_rows_tool, a read-only tool
that returns rows from maintained Jsonify datasets. Jsonify pipelines run on
a schedule and repair themselves when a source changes, so the agent reads a
current table instead of scraping.
- operator configures a named map of dataset -> serve URL; the model picks
by name and never sees the URL
- serve URLs must be https without userinfo and pass validate_public_http_url
before any request; the URL is redacted from error text
- paging with has_more/next_offset, version pinned after the first page,
limit clamped to the configured max_rows, system columns stripped
- config.example.yaml entry under tools; unit tests cover paging, clamping,
unknown dataset, unconfigured tool, http/userinfo rejection, private
address block and error redaction
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrmeK98ZKoxZNiUiz8kVeX
* fix(community): harden Jsonify tool paging, logging and error handling
- Stop paging on an empty page or a next_offset that does not advance
- Log config validation failures by field location and error type only
- Validate page shape; malformed or non-JSON pages return a clean error
- Transport errors report the exception type, never the URL
- Add truncated flag when rows were clipped or more rows remain
- Tests for each case
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrmeK98ZKoxZNiUiz8kVeX
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(community): screen tavily and exa web_search result URLs
The Tavily and Exa web_search tools returned provider URLs to the model
unfiltered, so a loopback, link-local or metadata-service address in a
result reached the agent. Brave and Serper already pass result URLs
through safe_public_result_url; do the same here, blanking a non-public
URL while keeping the row's title and snippet.
* fix(community): screen result URLs in the remaining web_search providers
youcom, tencent_wsa, serply (url and scholar pdf_url), groundroute, sofya,
ddg_search, fastcrw, firecrawl and searxng handed provider-supplied URLs
to the model unfiltered. Route each through safe_public_result_url, as
brave, serper, tavily and exa do: a non-public URL is blanked and the
row's title and snippet are kept.
The test now drives every listed provider's web_search tool with a
stubbed provider payload carrying a metadata, loopback or public URL.
* fix(community): screen brave, serper and webz web_search result URLs
The remaining text-search providers still returned result URLs verbatim.
Serper's comment documented that policy; it now screens organic links like
its image results. The shared provider test covers all fourteen providers
and asserts the snippet survives screening.
* docs(tests): describe text-search URL screening coverage
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(community): migrate GroundRoute tools to async HTTP
The GroundRoute provider sent its search and fetch requests through a sync
httpx.Client. A sync LangChain tool invoked through ainvoke runs on the
default executor, so the request held one of that pool's threads for the
whole network round-trip, and several providers queried in parallel queued
behind each other.
Make both tools coroutines over httpx.AsyncClient, resolve the API key and
the configured max_results in a single config read, and keep that read off
the event loop with asyncio.to_thread.
* refactor(community): drop the dead GroundRoute API key wrapper
The async migration resolved the key and max_results through
_resolve_tool_settings, which left _get_api_key as a second seam around
the same config read: only TestGetApiKey still exercised it. web_fetch
now reads its key through the same resolver, and the tests point at
_resolve_tool_settings.
Also record which httpx client factory the provider reaches for in the
runtime anchor. MockTransport runs its handler on the calling thread
without real blocking I/O, so a provider that stayed async but sent the
request through a sync client would satisfy the thread assertion; the
recorded factory is what catches that partial regression.
---------
Co-authored-by: treasuresyq <309743290+treasuresyq@users.noreply.github.com>