* fix(view-image): read remote sandbox images from sandbox * docs(tools): clarify view_image sandbox behavior * fix(view-image): address sandbox lifecycle review * fix(view-image): preserve image provenance across sandbox replacement * fix(view-image): address provider recovery review * fix(view-image): drain cancelled tool reads
11 KiB
Tool System (packages/harness/deerflow/tools/)
get_available_tools(groups, include_mcp, model_name, subagent_enabled) assembles:
- Config-defined tools - Resolved from
config.yamlviaresolve_variable() - MCP tools - From enabled MCP servers (lazy initialized, cached with resolved-path + content-signature invalidation)
- Built-in tools:
present_files- Make output files visible to user (only/mnt/user-data/outputs); virtual paths useresolve_runtime_user_id(runtime)so validation resolves the same user-scoped outputs directory established byThreadDataMiddlewareask_clarification- Request clarification (intercepted by ClarificationMiddleware, which preserves text fallback and addsartifact.human_inputfor Web UI Human Input Cards). Beyond free text and single choice, the request-side v2 protocol supportsfields(structured form card collecting several values at once; field types: text/textarea/number/select/multi_select/checkbox/date, validated and normalized server-side in the middleware — invalid entries are dropped, unknown types degrade totext; a standalone multi-select question is a one-field form). Replies stay on the v1 response protocol (text/option): the form card submits a readable text summaryview_image- Read image bytes for vision-capable models; live sandbox bytes win for the same sandbox generation, replacement-sandbox recovery uses only SHA-256-verified synchronized host bytes, and async tool invocation drains blocking reads before cancellation may release the sandbox leasesetup_agent- Bootstrap-only: persist a brand-new custom agent'sSOUL.mdandconfig.yaml. Bound only whenis_bootstrap=True.update_agent- Custom-agent-only: persist self-updates to the current agent'sSOUL.md/config.yamlfrom inside a normal chat (partial update + atomic write). Bound whenagent_nameis set andis_bootstrap=False.
- Subagent tool (if enabled):
task- Delegate to subagent (prompt,subagent_type, optionalacceptance_criteria, and an optional model-visibledescriptionused only as a short progress label). Execution never depends ondescription; lifecycle display falls back topromptwhen a provider omits it. Subagent reports are self-reports: the docstring directs the lead to expect[rN]receipt citations and verifiable handles whileverification.receipts_enabled(and explicitly qualifies that disabled receipts mean no citations and no citation verdict), to read the delegation ledger's citation cross-check as execution evidence only, and to attachacceptance_criteriafor objectively checkable outcomes (canonical formsfile:<path> exists|non-empty,file_written:<path>,tests_passed:<command>); criteria are handed to the executor and appended to the subagent's task message as untrusted data (seesubagents/report_contract.py). Polling safety timeouts carry the latest published tool receipts into the terminal task metadata before requesting background cancellation.batch_task,batch_status,cancel_batch- Explicit durable batch submission/progress/cancellation. Added only while the startup SQL-backed batch submitter is installed; large results stay in the owner-scoped API/JSONL export rather than the lead context. Items accept optionalacceptance_criteria; item queries and exports expose the separateacceptance_verdict. Progress counts describe execution, not acceptance; unmet and UNVERIFIED conditions never trigger automatic retries.- Direct
create_deerflow_agentintegrations receive cloned tools bound to their explicitSubagentRuntime. The boundtaskforwards that runtime's exact execution controller and optional caller-ownedAppConfiginto registry/model/tool resolution andSubagentExecutor; bound batch tools use the same config snapshot and resolve only that runtime's submitter before falling back to no other application's active worker. Keep the original tool name/schema unchanged so model contracts and user-tool deduplication remain stable.
The ordinary task boundary carries one narrow parent-loop middleware recorder into the isolated subagent runtime under separate loop-detection and tool-promotion keys. It schedules only record_middleware calls back onto the loop that owns RunJournal, keeps an execution-local atomic promotion claim so parallel searches do not double-report one new schema, is fenced and drained once before task returns, and never exposes the journal or event store to the child loop. Durable batch tasks have no parent run journal and do not use this bridge.
Scheduled-task runtime note:
- Scheduled background runs set
context.non_interactive=trueand therefore excludeask_clarificationfrom the lead-agent tool list. This keeps scheduler-triggered runs from stalling on human confirmation mid-execution.non_interactiveis an internal-only context key: it is merged frombody.contextonly when the request authenticated as the process-internal user (the scheduler path), never from arbitrary HTTP/IM clients.
Durable MCP task-management tools are added only while the process-local task submitter is installed. They expose bounded local task fields, including whether cancellation was requested, but never the remote handle. Cancellation records that request durably and returns immediately; the background service owns the remote call and retries. These remain ordinary business tools under an active skill's allowed-tools policy and must be declared explicitly.
Community tools (packages/harness/deerflow/community/): optional integrations, each in its own subpackage and wired through config.yaml. Documented examples:
tavily/- Web search (5 results default) and web fetch (4KB limit)sofya/- Web search (5 results default, per-result content capped at 2000 chars) and web fetch (4KB limit)jina_ai/- Web fetch via Jina reader API with readability extractionfirecrawl/- Web scraping via Firecrawl APIimage_search/- Image search via DuckDuckGoaio_sandbox/- Docker-based isolation (AioSandboxProvider)browser_automation/- Agentic browser control (statefulnavigate → observe → click/typeloop) via Playwright, distinct from the read-onlyweb_fetch/web_capturetools. Tools:browser_navigate,browser_snapshot,browser_click,browser_type,browser_get_text,browser_back,browser_screenshot,browser_close(configgroup: browser). A process-localBrowserSessionManagerowns one private, loop-affine Playwright event-loop thread (same pattern as the BoxLite provider) so a per-thread browser session survives across turns regardless of the caller's loop (Gateway / TUI / test). Each action returns a fresh page snapshot whose interactive elements are addressed by a stable numeric[ref]index (stamped asdata-df-refduring snapshot), so the model acts on what it just observed instead of holding stale handles or guessing selectors. URLs are SSRF-screened via the sharedvalidate_public_http_url(opt-outallow_private_addressesonly for intentional internal targets). CDP attachment cannot install the request guard on an existing Chrome context, socdp_urlfails closed unless the operator explicitly setsallow_unguarded_cdp: truefor a trusted local browser. Browser REST/Live access also requires an exact non-NULL thread owner, rather than the general legacy shared-thread policy, because retained pages may contain authenticated state. Session admission is a hardmax_sessionscap: pinned Live/operation sessions are never evicted, and a new thread is rejected when no unpinned session can be closed; one Live viewer owns a session at a time. Optional dependency:cd backend && uv sync --extra browser && uv run playwright install chromium;scripts/detect_uv_extras.pypreserves the extra whenconfig.yamlenablesbrowser_navigate, and Gateway startup fails fast if configured browser control cannot import Playwright. Tests:tests/test_browser_automation.py(mocked tools + a real-Chromium integration test guarded byimportorskip);tests/manual_browser_live_check.pyis a manual DeepSeek-driven end-to-end check (not collected by pytest). Live UI input dispatch is kept independent from JPEG capture: non-move actions start a rate-limited background refresh loop, so pointer, wheel, or keyboard input stays responsive while continuous gestures still produce frames throughout the interaction.
Additional providers also live here (boxlite, brave, browserless, crawl4ai, ddg_search, e2b_sandbox, exa, fastcrw, groundroute, infoquest, searxng, serper, serply, sofya, tencent_wsa, tenki); see each subpackage for specifics. tencent_wsa uses Tencent Cloud Web Search's service API key endpoint (TENCENTCLOUD_WSA_APIKEY), not Tencent Cloud SecretId/SecretKey signing. Its max_results is capped at 50; requests above 10 use Tencent's optional Cnt parameter, which needs a Tencent Cloud plan that supports it. E2B bootstrap is required. If it fails, the provider kills and closes the unusable remote sandbox. New sandbox creation raises an error. Warm-pool reclaim and remote discovery discard the sandbox and continue acquisition. E2B mounts remain optional.
E2B output sync records remote file versions and actual host file metadata in a thread-local manifest. The manifest binds to the remote sandbox ID. A complete output listing removes entries for deleted files. This avoids repeat downloads when the host filesystem rounds modification times. A single release-time sync pass is bounded by aggregate ceilings (_MAX_SYNC_TOTAL_BYTES, _MAX_SYNC_FILES, _SYNC_DEADLINE_SECONDS) on top of the per-file _MAX_DOWNLOAD_SIZE cap, so a pathological outputs tree cannot make release download unboundedly; a truncated pass logs what it dropped and leaves the manifest un-pruned (only entries observed in that pass are reconciled), so files it never reached are retried on the next release rather than being forgotten.
ACP agent tools:
invoke_acp_agent- Invokes external ACP-compatible agents fromconfig.yaml- ACP launchers must be real ACP adapters. The standard
codexCLI is not ACP-compatible by itself; configure a wrapper such asnpx -y @zed-industries/codex-acpor an installedcodex-acpbinary - MiniMax Code speaks ACP directly: configure
command: mcodewithargs: ["acp"]. It receives DeerFlow's enabled MCP servers and uses the per-thread ACP workspace; the Gateway process must have an authenticatedmcodeexecutable onPATH - ACP results collect only
agent_message_chunktext. Thought chunks remain internal and must not be concatenated into the tool result - Missing ACP executables now return an actionable error message instead of a raw
[Errno 2] - Each ACP agent uses a per-thread workspace at
{base_dir}/users/{user_id}/threads/{thread_id}/acp-workspace/. The workspace is accessible to the lead agent via the virtual path/mnt/acp-workspace/(read-only). In docker sandbox mode, the directory is volume-mounted into the container at/mnt/acp-workspace(read-only); in local sandbox mode, path translation is handled bytools.py