Replace the full-metadata <available_skills> system-prompt block with a
compact <skill_index> (names only) and an on-demand describe_skill tool
when skills.deferred_discovery: true (default: false / backward compat).
New modules:
- skills/catalog.py — SkillCatalog (immutable, searchable; select: has no
cap, keyword/prefix search caps at MAX_RESULTS=5)
- skills/describe.py — build_describe_skill_tool(catalog) closure;
build_skill_search_setup() wires SkillSearchSetup into both the
LangGraph agent factory (agent.py) and DeerFlowClient (client.py)
Changes:
- Skill @dataclass(frozen=True); allowed_tools/required_secrets list→tuple
- Skill First prompt line gated on skill_names (deferred vs legacy wording)
- get_skills_prompt_section: short-circuit storage on deferred path;
merge user_id (upstream) + skill_names (this PR) params
- describe_skill tool parameter named "name" (matches prompt wording)
- select: branch removes [:MAX_RESULTS] cap (exact request, not ranking)
- AGENTS.md: document deferred_discovery config field + new modules
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): add GitHub event-driven agents (#3754)
Add a webhook-driven GitHub channel with fail-closed webhook routing, deterministic per-agent PR/issue threads, mention-gated trigger fan-out, GitHub App token injection for sandboxed gh/git commands, and backend/AGENTS.md documentation.
* fix(llm-middleware): classify bare IndexError as transient
Upstream chat providers occasionally return 200 OK with an empty
generations list (observed against Volces "coding" on
ark.cn-beijing.volces.com). When that happens,
langchain_core.language_models.chat_models.ainvoke raises
``IndexError: list index out of range`` at
``llm_result.generations[0][0].message`` and kills the run.
Treat a bare IndexError reaching the middleware as a transient
upstream-payload glitch and route it through the existing
retry/backoff path instead of failing the whole agent run. The
retry budget and backoff schedule are unchanged.
Adds three regression tests covering the classifier and both the
recover-on-retry and exhausted-retries paths.
* fix(runtime): ignore stale LLM fallback markers from prior runs
When a run on a thread ends with the LLM-error-handling middleware emitting
a `deerflow_error_fallback`-marked AIMessage (e.g. after the IndexError
empty-generations classification fix lands), that message is persisted to
the thread's checkpoint as part of the messages channel. LangGraph replays
the full message history in `stream_mode="values"` chunks, so every
subsequent run on the same thread re-streams the stale fallback marker —
and the worker's chunk scanner faithfully picks it up, flipping
`RunStatus.success` to `RunStatus.error` for runs that themselves had
no LLM failure at all.
Snapshot the set of pre-existing message ids from the pre-run checkpoint
and thread it through `_extract_llm_error_fallback_message` /
`_try_extract_from_message` as a filter. Markers on history messages are
ignored; markers on fresh messages produced during this run still trip
the error path. Falls back to an empty set when the checkpointer is
absent or the snapshot can't be captured, preserving the prior behavior
on first-run / no-state paths.
Adds unit tests for the new filter (helper-level and `_collect_pre_existing_message_ids`)
plus an integration test exercising the full `run_agent` path with a stale
history checkpointer.
* fix(channels): make github channel fire-and-forget to avoid httpx.ReadTimeout on long runs
GitHub agent runs (clone -> edit -> test -> push -> PR) routinely exceed
the langgraph_sdk default 300s read deadline. The manager's runs.wait
call kept an HTTP stream open for the entire run lifetime, so the long
run blew up with httpx.ReadTimeout and the outer except branch then
released the dedupe key and emitted a false 'internal error' outbound.
The GitHub channel's outbound send is log-only by design: agents post to
the issue/PR via the gh CLI in the sandbox when they choose to comment
or create a PR. There is nothing for the manager to ferry back, so the
long-poll was pure overhead.
This change adds ChannelRunPolicy.fire_and_forget (default False) and
sets it True for the github channel. When fire_and_forget is True,
_handle_chat dispatches via client.runs.create (short POST, returns
once the run is pending) instead of client.runs.wait, and skips the
response-extraction + outbound-publish block. ConflictError on a busy
thread still trips the standard THREAD_BUSY_MESSAGE path so behavior on
the busy case is preserved for any future non-github fire-and-forget
channel.
Other (non-github) channels are unchanged: their policy defaults
fire_and_forget=False and they continue to dispatch via runs.wait.
Adds 6 regression tests in tests/test_channels.py::TestGithubFireAndForget:
- Default ChannelRunPolicy.fire_and_forget is False.
- The github policy registers fire_and_forget=True.
- github inbound calls runs.create, not runs.wait, with the right kwargs.
- github inbound publishes no outbound on success.
- ConflictError from runs.create still emits THREAD_BUSY_MESSAGE.
- Non-github channels (slack) still dispatch via runs.wait.
* test(lead-agent): accept user_id kwarg in skill-policy test stubs
The two GitHub-channel tests added in #3754 stubbed
_load_enabled_skills_for_tool_policy with a lambda that only accepted
`available_skills` and `app_config`, but the real function (and its call
site in agent.py) also passes `user_id`. This raised TypeError on every
run, failing backend-unit-tests.
Add `user_id=None` to match the three sibling stubs in the same file.
* refactor(gateway): disambiguate context-key set names
The two frozensets _INTERNAL_ONLY_CONTEXT_KEYS and _CONTEXT_ONLY_KEYS
shared a confusable "CONTEXT_ONLY" token in different orders, and the
first broke the _CONTEXT_<X>_KEYS pattern of its sibling
_CONTEXT_CONFIGURABLE_KEYS. Rename to make the distinct axes explicit:
_CONTEXT_INTERNAL_CALLER_KEYS - WHO: internal callers (scheduler) only
_CONTEXT_RUNTIME_ONLY_KEYS - WHERE: runtime context only, never configurable
Pure rename, no behavior change.
* feat: add scheduled tasks MVP
* fix: harden scheduled task execution semantics
* feat(scheduled-tasks): preset-driven schedule form with timezone and live preview
Replace the raw cron input with a preset Select (hourly/daily/weekly/monthly/custom)
plus structured inputs (time picker, weekday toggles, day-of-month), datetime-local
for one-time tasks, a timezone selector defaulting to the browser timezone, and a
live human-readable preview. Reuses one ScheduledTaskScheduleInput for create and
edit; backend contract unchanged; zero new deps (pure Intl + DST-safe offset helpers).
* feat(scheduled-tasks): full-page i18n + recipe templates + E2E locale pin
Localize the rest of the scheduled-tasks page (filters, detail pane, actions,
edit form, run list, enum values) via t.scheduledTasks.* in en/zh. Add four
built-in recipe templates (GitHub Trending, news digest, issue triage, weekly
report) exposed as a chip row that pre-fills title + prompt + schedule. Pin
Playwright locale to en-US so E2E selectors stay stable against i18n. No backend
change, no new deps.
* fix(scheduled-tasks): idempotent 0003 migration, update head constants, future-date once test
Merge with main surfaced three CI failures:
- 0003_scheduled_tasks create_table collided with legacy test seeds that
build from full metadata; guard with inspector.has_table so the revision
no-ops when the table already exists (0004/0005 are already idempotent via
_helpers.py).
- persistence bootstrap concurrency/regression tests pinned HEAD to main's
0002_runs_token_usage; bump to the new head 0005_scheduled_task_thread_nullable.
- once-task router test used a fixed past run_at and tripped the
must-be-in-the-future validation; use a future date.
* address review: ok-check, 502 for trigger failure, mock fields, migration filename, doc fences
- fetchThreadScheduledTasks now checks response.ok like the other fetchers.
- trigger endpoint returns 502 (not 409) when dispatch fails outright, so
clients can distinguish a real conflict from a server-side failure.
- E2E mock normalizes scheduled-task objects with context_mode/last_thread_id
and nullable thread_id, matching the backend contract the UI renders against.
- Rename 0002_scheduled_tasks.py -> 0003_scheduled_tasks.py to match its
revision id (file was renamed in spirit already; filename now follows).
- CONFIGURATION.md: close the Tool Groups yaml fence and drop the stray fence
after the Scheduler notes so the sections render correctly.
* fix(scheduled-tasks): harden lease, poller, config, and frontend UX after review
* fix(scheduled-tasks): harden run lifecycle, overlap skip, non_interactive gating, and DST conversion after review
- defer a once task's terminal status to the run-completion hook; the task
stays running until the real outcome, and a startup sweep cancels once
tasks orphaned by a crash (launch-time 'completed' could stick forever)
- record interrupted runs as a distinct 'interrupted' run status with a
readable message; an interrupted once task ends 'cancelled', not 'failed'
- enforce overlap_policy=skip for fresh_thread_per_run via an active-run
pre-check (same-thread ConflictError can never fire across fresh threads)
- protect terminal run statuses from the late launch-path 'running' write
- honor context.non_interactive only for internally-authenticated callers;
arbitrary clients can no longer strip ask_clarification
- fix DST-stale timezone offset in zonedLocalToUtcIso by re-deriving the
offset at the resolved instant (once tasks fired an hour late around
spring-forward and the create->edit round-trip diverged)
- drop dead ScheduledTaskRunRepository.update_by_run_id; share one Gateway
API error helper between channels and scheduled-tasks frontends
* fix(scheduled-tasks): close review round-3 gaps in guards, concurrency, and API ergonomics
- scrub internal-only context keys (non_interactive) from the assembled run
config for non-internal callers: gating body.context alone left the same
key smuggle-able through the free-form body.config copied verbatim by
build_run_config
- guard update_after_launch with protect_terminal so the launch bookkeeping
write cannot clobber a once task already finalized by a fast-failing run's
completion hook (parent-row sibling of the run-row guard)
- reject a manual trigger while the task has an active run (409) instead of
launching a duplicate concurrent run on fresh_thread_per_run
- re-arm a terminal once task to enabled when PATCH pushes run_at into the
future; previously the endpoint returned 200 with a next_run_at that could
never be claimed
- make max_concurrent_runs a real global cap: each poll claims only into the
remaining budget of active (queued/running) scheduled runs
- paginate GET /scheduled-tasks/{id}/runs (limit<=200, offset) and push the
thread filter of /threads/{id}/scheduled-tasks into SQL
- stamp context.user_id on scheduler-launched runs, matching IM channels, so
user-scoped guardrail providers see the owning user
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(skills): per-user skill isolation (#2905)
Implement user-scoped skill storage that isolates custom skills between
users while sharing public skills globally.
Key changes:
- Add UserScopedSkillStorage class for per-user custom skill directories
- Introduce get_or_new_user_skill_storage() factory with user_id context
- Auth middleware sets effective_user_id for request-scoped storage
- Agent/prompt/middleware now use user-scoped storage and prompt cache
- Sandbox mounts user-scoped skill directories for search/read tools
- Add validate_skill_file_path() to SkillStorage for path security
- Migration script supports --all-users bulk migration
- Frontend: add editable field to Skill type, error check in enableSkill
- All skill categories can be toggled (custom skills default to enabled)
- Update skill-creator SKILL.md with isolation-aware instructions
Tests:
- Add test_user_scoped_skill_storage.py (new)
- Update all existing skill tests for user-scoped storage
- Update sandbox, client, and router tests
* fix(skills): address second-round PR review feedback (#3889)
- P1-1: restrict legacy skill mount to users without custom skills
- P1-2: fail-closed for _is_disabled_skill_path (OSError → return True)
- P2-1: AND-merge global extensions_config skill disabled state
- P2-2: atomic write for _skill_states.json (mkstemp + replace)
- P2-3: normalize X-DeerFlow-Owner-User-Id in trusted boundary
- P2-4: LRU-bounded _enabled_skills_by_config_cache (OrderedDict, maxsize=256)
- P2-5: clear global prompt cache on PUBLIC skill toggle
- P2-6: invalidate skill caches on client.update_skill
* fix(tests): correct tool policy test after merge
* fix(skills): use DEFAULT_SKILLS_CONTAINER_PATH in UserScopedSkillStorage
The "/mnt/skills" literal in UserScopedSkillStorage.__init__ triggers
test_skill_container_path_defaults::test_mnt_skills_literal_is_owned_by_skill_constants_module
on CI. Migrate the default to the existing deerflow.constants constant,
matching the pattern already used by LocalSkillStorage, SkillStorage, and
the durable/tool_error middlewares.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: emit structured runtime metadata
* fix: avoid subagent import cycle in replay gateway
* fix: preserve legacy subtask result parsing
* refactor: tighten runtime metadata contracts
* fix(middleware): keep recovery hint on task exception wrapper content
The structured-metadata stamp overwrote the wrapper text with the bare
task-failure message, dropping the model-facing 'Continue with available
context, or choose an alternative tool.' guidance that every other tool
exception keeps. Append the shared hint after the formatted message.
* fix(subagents): require lowercase hex for result_sha256 reader
Length-only validation accepted any 64-char string; a faulty serializer
or relaying wrapper could store a non-digest value in the delegation
ledger. Enforce the producer's hexdigest shape with a fullmatch.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: add redis stream bridge
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(gateway): address redis stream bridge review
Redis was imported eagerly through deerflow.runtime and declared as a hard dependency, which made memory-only installs load redis.asyncio at startup and left the lazy factory import ineffective. Move redis behind an optional extra, remove the public eager re-export, and keep make_stream_bridge as the only runtime import path with an actionable install hint when the extra is missing.
Because Docker deployments now default the stream bridge to Redis via DEER_FLOW_STREAM_BRIDGE_REDIS_URL, install the redis extra explicitly in Docker/dev container flows and teach the local uv-extra detector to infer redis from both stream_bridge.type and the Redis URL env var. This keeps Docker working while preserving slim non-Docker installs.
Harden the Redis bridge by batching XREAD replay, replacing brittle ResponseError string matching with a single fallback to 0-0 for malformed Last-Event-ID values, documenting connection/retention/fail-hard behavior, and adding fake plus opt-in real Redis coverage for XADD/XREAD, replay, invalid IDs, and MAXLEN trimming.
* fix(config): bump config version for stream bridge
* fix redis stream bridge terminal handling
* fix: repair uv.lock, format redis.py, and align Dockerfile extras test
The uv.lock file was missing a closing bracket for the redis extras
section, redis.py had a formatting issue caught by ruff, and the
Dockerfile extras test did not account for the hardcoded --extra redis
flag.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* docs(specs): read-before-write gate design for issue #3857 output layer
* docs(plans): read-before-write gate implementation plan (#3857)
* refactor(sandbox): extract read_current_file_content helper (#3857)
* feat(middlewares): read-before-write version gate for file tools (#3857)
* test(middlewares): pin async read-before-write gate paths (#3857)
* feat(config): wire ReadBeforeWriteMiddleware into runtime chain, default on (#3857)
* docs(sandbox): document read-before-write gate in tool docstrings and AGENTS.md (#3857)
* docs(plans): align plan doc with landed config_version (17) and drop machine-specific paths
Addresses Copilot review comments on #3912.
* fix(middlewares): read-before-write gate — error-string sandboxes fail open; serialize gate+execution per path (#3912 review)
- AIO/E2B read_file reports failures (incl. missing files) as 'Error: ...'
strings instead of raising; the gate treated that string as existing file
content and blocked first-write creation. Error-string reads now count as
uninspectable: gate fails open, no mark is stamped.
- LangGraph runs one AIMessage's tool calls concurrently, so two same-turn
writes could both pass on one stale mark before either mutation landed
(and a read mark could hash a version the model never saw). Gate check +
tool execution (and read + mark stamping) now share a per-(thread, path)
critical section, separate from the tool-internal file_operation_lock.
* feat(mcp): add per-server tool_call_timeout for MCP tool calls
Add a configurable timeout for individual MCP tool calls to prevent
agent runs from blocking indefinitely when an MCP server becomes
unresponsive (e.g., rate-limited HTTP API, hung subprocess).
Uses the MCP SDK's built-in read_timeout_seconds parameter on
ClientSession.call_tool, which handles the timeout within the
session's own task — avoiding cross-task cancellation issues with
the session pool (ref #3379, #3203).
Config field is named tool_call_timeout (not timeout) to avoid
collision with langchain-mcp-adapters' existing timeout field on
HTTP/SSE connections.
Closes#3840
* fix(mcp): read tool_call_timeout from McpServerConfig, not connection dict
The previous implementation put tool_call_timeout into the connection dict
returned by build_server_params, which langchain's create_session then
passed to _create_stdio_session(), causing TypeError. Now reads the timeout
directly from ExtensionsConfig.mcp_servers where the wrapper is built,
keeping it out of the connection dict entirely.
Fixes P1 bug from review on #3843.
* test(mcp): regression test for tool_call_timeout not leaking into connection dict
Adds two tests:
- test_build_server_params_excludes_tool_call_timeout: verifies the connection
dict returned by build_server_params() does NOT contain tool_call_timeout
- test_stdio_tool_call_timeout_does_not_raise_typeerror: end-to-end test that
get_mcp_tools() with a stdio server having tool_call_timeout configured loads
tools without TypeError from _create_stdio_session()
Regression for PR #3843 P1 bug.
* fix(mcp): only pass read_timeout_seconds when tool_call_timeout is set
When tool_call_timeout is None, don't pass read_timeout_seconds=None to
session.call_tool(). This avoids breaking existing tests that assert on
exact call_tool arguments without the extra kwarg.
* docs(mcp): clarify stdio tool timeout
* feat(observability): add trace-id correlation and enhanced logging
- add opt-in gateway request trace correlation via X-Trace-Id
- enhance logging with configurable trace_id-aware formatting
- propagate deerflow_trace_id into runtime context and Langfuse metadata
- keep enhanced logging disabled by default to preserve existing behavior
* fix: harden trace correlation wiring
- Make logging enhancement a restart-required startup snapshot and remove per-request config reads from TraceMiddleware
- Restrict trace ids to printable ASCII before writing them to response headers, logs, and Langfuse metadata
- Gate implicit DeerFlowClient trace-id creation behind logging.enhance.enabled while preserving explicit caller opt-in
- Bind embedded client trace context per stream step to avoid generator ContextVar leaks and cross-context reset errors
- Rebind memory update trace ids in Timer/executor worker paths so enhanced logs keep the captured correlation id
- Remove unrelated __run_journal context overwrite from the trace-correlation change set
* fix(gateway): avoid eager app construction on package import
* fix(gateway): avoid config load during app import
Keep Gateway app construction import-safe when config.yaml is absent by
disabling TraceMiddleware only for that construction-time fallback path.
Startup lifespan still performs strict config loading before serving.
build_run_config() copied every top-level request key (except
configurable/context) verbatim into the LangGraph RunnableConfig, including
recursion_limit. The server default of 100 was fully overridable by the caller
with no upper bound, so a request like {"config": {"recursion_limit": 100000000}}
could make a single run execute effectively unbounded LangGraph super-steps
(each >= 1 LLM call), enabling runaway API cost / DoS.
Validate the client value server-side and clamp it into a safe range:
- valid positive ints are capped at a configurable ceiling
(AppConfig.max_recursion_limit, default 1000 to match the existing
frontend/public-skill default so legitimate deep runs are unaffected)
- invalid/non-positive/bool/None values fall back to the 100 server default
- applied on both the configurable and the LangGraph >= 0.6.0 context paths
- WARNING logged on clamp for observability
Add unit tests (including a configurable-ceiling case), expose
max_recursion_limit in config.example.yaml (config_version 16), and document
the ceiling/fallback in backend/docs/API.md.
Co-authored-by: DengY11 <DengY11@users.noreply.github.com>
* fix(sandbox): stop blocking bash commands from hanging the turn
Starting a server through the host bash tool (e.g. `python -m http.server`)
could hang the whole turn for the full 600s timeout. `LocalSandbox.execute_command`
used `subprocess.run(capture_output=True)`, whose captured pipes are inherited
by any process the command spawns — so a backgrounded long-lived process
(`server &`) keeps the read end open and blocks `communicate()` until the
timeout fires, even though the foreground command already returned. Commands
that read stdin blocked the same way, and on timeout only the direct child was
killed, leaving orphaned process groups.
Rework the POSIX path to capture stdout/stderr via temp files instead of pipes,
take stdin from /dev/null, and run the command in its own session/process group:
- Backgrounded long-lived processes (servers) now return immediately while the
process keeps running.
- A command reading stdin gets immediate EOF instead of blocking.
- A genuinely blocking foreground command is bounded by a configurable
wall-clock timeout; on timeout the whole process group is killed and the agent
gets an explanatory notice telling it to background long-lived processes.
The timeout is configurable via `sandbox.bash_command_timeout` (default 600).
The Windows path is unchanged. Adds focused regression tests and updates docs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(sandbox): instruct the agent to background long-lived processes
The code fix bounds a foreground server with a timeout, but the turn still
waits the full timeout before the run continues. Add the prompt-side half:
the bash tool description now tells the model to ALWAYS start long-lived
processes (e.g. web servers) in the background with output redirected, so the
tool returns immediately. The timeout notice points at the same readable
workspace log path. Pins the guidance with a test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(sandbox): make fallback-kill exception explicit and observable
Address automated review: the inner `except OSError: pass` in
_terminate_process_group silently swallowed the case where the direct-child
fallback kill found the process already gone. Make the intent explicit with a
comment and a debug log instead of a bare pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(sandbox): address bash timeout review feedback
* fix(sandbox): document fd cleanup races
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
get_model_config / get_tool_config / get_tool_group_config did a next(...)
linear scan of self.models / self.tools / self.tool_groups on every call.
These sit on hot paths: get_tool_config runs 2-3x per community-tool
invocation (web_search etc.) and get_model_config several times per agent
build, across 40+ call sites.
Build name -> config dicts once in a mode="after" validator (stored as
PrivateAttr), so each getter is an O(1) dict lookup. A config reload
constructs a fresh AppConfig, which rebuilds the indexes; setdefault keeps
first-match-wins on duplicate names, matching the prior next(...) semantics.
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(memory): add guaranteed injection for correction facts with graceful fallback
When the token budget is tight, high-value facts (e.g. user corrections)
can be silently evicted by lower-priority regular facts. This change:
- Introduces configurable 'guaranteed_categories' (default: [correction])
whose facts draw from a separate 'guaranteed_token_budget', ensuring
they are never dropped due to budget pressure.
- Adds a graceful fallback to confidence-only ranking when the
guaranteed-category path raises an unexpected exception.
- Refactors fact selection into a header-agnostic helper
(_select_fact_lines) with explicit token accounting in the caller,
eliminating double-counting of separators.
- Emits a single 'Facts:' header regardless of whether both guaranteed
and regular facts are present.
- Extends the final safety truncation limit to account for the
additional guaranteed budget so guaranteed facts survive end-to-end.
* refactor(memory): address review feedback on guaranteed injection
- Restore strict break-on-overflow in `_select_fact_lines` to preserve
the caller's confidence-ordered ranking; add a regression test locking
in the invariant that a shorter lower-confidence fact never slips
ahead of a skipped higher-confidence one.
- Account for the inter-group `\n` separator between guaranteed and
regular fact blocks in the regular budget (1-token precision fix).
- Clarify docstrings on `format_memory_for_injection` and
`MemoryConfig.guaranteed_token_budget` to distinguish the common
*displacement* case (total stays within `max_tokens`) from the rarer
*additive* case (safety-truncation ceiling raised when guaranteed
lines alone would overflow).
* fix(memory): address P1 safety truncation + P2s from review
- Structure-aware safety truncation: Facts block is now a protected
suffix so guaranteed-category facts can never be silently discarded
by a prefix-cut on overflow. Only the preceding (user/history)
sections are eligible for truncation.
- Extend the same protected-suffix treatment to the except/fallback
path by returning fact lines alongside the formatted section from
_fallback_format_facts, avoiding string parsing.
- Single inter-section separator: facts section no longer embeds its
own leading \n\n; the final "\n\n".join(sections) is the single
source of truth for section-to-section spacing.
- Bare string for guaranteed_categories now raises TypeError instead
of silently iterating single characters.
- Category-less / malformed facts no longer default-promote into the
guaranteed "context" pool — only facts with an explicit category
field qualify.
- Lift valid_facts pre-filter outside the try so the fallback path
reuses it instead of re-doing validation work.
- MemoryConfigResponse + DeerFlowClient.get_memory_config now expose
guaranteed_categories / guaranteed_token_budget.
- config.example.yaml: document the two new fields and bump
config_version from 12 to 13.
- Add regression tests for every finding.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(config): coerce null object config sections to their defaults
#3434 made commented-out list sections (models/tools/tool_groups) parse as []
instead of crashing, but the same scenario still crashes for object sections:
commenting out every key under e.g. memory: / summarization: / guardrails:
makes PyYAML parse the value as None, and AppConfig then raises "Input should
be a valid dictionary" for that section — breaking the documented
`cp config.example.yaml config.yaml` first-run flow.
Generalize the handling: a model_validator(mode="before") drops None-valued
sections so each field falls back to its default (list sections -> [], object
sections -> their default config). This subsumes the previous list-only
field_validator and the database special-case. Required sections without a
default (sandbox) still error when null.
Adds test_app_config_coerces_commented_out_object_sections; the existing
list-section regression test still passes.
* docs+test: address review on null-section coercion
- Correct the _drop_null_config_sections docstring: it does not subsume the
database special-case; _apply_database_defaults still owns `database` and
applies concrete defaults beyond null-coercion.
- Strengthen test_app_config_coerces_commented_out_object_sections to assert
each null section falls back to its expected default config type, not just
that it is non-None.
- Add test_app_config_null_required_section_still_errors covering the
required-section (sandbox) "still errors when null" claim.
---------
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
* fix(channels): require bound identity for user-owned IM messages
* make format
* docs: document bound identity channel config
* refactor: reuse channel connection config
* refactor _requires_bound_identity()
* refactor from_app_config()
* make format
* fix: reject unbound channel chats before semaphore
* security enhancement
* make format
* fix: enforce bound-identity admission at command entry point
The bound-identity gate only ran for non-command messages in
_handle_message() and as a fallback inside _handle_chat(). Commands had
no equivalent boundary, so an unbound platform user could send /new and
reach _create_thread() directly, creating an unowned Gateway thread and
empty checkpoint. Info commands (/status, /models, /memory) likewise
leaked Gateway state to unbound users.
Add the same _requires_bound_identity() check at the top of
_handle_command(), rejecting via _reject_unbound_channel_message() before
any thread creation or Gateway query. The gate is a no-op in legacy
open-bot mode (require_bound_identity=False) and auth-disabled mode.
Provider-level binding flows (/connect, /start) are consumed by the
provider adapter before reaching the manager, so they are unaffected.
Tests:
- unbound auth-enabled /new is rejected before threads.create
- bound auth-enabled /new still creates the thread
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(channels): carry workspace fallback decision on inbound messages
* fix(channels): recheck bound identity by normalized workspace
* fix(channels): avoid duplicate bound identity checks
* fix(channels): preserve verified routing for bound identity rejects
* fix(channels): clarify bound identity upgrade failures
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(subagents): raise general-purpose max_turns to 150 and default timeout to 30min
Deep-research subtasks failed out of the box with GraphRecursionError (Recursion limit of 100 reached): the built-in general-purpose subagent caps at max_turns=100. Raise it to 150 and bump the default subagent timeout from 900s (15min) to 1800s (30min) so the extra turns have time to run instead of shifting the failure to a timeout. The lead agent recursion_limit (100) is unchanged; the failures are subagent-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(subagents): clarify lead recursion_limit is independent of subagent max_turns
Add comments at both lead recursion_limit=100 sites (gateway services + channel manager) explaining the lead's LangGraph super-step budget is separate from subagent depth, so the two 100s are not conflated. Comment-only, no behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(subagents): clarify built-in vs custom timeout scope; pin bash max_turns in test
Review follow-ups: (1) clarify SubagentConfig docstring + global timeout field/comment that the 1800 default applies to built-in subagents (custom agents keep their own timeout_seconds); (2) pin bash.max_turns==60 in the defaults regression test so the config.example.yaml doc cannot drift; (3) rename test_default_timeout_preserved_when_no_config -> test_explicit_global_timeout_propagates_to_general_purpose since it intentionally exercises an explicit non-default 900. No runtime behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Add user-owned IM channel connections
* Fix dev startup and channel connect popup
* Use async channel connect flow
* Harden dev service daemon startup
* Support local IM channel connections
* Align IM connections with local channels
* Fix safe user id digest algorithm
* Address Copilot IM channel feedback
* Address IM channel review comments
* Support all integrated IM channel connections
* Format additional channel connection tests
* Keep unavailable channel connect buttons clickable
* Fix IM channel provider icons
* Add runtime setup for enabled IM channels
* Guard global shortcut key handling
* Keep configured IM channels editable
* Avoid password autofill for channel secrets
* Make channel threads visible to connection owners
* Persist IM runtime config locally
* Allow disconnecting runtime IM channels
* Route no-auth channel sessions to local user
* Use default user for auth-disabled local mode
* Show IM channel source on threads
* Prefill IM channel runtime config
* Reflect IM channel runtime health
* Ignore Feishu message read events
* Ignore Feishu non-content message events
* Let setup wizard enable IM channels
* Fix frontend formatting after merge
* Stabilize backend tests without local config
* Isolate channel runtime config tests
* Address channel connection review comments
* Use sha256 user buckets with legacy migration
* Ensure runtime IM channels are ready after restart
* Persist disconnected IM channel state
* Address channel connection review comments
* Address channel connection review findings
Frontend connect flow:
- Open the runtime-config dialog only when a provider still needs
credentials; configured providers go straight to the connect flow, so
the binding-code/deep-link path is reachable from the UI again.
- After saving credentials, continue into the connect flow when a user
binding is still required (multi-user mode) instead of stopping at a
"Connected" toast.
- Extract shared provider-state helpers to core/channels/provider-state
and add unit + e2e coverage for the direct-connect and
configure-then-connect paths.
Provider status semantics:
- Report connection_status from the user's newest connection row;
with no binding it is not_connected, except in auth-disabled local
mode where a configured running channel is effectively connected.
Concurrency and event-loop correctness:
- Offload ChannelRuntimeConfigStore construction and writes, channel
service construction, and Slack connection replies to threads; add a
tests/blocking_io/ anchor for the runtime-config handlers.
- Consume binding codes with a conditional UPDATE so a code can only be
used once under concurrent workers; retry upsert_connection as an
update when a concurrent insert wins the unique constraint.
- Serialize ensure_channel_ready per channel so concurrent provider
polls cannot double-start a channel worker.
Config and migration hardening:
- Stop mutating the get_app_config()-cached Telegram provider config;
the runtime store now owns the UI-entered bot username.
- Register channel_connections in STARTUP_ONLY_FIELDS with the
standardized startup-only Field description.
- Match the legacy unsafe-id bucket by recomputing its exact SHA-1 name
so another user's same-prefix bucket can never be migrated.
- Remove the unused Telegram process_webhook_update path and document
src/core/channels in the frontend docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address PR review comments on authz scoping and channel runtime
Security (review feedback from ShenAC-SAC):
- Scope internal-token callers to the connection owner carried in
X-DeerFlow-Owner-User-Id instead of bypassing owner checks outright,
in both require_permission(owner_check=True) and the stateless run
endpoints. Internal callers keep access to their own and
shared/legacy threads, and may claim a default-owned channel thread
for its real owner, but a leaked internal token no longer grants
cross-user thread access.
- Require admin privileges for POST/DELETE /api/channels/{provider}/
runtime-config: runtime credentials and channel workers are
instance-wide shared state (same model as the MCP config API).
Read-only provider listing stays available to all users.
Performance (review feedback from willem-bd):
- Skip the redundant thread channel-metadata PATCH after the first
successful backfill per thread.
- Reuse the per-connection Slack WebClient until its token changes
instead of constructing one per outbound message.
- Reconcile channel readiness for all providers concurrently in
GET /api/channels/providers.
Also resolve the code-quality unused-import flag in the blocking-io
anchor by pre-importing the channel service via importlib.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix prettier formatting in provider-state test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Reconcile UI runtime channel config with config reload on restart
Main now reloads a channel's config.yaml entry on restart_channel()
(#3514, issue #3497). Adapt the user-owned connection flow to coexist:
- configure_channel() restarts with reload_config=False — the caller
just supplied the authoritative config (browser-entered credentials
that are never written to config.yaml), so a file reload must not
clobber it with the stale on-disk entry.
- _load_channel_config() re-applies the UI runtime-store overlay used
at startup, so an operator-triggered restart keeps browser-entered
credentials for channels without a config.yaml entry and does not
resurrect a channel disconnected from the UI.
- Offload the reload's disk IO (config.yaml + runtime store) with
asyncio.to_thread, matching the blocking-IO policy on this branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
When memory is enabled, the first conversation with a legacy shared agent
creates a per-user agent directory containing only memory.json (no
config.yaml). On the second turn, resolve_agent_dir() returned this
incomplete directory, causing load_agent_config() to fail with
"Agent config not found".
Require config.yaml to exist alongside the directory for both the
per-user and legacy paths, so that memory-only directories fall
through correctly. This aligns resolve_agent_dir with the existing
config.yaml check in list_custom_agents.
Refs: https://github.com/bytedance/deer-flow/issues/3390
* feat(memory): add memory.token_counting config to avoid tiktoken network dependency (#3429)
Add a `memory.token_counting` option (`tiktoken` | `char`) so deployments in
network-restricted environments can opt out of tiktoken entirely. In `char`
mode the memory-injection budget uses a network-free character-based estimate
and never triggers the BPE download from openaipublic.blob.core.windows.net,
which could otherwise block for tens of minutes (see #3402).
Also harden the default `tiktoken` path:
- cache an in-flight LOADING sentinel so concurrent callers fall back
immediately instead of spawning more blocking get_encoding threads when the
first load is still running (e.g. under the 5s startup warm-up timeout);
- cache failures with a timestamp and retry after a cooldown so a transient
network outage self-heals back to accurate counting without a restart;
- skip startup warm-up entirely in char mode.
The new config is surfaced via the memory config API and config.example.yaml
(config_version bumped). Default remains `tiktoken`, so existing deployments
are unaffected.
* fix(memory): use CJK-aware char token estimate and address review feedback
- Replace the flat len(text)//4 fallback with a CJK-aware estimate so
Chinese/Japanese/Korean memory content does not over-fill the injection budget
- Document the internal tiktoken retry cooldown and char-mode escape hatch
- Sync CLAUDE.md / config.example.yaml / MEMORY_IMPROVEMENTS.md wording
- Fix MemoryConfigResponse mocks/assertions and add CJK estimate tests
In Docker production deployments, LocalSandboxProvider runs inside the
deer-flow-gateway container, so any `sandbox.mounts[].host_path` from
config.yaml is resolved against the gateway container's filesystem — not
the host machine. When the path isn't also bind-mounted into the gateway
service, the mount was silently dropped with only a WARNING log, leaving
agents reading an empty directory in production while the same config
worked under `make dev`.
Escalate the missing-host_path branch to logger.error with explicit
guidance about Docker bind mounts and docker-compose, so the failure is
hard to miss in default log configurations. Skip behaviour is preserved
to avoid breaking existing deployments.
Also clarify the misleading `VolumeMountConfig.host_path` field
description so it documents reality for both providers:
- LocalSandboxProvider checks host_path from inside the gateway process
(host in `make dev`, container in `make up`).
- AioSandboxProvider (DooD) passes host_path straight to `docker -v`
for the sandbox container, where the host Docker daemon resolves it
from the host machine's perspective.
config.example.yaml's `sandbox.mounts` comment gets a Note: block
pointing operators at the docker-compose bind-mount requirement so the
Docker-mode gotcha is discoverable from the canonical template.
Adds a regression test that:
- confirms missing host_path is still skipped (no behaviour break);
- asserts an ERROR record is emitted referencing the offending paths;
- asserts the message contains actionable Docker/gateway/docker-compose
keywords so future refactors can't quietly downgrade it.
Refs: https://github.com/bytedance/deer-flow/issues/3244
Copying config.example.yaml to config.yaml and starting DeerFlow crashed with `pydantic ValidationError: models — Input should be a valid list [input_value=None]`, because the example ships every entry under `models:` commented out, so PyYAML parses the key as null. Reported in #1444.
Add a field_validator(mode="before") on AppConfig that coerces null models/tools/tool_groups to [] (matching their default_factory=list), and emit an actionable warning from from_file when no models are configured (pointing to config.example.yaml / make setup). Adds regression tests.
Closes#1444
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(config): make the reload boundary discoverable from code, not just docs
Closes#3144.
The hot-reload contract — per-run fields are resolved through
`get_app_config()` on every request, infrastructure fields snapshot at
gateway startup — landed in `backend/CLAUDE.md` as part of #3131. A
maintainer reading `get_config()` or an `AppConfig` field still had to
context-switch to that document to know which fields require a process
restart, and there was no enforcement that the prose list stayed in
sync with the code.
This commit moves the boundary to a machine-readable single source of
truth and surfaces it where the code lives:
- New `deerflow.config.reload_boundary` module owns the registry of
restart-required fields (`STARTUP_ONLY_FIELDS`) and a tiny helper
API (`is_startup_only_field`, `iter_startup_only_field_paths`,
`format_field_description`). The standardised `"startup-only:"`
prefix is exported as `STARTUP_ONLY_PREFIX` so future scanners /
lint hooks / doc generators can pivot off it without re-parsing
prose.
- `AppConfig`'s `database`, `checkpointer`, `run_events`,
`stream_bridge`, `sandbox`, and `log_level` fields now build their
`Field(description=...)` from `format_field_description(...)`. The
same text shows up in IDE hover (Pydantic v2 exposes `description`
via `model_fields[...]`).
- `channels` is restart-required too but lives outside the AppConfig
Pydantic schema (the config section is consumed directly by
`start_channel_service`). The registry owns it so the boundary is
not split between two places.
- `get_config()` docstring points to the registry instead of leaving
the reader to find `CLAUDE.md`. The `CLAUDE.md` table collapses to
a one-liner pointing back at `reload_boundary.py` so the boundary
has one canonical location, not two.
Drift coverage in `tests/test_reload_boundary.py`:
- Every registered field has a non-trivial reason.
- Iterator / membership helpers stay in sync with the dict.
- Every registry entry that maps to an `AppConfig` field also carries
the `"startup-only:"` prefix in the schema (catches "forgot to
update the schema").
- Reverse drift: any AppConfig field whose description starts with
the prefix must be registered (catches "marked restart-required in
the schema but forgot the registry").
- The runtime introspection that IDE hover depends on
(`AppConfig.model_fields["database"].description`) is pinned, so a
future Pydantic upgrade or schema swap that breaks the hover surface
shows up as a test failure rather than a silent regression.
Refs: bytedance/deer-flow#3138 (split summary), #3107 (origin), #3131
(prior boundary fix in prose form).
* fix(config): preserve field doc and correct log_level reload reason
Two follow-ups on the PR #3153 review:
1. The `log_level` STARTUP_ONLY_FIELDS reason previously claimed
`apply_logging_level()` mutates the root logger level. It does not:
only the `deerflow` / `app` logger levels are set, and root handler
thresholds are conditionally lowered so messages from those loggers
can propagate. Reword to match the actual behavior so operators
reading IDE hover get accurate restart guidance.
2. `format_field_description(field_path)` was the sole `Field(description=)`
for every restart-required field, which silently overwrote the
original human-facing documentation — most visibly the `log_level`
field that used to list debug/info/warning/error and clarify that
third-party libraries are not affected. Extend the helper with a
keyword-only `field_doc` parameter that composes the startup-only
marker with the original prose so IDE hover documents both *why*
the field is restart-required and *what* it actually accepts.
Updated all six restart-required AppConfig fields (`log_level`,
`database`, `sandbox`, `run_events`, `checkpointer`, `stream_bridge`)
to pass their original descriptions through the helper.
Tests: two new cases in `test_reload_boundary.py` pin (a) the helper
composition and (b) every AppConfig restart-required field still
surfaces a recognisable substring of its original documentation.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(#3189): prevent write_file streaming timeout on long reports
Adds a layered defense against StreamChunkTimeoutError caused by oversized
single-shot write_file tool calls:
- factory: default stream_chunk_timeout to 240s for OpenAI-compatible
clients (overridable via ModelConfig.stream_chunk_timeout in config.yaml)
- sandbox/tools: server-side 80 KB length guard on non-append write_file
calls (configurable via DEERFLOW_WRITE_FILE_MAX_BYTES env var, 0 disables);
rejects oversized payloads with a structured error pointing the model at
str_replace or append=True
- middleware: classify StreamChunkTimeoutError as transient but cap retries
at 1 via per-exception _RETRY_BUDGET_OVERRIDES (same-payload retry on a
chunk-gap timeout buffers the same way upstream; full 3-attempt loop
would stack 6-12 min of dead air)
- middleware: surface an actionable user-facing message for stream-drop
exceptions instead of leaking the raw langchain stack
- prompts: add a routing-style File Editing Workflow hint to both lead_agent
and general_purpose subagent prompts, pointing the model at str_replace
for incremental edits (mirrors Claude Code's Edit / Codex's apply_patch)
- tests: behavioural coverage for size guard, retry budget override,
stream-drop user message, factory default injection
Refs #3189
* fix(#3189): drop stream_chunk_timeout for non-OpenAI providers
Address CR feedback on PR #3195:
- factory: pop `stream_chunk_timeout` from kwargs for any model_use_path other than `langchain_openai:ChatOpenAI` instead of returning early. `ModelConfig.stream_chunk_timeout` is part of the shared schema, so a user-supplied value on a non-OpenAI provider would otherwise be forwarded to its constructor and raise `TypeError: unexpected keyword argument`.
- factory: rewrite docstring to describe the actual `exclude_none=True` behaviour (explicit null is excluded and falls back to the default) instead of the misleading "None falling out via exclude_none=True keeps its value".
- tests: add regression coverage asserting the kwarg is stripped before reaching a non-OpenAI provider's constructor.
Refs: bytedance#3189
* fix(#3189): restrict stream-drop user copy to StreamChunkTimeoutError only
Per CR on #3195: narrow _STREAM_DROP_EXCEPTIONS to StreamChunkTimeoutError. Generic httpx RemoteProtocolError / ReadError fall back to the standard 'temporarily unavailable' copy, since they routinely fire on transient network blips where the 'split the output' guidance is misleading. Retry/backoff classification is unchanged — both remain transient/retriable. Tests updated to reflect new copy, plus a symmetric regression test for ReadError.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The official MCP configuration schema uses `transport` to specify the
transport mechanism (stdio/sse/http), but `McpServerConfig` only honored
`type` and defaulted to `stdio`. Remote MCP servers configured with just
`transport: sse` were therefore misidentified as stdio and failed with
"with stdio transport requires 'command' field".
Add a model validator that promotes `transport` to `type` when only
`transport` is provided, while keeping `type` authoritative when both
are set. This matches the MCP-spec field name without breaking existing
configurations.
Fixes#3238
* feat(agent): add ToolOutputBudgetMiddleware for oversized tool output protection
Closes#3289. Adds a unified middleware that enforces per-result budgets
on ALL tool outputs (MCP, sandbox, community, custom), preventing
oversized external tool results from blowing the model context window.
Design informed by claude-code (persistToolResult), hermes-agent
(tool_result_storage), and pi (OutputAccumulator) — the three most
mature implementations in production coding-agent frameworks.
Key features:
- Disk externalization: oversized outputs written to thread-local
.tool-results/ directory, replaced with compact preview + file
reference. Model can read full output via read_file with offset/limit.
- Fallback truncation: head+tail truncation when disk is unavailable
(no thread_data, write failure), ensuring the context is always
protected.
- read_file exemption: prevents persist-read-persist infinite loops
(independently discovered by claude-code, hermes-agent, and pi).
- Per-tool threshold overrides via config.
- Line-boundary-aware truncation (no partial lines in previews).
- Multimodal content passthrough (images/structured blocks skip budget).
- Historical ToolMessage patching in wrap_model_call for checkpoint
recovery scenarios.
Related: #3222 (design RFC), #1844 (comprehensive context management),
#3137 (write_file args compaction), #1677 (sandbox tool truncation).
* test: add MCP content_and_artifact format coverage
Add 5 tests for MCP tool output format (list of content blocks):
- text content blocks are extracted and budgeted
- multiple text blocks are joined and budgeted
- image content blocks are skipped (multimodal passthrough)
- mixed text+image blocks are skipped
- small text blocks pass through unchanged
Total test count: 59 (was 54).
* fix(agent): address Codex review findings for ToolOutputBudgetMiddleware
Three issues identified by Codex code review, all fixed:
1. `enabled` config field was unused — middleware now checks
`config.enabled` and skips all processing when disabled.
2. `_build_fallback` could exceed `fallback_max_chars` — the marker
text itself (~139 chars) was not deducted from the budget. Now
pre-computes marker overhead and falls back to hard slice when
max_chars is smaller than the marker.
3. Sync file I/O in async path — `awrap_tool_call` now delegates
`_patch_result` to `asyncio.to_thread` to avoid blocking the
event loop during disk writes.
Tests updated to use realistic fallback_max_chars values (500+)
that can accommodate the marker overhead, plus two new tests:
- `test_result_never_exceeds_max_chars` (parametric across sizes)
- `test_very_small_max_chars_does_not_crash`
* fix(agent): address Copilot review — path traversal, async perf, shared config
1. Path traversal defense: sanitize tool_name via _sanitize_tool_name()
(strips separators, .., absolute paths), validate storage_subdir is
relative, and verify resolved filepath stays inside storage_dir.
2. Async hot-path optimization: add _needs_budget() cheap check before
asyncio.to_thread offload — small outputs (99% of calls) skip the
thread overhead entirely.
3. Replace shared module-level _DEFAULT_CONFIG with _default_config()
factory to prevent cross-instance mutation of mutable fields.
12 new tests: TestSanitizeToolName (5), TestExternalizePathTraversal (3),
TestNeedsBudget (4).
* fix(agent): correct preview hint to match read_file actual API
read_file uses start_line/end_line (1-indexed line numbers), not
offset/limit. The previous wording was copied from hermes-agent
which has a different read_file interface.
* perf(agent): hoist hot-path imports, add model-call pre-scan (review #3303)
Address maintainer review feedback:
1. Hoist inline imports to module level — `import asyncio` (was in
awrap_tool_call hot path) and `from dataclasses import replace`
(was in _patch_result) now live at module top.
2. Add a cheap pre-scan to _patch_model_messages so the historical
message list is not rebuilt on every model call when nothing is
oversized (the common case once results are budgeted at tool-call
time). Also adds the same _needs_budget gate to the sync
wrap_tool_call for symmetry with awrap_tool_call.
The pre-scan is refactored into per-tool-aware helpers
(_effective_trigger / _tool_message_over_budget) that mirror the exact
trigger conditions in _budget_content — including tool_overrides — so
the fast-path can never produce a false negative (silently skipping
budgeting for a tool with a low per-tool threshold).
7 new regression tests lock the per-tool-override-through-pre-scan path
and the model-call early return.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(runtime): suppress tool execution when provider safety-terminates with tool_calls
When a provider stops generation for safety reasons (OpenAI/Moonshot
finish_reason=content_filter, Anthropic stop_reason=refusal, Gemini
finish_reason=SAFETY/BLOCKLIST/PROHIBITED_CONTENT/SPII/RECITATION/
IMAGE_SAFETY/...), the response may still carry truncated tool_calls.
LangChain's tool router treats any non-empty tool_calls as executable,
so partial arguments (e.g. write_file with a half-finished markdown)
get dispatched and the agent loops on retry.
Add SafetyFinishReasonMiddleware at after_model: detect safety
termination via a pluggable detector registry, clear both structured
tool_calls and raw additional_kwargs.tool_calls / function_call,
preserve response_metadata.finish_reason for downstream observers,
stamp additional_kwargs.safety_termination for traces, append a
user-facing explanation to message content (list-aware for thinking
blocks), and emit a safety_termination custom stream event so SSE
consumers can reconcile any "tool starting..." UI.
Default detectors cover OpenAI-compatible content_filter, Anthropic
refusal, and Gemini safety enums (text + image). Custom providers are
added via reflection (same pattern as guardrails). Wired into both
lead-agent and subagent runtimes.
Closes#3028
* fix(runtime): persist safety_termination as a middleware audit event
Address review on #3035: the SSE custom event is great for live
consumers but invisible to post-run audit. RunEventStore should carry
its own row so operators can answer "which runs were safety-suppressed
today?" from a single SQL query without joining the message body.
Worker now exposes the run-scoped RunJournal via
runtime.context["__run_journal"] (sentinel key, internal channel).
SafetyFinishReasonMiddleware calls the previously-unused
RunJournal.record_middleware, which emits
event_type = "middleware:safety_termination"
category = "middleware"
content = {name, hook, action, changes={
detector, reason_field, reason_value,
suppressed_tool_call_count,
suppressed_tool_call_names,
suppressed_tool_call_ids,
message_id, extras}}
Tool *arguments* are deliberately excluded — those are the very content
the provider filtered and persisting them would defeat the purpose of
the safety filter (per review note in #3035).
Graceful skips when journal is absent (subagent runtime, unit tests,
no-event-store local dev). Journal exceptions never propagate into the
agent loop.
Refs #3028
* fix(runtime): satisfy ruff format + address Copilot review
- ruff format on safety_finish_reason_config.py and e2e demo (CI lint
failed on ruff format --check; backend Makefile lint target runs
ruff check AND ruff format --check).
- Docstring on SafetyFinishReasonConfig now says resolve_variable to
match the actual loader used in from_config (the wording was
resolve_class previously; behavior is unchanged — resolve_variable
mirrors how guardrails.provider is loaded).
- Switch the AIMessage type check in SafetyFinishReasonMiddleware._apply
from getattr(last, "type") == "ai" to isinstance(last, AIMessage),
matching TokenUsageMiddleware / TodoMiddleware / ViewImageMiddleware
/ SummarizationMiddleware which are the dominant pattern.
Refs #3028
* fix(tracing): propagate session_id and user_id into Langfuse traces
Adds Langfuse v4 reserved trace attributes (langfuse_session_id,
langfuse_user_id, langfuse_trace_name, langfuse_tags) to
RunnableConfig.metadata inside the run worker, so the langchain
CallbackHandler can lift them onto the root trace.
- New deerflow.tracing.metadata.build_langfuse_trace_metadata() returns
the reserved keys when Langfuse is in the enabled providers, else {}.
- worker.run_agent merges them with setdefault so caller-supplied keys
win, allowing per-request overrides from upstream metadata.
- session_id mirrors the LangGraph thread_id; user_id reads
get_effective_user_id() (falls back to "default" in no-auth mode).
- trace_name defaults to "lead-agent"; tags carry env and model name
when DEER_FLOW_ENV (or ENVIRONMENT) and a model name are present.
Closes#2930
* fix(tracing): attach Langfuse callback at graph root so metadata propagates
The first commit injected ``langfuse_session_id`` / ``langfuse_user_id`` /
``langfuse_trace_name`` / ``langfuse_tags`` into ``RunnableConfig.metadata``,
but on ``main`` the Langfuse callback is attached at *model* level
(``models/factory.py``). LangChain still threads ``parent_run_id`` through
the contextvar, so the handler sees the model as a nested observation and
``__on_llm_action`` strips the ``langfuse_*`` keys
(``keep_langfuse_trace_attributes=False``). The trace's top-level
``sessionId`` / ``userId`` therefore stayed empty in deer-flow's LangGraph
runtime — confirmed live against a real Langfuse instance.
This commit moves the callback to the **graph invocation root** so the
handler fires ``on_chain_start(parent_run_id=None)`` and runs the
``propagate_attributes`` path that actually lifts ``session_id`` /
``user_id`` onto the trace:
- ``models/factory.py``: add ``attach_tracing`` keyword (default ``True``)
so standalone callers (``MemoryUpdater``, etc.) keep their direct
model-level tracing.
- ``agents/lead_agent/agent.py``: call ``build_tracing_callbacks()`` once
inside ``_make_lead_agent`` and append the result to
``config["callbacks"]``; the four in-graph ``create_chat_model`` sites
(bootstrap, default agent, sync + async summarization) pass
``attach_tracing=False`` to avoid duplicate spans.
- ``agents/middlewares/title_middleware.py``: same ``attach_tracing=False``
for the title-generation model, since it inherits the graph's
RunnableConfig via ``_get_runnable_config``.
Test updates:
- ``tests/test_lead_agent_model_resolution.py`` and
``tests/test_title_middleware_core_logic.py``: extend the fake
``create_chat_model`` signatures / mock assertions to accept the new
``attach_tracing`` kwarg.
- ``tests/test_worker_langfuse_metadata.py``: switch the no-user fallback
test from direct ContextVar mutation to ``monkeypatch.setattr`` on
``get_effective_user_id`` to avoid pollution across the langfuse OTel
global tracer provider.
- ``tests/conftest.py``: add an autouse fixture that resets
``deerflow.config.title_config._title_config`` to its pristine default
after every test. Any test that loads the real ``config.yaml`` (via
``get_app_config()``) calls ``load_title_config_from_dict`` and mutates
the module-level singleton, which previously poisoned the
title-middleware suite when run after, e.g., the new
``test_worker_langfuse_metadata.py`` cases. The fixture is independent
of this PR's main change but unblocks the cross-file test run.
Live verification (same Langfuse instance as before):
- Drove ``worker.run_agent`` against the real ``make_lead_agent`` +
``gpt-4o-mini`` for three distinct ``user_context`` identities
(``fancy-engineer``, ``alice-pm``, ``bob-designer``).
- Each run produced one ``lead-agent`` trace whose top-level
``sessionId`` / ``userId`` / ``tags`` carry the expected values, e.g.
``session=e2e-2930-8f347c-alice-pm user=alice-pm name='lead-agent'
tags=['model:gpt-4o-mini']``.
Refs #2930.
* fix(tracing): extend root-callback + metadata injection to the embedded client
Addresses Copilot review on PR #2944.
Commit 2 disabled model-level tracing for ``TitleMiddleware`` and
``_create_summarization_middleware`` because ``_make_lead_agent`` now
attaches the tracing callbacks at the graph invocation root. But the
embedded ``DeerFlowClient`` does not call ``_make_lead_agent`` — it
calls ``_build_middlewares`` directly and never appends the tracing
handlers to its ``RunnableConfig``. So under the embedded path,
title-generation and summarization LLM calls were left untraced —
a regression introduced by this PR.
This commit mirrors the gateway worker's injection in
``DeerFlowClient.stream``:
- Append ``build_tracing_callbacks()`` to ``config["callbacks"]`` so
the Langfuse handler sees ``on_chain_start(parent_run_id=None)`` at
the graph root and runs the ``propagate_attributes`` path.
- Merge ``build_langfuse_trace_metadata(...)`` into
``config["metadata"]`` with ``setdefault`` so caller-supplied keys
still win.
- ``_ensure_agent`` now creates its main model with
``attach_tracing=False`` to avoid duplicate spans now that the
callback lives at the graph root.
Docs:
- ``backend/CLAUDE.md`` Tracing section rewritten to describe the
graph-root attachment model (replacing the inaccurate
"at model-creation time" wording).
- ``README.md`` Langfuse section now lists both injection points
(worker + client) instead of only the worker path.
Tests:
- ``tests/test_client_langfuse_metadata.py`` (new, 3 cases):
callbacks + metadata are injected when Langfuse is enabled,
caller-supplied metadata overrides win via ``setdefault``, and the
injection is inert when Langfuse is disabled.
Live verification on the real Langfuse instance:
=== user=fancy-client ===
id=cbd22847.. session=client-2930-6b9491-fancy-client user=fancy-client name='lead-agent'
=== user=alice-client ===
id=b4f6f576.. session=client-2930-6b9491-alice-client user=alice-client name='lead-agent'
Refs #2930.
* refactor(tracing): address maintainer review on PR #2944
Addresses @WillemJiang's 5 comments.
1. Duplicated metadata-injection code between worker.py and client.py
New ``deerflow.tracing.inject_langfuse_metadata(config, ...)`` helper
takes the 10-line build + merge + setdefault logic that was duplicated
in ``runtime/runs/worker.py`` and ``client.py``. Both callers now share
a single source of truth, so the two paths cannot drift.
2. Direct private-attribute mutation in conftest.py and tests
Added public ``reset_tracing_config()`` / ``reset_title_config()``
functions. ``tests/conftest.py`` and every test that previously did
``tracing_module._tracing_config = None`` or
``title_module._title_config = TitleConfig()`` now goes through the
public API. A future internal rename will surface as an ImportError
instead of a silent no-op.
3. client.py reading os.environ directly
``DeerFlowClient.__init__`` grows an optional ``environment`` parameter
so programmatic callers can pass the deployment label explicitly.
``stream()`` consults ``self._environment`` first and only falls back
to ``DEER_FLOW_ENV`` / ``ENVIRONMENT`` env vars when nothing was
passed in. Backwards compatible — env-var behaviour preserved for
callers that opt to keep using it.
4. build_tracing_callbacks() cached on hot path
Not implemented. Inspected the langfuse v4 ``langchain.CallbackHandler``
constructor: it only resolves the module-level singleton client via
``get_client()`` and initialises a few dicts (no I/O, no env parsing
at construction time). The build is essentially free. Caching would
trade a non-measurable speedup for two real risks: handler instances
carry per-run state internally (``_run_states``, ``_root_run_states``,
``last_trace_id``), and tracing config can be reloaded by env-var
changes between runs. Will revisit if profiling ever shows it as
a hot spot.
5. attach_tracing=False easy to forget at new in-graph call sites
- Module docstring at the top of ``lead_agent/agent.py`` documents
the invariant ("every in-graph ``create_chat_model`` MUST pass
``attach_tracing=False``") and enumerates the current sites.
- New regression test
``test_make_lead_agent_attaches_tracing_callbacks_at_graph_root`` in
``tests/test_lead_agent_model_resolution.py`` locks both halves of
the invariant: ``config["callbacks"]`` carries the tracing handler
after ``_make_lead_agent``, AND every ``create_chat_model`` call
captured by the test passes ``attach_tracing=False``. A future
in-graph site that forgets the flag will fail this test.
Lint clean. Full touched-suite bundle: 246 passed.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* Fix env resolution in MCP config lists
* fix:unset env variable and consistent function
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
When a sandbox container crashes (e.g. due to an internal error), the
agent enters a connection-refused loop because AioSandboxProvider.get()
returns a cached but dead sandbox object. Add a liveness check in get()
that detects crashed containers via backend.is_alive() and evicts them
from all caches, allowing ensure_sandbox_initialized() to transparently
recreate a fresh container on the next acquire().
The behavior is controlled by a new config option
(default: true). Set to false to skip health checks and preserve the
old behavior of returning stale cached sandboxes.
Closes#2788
* Make loop detection configurable
Expose LoopDetectionMiddleware thresholds through config.yaml while preserving existing defaults and allowing the middleware to be disabled.
Refs bytedance/deer-flow#2517
* feat(loop-detection): add per-tool tool_freq_overrides to Phase 1
Adds ToolFreqOverride model and tool_freq_overrides field to
LoopDetectionConfig, wires it through LoopDetectionMiddleware, and
documents the option in config.example.yaml.
Resolves the gap flagged in the #2586 review: without per-tool overrides,
users hit by #2510/#2511 (RNA-seq workflows exceeding the bash hard limit)
had no way to raise thresholds for one tool without loosening the global
limit for every tool.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* docs(loop-detection): document tool_freq_overrides in LoopDetectionMiddleware docstring
Add the missing Args entry for tool_freq_overrides, explaining the
(warn, hard_limit) tuple structure and how per-tool thresholds supersede
the global tool_freq_warn / tool_freq_hard_limit for named tools.
Also run ruff format on the three files flagged by the lint check.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(loop-detection): validate LoopDetectionMiddleware __init__ params eagerly
Raise clear ValueError at construction time instead of crashing at
unpack-time inside _track_and_check when bad values are passed:
- tool_freq_overrides: must be 2-tuples of positive ints with hard_limit >= warn
- scalar thresholds: warn_threshold, hard_limit, tool_freq_warn,
tool_freq_hard_limit must be >= 1 and hard limits must >= their warn pairs
- window_size, max_tracked_threads must be >= 1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(test): isolate credential loader directory-path test from real ~/.claude
The test didn't monkeypatch HOME, so on any machine with real Claude Code
credentials at ~/.claude/.credentials.json the function fell through to
those credentials and the assertion failed. Adding HOME redirect ensures
the default credential path doesn't exist during the test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style(test): add blank lines after import pytest in TestInitValidation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(loop-detection): collapse dual validation to LoopDetectionConfig
Modifications
- LoopDetectionMiddleware.__init__: stripped of all ValueError raises;
becomes a plain field-assignment constructor.
- LoopDetectionMiddleware.from_config: classmethod that builds the
middleware from a Pydantic-validated LoopDetectionConfig and handles
the ToolFreqOverride -> tuple[int, int] conversion.
- agents/factory.py: SDK construction routed through
LoopDetectionMiddleware.from_config(LoopDetectionConfig()) so the
defaults path is Pydantic-validated too.
- agents/lead_agent/agent.py: uses from_config instead of unpacking
config fields by hand.
- tests/test_loop_detection_middleware.py: deleted TestInitValidation
(16 methods exercising the removed __init__ checks); added
TestFromConfig (4 tests: scalar field mapping, override tuple
conversion, empty overrides, behavioral smoke test).
Result: one validation layer (Pydantic), zero duplication, no __new__
hacks. Both production construction sites flow through LoopDetectionConfig.
Test results
make test -> 2977 passed, 18 skipped, 0 failed (137s)
make format -> All checks passed; 411 files left unchanged
* feat(agents): make loop_detection configurable in create_deerflow_agent
Adds a `loop_detection: bool | AgentMiddleware = True` field to
RuntimeFeatures, mirroring the existing pattern used by `sandbox`,
`memory`, and `vision`. SDK users can now disable LoopDetectionMiddleware
or replace it with a custom instance built from their own
LoopDetectionConfig — e.g.
`LoopDetectionMiddleware.from_config(my_cfg)` — instead of being stuck
with the hardcoded defaults previously installed by the SDK factory.
The lead-agent path (which already reads AppConfig.loop_detection) is
unchanged, and the default `True` preserves prior always-on behavior for
all existing callers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: knight0940 <631532668@qq.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Amorend <142649913+knight0940@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(agent): add update_agent tool for in-chat custom-agent self-updates (#2616)
Custom agents had no built-in way to persist updates to their own SOUL.md /
config.yaml from a normal chat — `setup_agent` was only bound during the
bootstrap flow, so when the user asked the agent to refine its description
or personality, the agent would shell out via bash/write_file and the edits
landed in a temporary sandbox/tool workspace instead of
`{base_dir}/agents/{agent_name}/`.
Changes:
- New `update_agent` builtin tool with partial-update semantics (only the
fields you pass are written) and atomic temp-file + os.replace writes so
a failed update never corrupts existing SOUL.md / config.yaml.
- Lead agent now binds `update_agent` in the non-bootstrap path whenever
`agent_name` is set in the runtime context. Default agent (no
agent_name) and bootstrap flow are unchanged.
- New `<self_update>` system-prompt section is injected for custom agents,
instructing them to use `update_agent` — and explicitly NOT bash /
write_file — to persist self-updates.
- Tests: 11 new cases in `tests/test_update_agent_tool.py` covering
validation (missing/invalid agent_name, unknown agent, no fields),
partial updates (soul-only, description-only, skills=[] vs omitted),
no-op detection, atomic-write safety, and AgentConfig round-tripping;
plus 2 new cases in `tests/test_lead_agent_prompt.py` covering the
self-update prompt section.
- Docs: updated backend/CLAUDE.md builtin tools list and tools.mdx
(en/zh) with the new tool description.
* feat(agent): isolate custom agents per user
Store custom agent definitions under the effective user, keep legacy agents readable until migration, and cover API/tool/migration behavior with tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat: consistent write/delete targets & add --user-id to migration
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(harness): restore legacy skills path fallback (#2694)
* fix(format): make format
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* refactor: thread app config through lead prompt
* fix: honor explicit app config across runtime paths
* style: format subagent executor tests
* fix: thread resolved app config and guard subagents-only fallback
Address two PR review findings:
1. _create_summarization_middleware passed the original (possibly None)
app_config into create_chat_model, forcing the model factory back to
ambient get_app_config() and risking config drift between the
middleware's resolved view and the model's view. Pass the resolved
AppConfig instance through end-to-end.
2. get_available_subagent_names accepted Any-typed config and forwarded
it to is_host_bash_allowed, which reads ``.sandbox``. A
SubagentsAppConfig (also accepted upstream as a sum-type input) has
no ``.sandbox`` attribute and would be silently treated as "no
sandbox configured", incorrectly disabling the bash subagent. Guard
on hasattr and fall back to ambient lookup otherwise.
Adds regression tests for both paths.
* chore: simplify hasattr guard and tighten regression tests
- Collapse if/else into ternary in get_available_subagent_names; hasattr(None, ...) is False so the explicit None check was redundant.
- Drop comments that narrate the change rather than explain non-obvious WHY (test names already convey intent).
- Replace stringly-typed sentinel "no-arg" in regression test with direct args tuple comparison.
---------
Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
* fix(harness): resolve runtime paths from project root
* docs(config): update
* fix(config): address runtime path review feedback
* test(config): fix skills path e2e root
* test(config): cover legacy config fallback when project root lacks config files
Verifies that when DEER_FLOW_PROJECT_ROOT is unset and cwd has no
config.yaml/extensions_config.json, AppConfig and ExtensionsConfig fall back
to the legacy backend/repo-root candidates — the backward-compat path
requested in PR #2642 review.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Centralize log level parsing in `logging_level_from_config()` and
application in `apply_logging_level()` within `deerflow.config.app_config`.
- Gateway lifespan applies configured log level on startup
- `debug.py` uses shared helpers instead of local duplicates
- `apply_logging_level()` targets only `deerflow`/`app` logger hierarchies
so third-party library verbosity is not affected; root handler levels
are only lowered (never raised) to allow configured loggers through
without suppressing third-party output; root logger level is not modified
- Config field description updated to clarify scope
- Tests save/restore global logging state to avoid test pollution
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
