94 Commits

Author	SHA1	Message	Date
Daoyuan Li	74392e1470	Fix require_mention gating on whitespace-only bot_login/mention_login (#4055) ... github.bot_login and trigger.mention_login were read raw in the require_mention precedence chain, so a whitespace-only value (e.g. " ") never fell through to the working fallback the chain documents - Python truthiness lets " " win an `or` chain over agent.name or the operator default. The third link (channels.github.default_mention_login) was already correctly normalized and pinned by test_operator_default_blank_string_treated_as_none; this closes the gap for the other two by normalizing GitHubTriggerConfig.mention_login and GitHubAgentConfig.bot_login once, at the config layer, via a field_validator mirroring the pattern already used elsewhere in the config package.	2026-07-12 15:51:22 +08:00
heart-scalpel	3bc3af2530	fix(runs): close multi-worker ownership gaps in run atomicity (#3948) (#4003) ... * feat(runs): cross-process run ownership with lease + reconciliation (#3948) Implements work items 2 and 3 of the multi-worker P0 plan (docs/multi_worker.md). Work item 1 (Postgres startup gate, #3960) already landed; this PR makes run creation race-safe across worker processes and lets Postgres deployments recover orphaned inflight runs from crashed workers without mis-marking live runs as orphans. Work item 2 — cross-process atomic create_or_reject - Alembic revision 0004_run_ownership adds runs.owner_worker_id, runs.lease_expires_at, idx_runs_lease, and a partial unique index uq_runs_thread_active (one pending/running run per thread). The index is declared on RunRow.__table_args__ with sqlite_where + postgresql_where (mirroring uq_channel_connection_active_identity) so the empty-DB bootstrap path — which runs Base.metadata.create_all + alembic stamp head without executing any revision's upgrade() — also lands it on fresh deployments. Migration 0004 additionally creates it idempotently for legacy/versioned upgrades. - RunRepository.create_run_atomic is the new atomic primitive: - reject: INSERT directly; the partial unique index catches duplicate active runs; the manager surfaces the result as ConflictError. - interrupt/rollback: SELECT FOR UPDATE the conflicting rows, skip rows whose lease is still valid AND owned by another live worker (raise ConflictError — the INSERT would have failed on the index anyway, and a retry loop cannot make progress), cancel the rest in the same transaction, then INSERT the new row. Rows owned by this worker are interruptible regardless of lease state. - RunManager.create_or_reject dispatches to the store under the existing local lock; same-worker in-memory cancellation runs after the store commit succeeds. MemoryRunStore mirrors the same semantics for tests and database.backend=memory. Work item 3 — lease heartbeat + Postgres reconciliation - RunOwnershipConfig (lease_seconds=30, grace_seconds=10, heartbeat_enabled=false by default), registered as startup-only in reload_boundary.STARTUP_ONLY_FIELDS because the heartbeat background task is created once in langgraph_runtime() and is not rebuilt on config.yaml edits. - When heartbeat_enabled, each worker renewes leases on its own active runs with interval = lease_seconds / 3. The loop is bounded and stop-event-cancellable so shutdown is prompt. - reconcile_orphaned_inflight_runs now runs on every backend — the sqlite-only gate in app/gateway/deps.py is dropped in the same commit so there is no window where Postgres would mis-mark live Worker A runs as orphans. Reconciliation errors only runs whose lease is NULL (legacy pre-ownership rows) or older than grace_seconds. In single-worker mode (heartbeat off, NULL leases) all inflight rows reclaim immediately, preserving the pre-ownership recovery latency. - Heartbeat starts AFTER startup reconciliation and stops BEFORE the in-flight run drain on shutdown so the two cannot race. GATEWAY_WORKERS=1 with heartbeat_enabled=false keeps current behavior. Verified: 170 related tests + full backend suite (minus Docker-gated live tests) green; ruff check + ruff format clean. * fix(runs): tighten unique-violation handling and document clock-sync budget Three follow-up fixes to the cross-process run ownership work in #3948, surfacing during review. 1. _is_unique_violation: detect by driver-native signal, not message text The previous substring heuristic ("unique" + "violat", or "duplicate") missed SQLite's actual phrasing "UNIQUE constraint failed: <table>.<index>" — SQLite says "failed", not "violates", and never "duplicate". On SQLite the detector returned False, the reject path re-raised the raw IntegrityError, and clients saw HTTP 500 instead of ConflictError 409. The conversion is the load-bearing piece of the "store is source of truth" design but was untested — every atomic test used MemoryRunStore, which raises ConflictError directly and never reached this branch. Now prefers driver-native signals: psycopg pgcode/sqlcode "23505" and sqlite3 sqlite_errorcode SQLITE_CONSTRAINT_UNIQUE (reachable through SQLAlchemy IntegrityError.orig). Message matching stays as a fallback with SQLite's exact "unique constraint failed" phrase added. 2. interrupt/rollback: convert exhausted-retry IntegrityError to ConflictError The reject branch converts unique violations to ConflictError. The interrupt/rollback retry loop did not — on the 3rd attempt it re-raised the raw IntegrityError, leaking HTTP 500 for the same race condition that reject surfaces as 409. Symmetric conversion added after the loop; callers now see a consistent ConflictError regardless of strategy. 3. Document clock-sync requirement for multi-worker lease reconciliation reconcile_orphaned_inflight_runs compares another worker's UTC lease_expires_at against this worker's datetime.now(UTC). The only skew budget is grace_seconds (default 10s) — worst case, with the owning worker's heartbeat just about to fire, a peer whose clock is more than ~grace_seconds ahead can mis-reclaim a still-live run as an orphan. Documented in RunOwnershipConfig's docstring (with the math) and in config.example.yaml (with operational guidance), so operators in NTP-poor environments know to raise grace_seconds. Default unchanged: 10s is reasonable for NTP-synced K8s/cloud, and bumping it would slow recovery of genuinely dead workers (lease_seconds + grace_seconds from last heartbeat to reclaim). Tests: - test_create_run_atomic_reject_propagates_conflict_on_unique_violation: end-to-end against a real SQLite-backed RunRepository, pre-inserts an active run, asserts reject-strategy create surfaces as ConflictError rather than raw IntegrityError. - test_is_unique_violation_detects_real_sqlite_integrity_error: unit test for the detector against a real SQLite-raised IntegrityError; asserts driver-level sqlite_errorcode is SQLITE_CONSTRAINT_UNIQUE. - test_interrupt_exhausted_retries_surface_as_conflict_error: pins the symmetric 409 behavior after the retry loop exhausts. Verified: ruff check + ruff format clean; multi-worker + run_repository + owner_isolation + reload_boundary suites green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(runs): close multi-worker ownership gaps in lease heartbeat and unique-violation detection Five code-review fixes from docs/multi_worker.md: 1. Drop unused ``claim_inflight_runs`` primitive — no caller anywhere. ``create_run_atomic`` does its own inline claim (SELECT FOR UPDATE + cancel) inside the INSERT transaction; a separate claim primitive would split that into two transactions and open a claim→INSERT race. Removes ~40 lines across base.py / memory.py / sql.py plus the unused ``now_iso`` parameter, freeing future RunStore implementations from providing it. 2. Broaden ``_renew_leases`` filter to renew pending/running runs owned by this worker even when ``record.task is None``. The previous ``task is not None`` requirement skipped the brief window between ``create_run_atomic`` inserting the row and the worker spawning the agent task; under event-loop load that window can approach ``lease_seconds``, after which peer reconciliation marks the run ``error`` (visible) or a peer's ``create_or_reject("interrupt")`` silently kills the queued run. Filter now: ``task is None or not task.done()``. 3. Document the unsynchronised ``record.lease_expires_at = new_expiry`` write. ``lease_expires_at`` is the only field on an existing record this path mutates; ``set_status`` / ``_persist_status`` touch other fields, so there is no concurrent writer to race against. Re-acquiring ``self._lock`` would serialise unrelated run mutations for no gain. 4. Gate ``_is_unique_violation`` message fallbacks on ``isinstance(current, (SAIntegrityError, sqlite3.IntegrityError))``. The driver-code path (pgcode/sqlite_errorcode) remains load-bearing; substring fallbacks are now belt-and-suspenders only for cases where the driver attribute isn't reachable through the cause chain. Without the gate, any application exception whose ``str()`` happens to contain "duplicate key" / "unique" + "violat" (CHECK constraint, validation error) would silently surface as HTTP 409 instead of 500. 5. Route ``update_lease`` through ``_call_store_with_retry`` for consistency with every other store call, and wrap ``await self._renew_leases()`` in ``_heartbeat_loop`` with ``except Exception: logger.warning(...)``. Previously a transient error from the snapshot path or an unexpected exception would kill the heartbeat task silently — after which no lease is ever renewed again and every active run eventually looks orphaned. ``except Exception`` lets ``CancelledError`` (BaseException since 3.8) propagate so shutdown cancellation still works. Regression tests: - ``test_heartbeat_renews_pending_run_before_task_is_spawned`` - ``test_is_unique_violation_does_not_misclassify_application_exception`` Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(runs): harden multi-worker migration, memory atomicity, and tz-naive lease comparison Three follow-up fixes to the multi-worker run ownership work: - migration 0004 dedupe pass: cancel superseded duplicate active rows per thread before creating the partial UNIQUE index ``uq_runs_thread_active`` so dirty DBs (Postgres deployments that had reconciliation skipped by the old sqlite-only gate, or any env that ran GATEWAY_WORKERS>1 before this PR) do not abort the alembic upgrade and block gateway startup. Keeps the newest active row per thread, marks the rest as error with an explanatory message. - MemoryRunStore.create_run_atomic interrupt/rollback path: split the single- pass loop into two passes (collect candidates, validate, then mutate) so a ConflictError raised on a later candidate does not leave earlier candidates half-interrupted. Mirrors the SQL store's transactional rollback semantics; the entire test_multi_worker_run_ownership.py suite runs against memory so this divergence was giving false confidence. - RunRepository.create_run_atomic interrupt path: coerce tz-naive ``row.lease_expires_at`` to UTC before comparing against the aware ``cutoff``. SQLite drops tzinfo on read despite ``DateTime(timezone=True)`` (this file's own comment acknowledges it), so the Python-side comparison raised ``TypeError: can't compare offset-naive and offset-aware datetimes`` whenever heartbeat was enabled on SQLite and a lease was non-NULL. Defaults (heartbeat off -> leases always NULL) masked it, but there was no guard against the combination. Follows the existing "naive is UTC" convention from ``coerce_iso``. Each fix ships with a regression test pinning the behavior. Co-Authored-By: heart-scalpel <heart-scalpel@users.noreply.github.com> Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(runs): enforce heartbeat for multi-worker, fix memory-store datetime comparison, lazy-import ConflictError in store layer Three fixes from code review: 1. Extend the startup gate (GATEWAY_WORKERS>1) to also require run_ownership.heartbeat_enabled=true. Without heartbeat every run has a NULL lease, so reconciliation treats all inflight rows as orphans and Worker B would kill Worker A's live runs on every rolling update or scale-up. 2. Fix MemoryRunStore.list_inflight_with_expired_lease to parse created_at as datetime instead of ISO string lexical comparison, and handle tz-naive lease values uniformly with the SQL store. 3. Store layer (sql.py, memory.py) now lazy-imports ConflictError inside create_run_atomic instead of importing from the higher RunManager layer at module level. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(runs): add owner check to update_lease, document create() assumption, restore deleted comment - update_lease (SQL + memory) now requires owner_worker_id match in WHERE clause so the primitive is safe by construction against misuse - create() docstring notes it bypasses atomic create_run_atomic and assumes no active run exists for the thread - restore explanatory comment in MemoryRunStore.aggregate_tokens_by_thread that was dropped in an earlier commit Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(runs): add psycopg3 sqlstate detection and periodic orphan reconciliation - _is_unique_violation now checks sqlstate attribute (psycopg3 uses this instead of pgcode). On Postgres, the only supported multi-worker backend, detection was falling through to the message-substring fallback. - _heartbeat_loop now runs reconcile_orphaned_inflight_runs every 3rd cycle (every lease_seconds) to catch orphans whose lease expires between pod restarts. Single-worker deployments are unaffected (heartbeat off). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: heart-scalpel <heart-scalpel@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-11 16:05:30 +08:00
Vanzeren	c2002d9fac	feat(memory): add memory tool sets (#4023) ... * feat: add memory-as-tool mode alongside existing middleware mode - Add memory.mode config field (middleware|tool, default middleware) - Add search_memory_facts() for case-insensitive fact lookup - Add 4 memory tools: memory_search, memory_add, memory_update, memory_delete - Wire mode gating in factory.py and lead_agent/agent.py - 256 memory tests passing, zero regressions * fix: harden tool-mode memory scoping and docs * fix: address memory tool mode review feedback * fix(memory): address tool mode review feedback * fix: update config_version to 22 in values.yaml	2026-07-11 15:28:16 +08:00
Tianye Song	9097642658	feat(memory): add memory consolidation to synthesize fragmented facts (#3996) ... * feat(memory): add memory consolidation to synthesize fragmented facts When a fact category accumulates many individual entries, the LLM reviews them during the normal memory-update call (same invocation, no extra API cost) and decides whether groups of related facts can be synthesized into a single richer fact. This completes the memory lifecycle: extraction → guaranteed injection → staleness review → consolidation. - Select fragmented categories by min-facts threshold, surface the most fragmented groups first; prompt-layer caps aligned with apply-layer guardrails so the LLM never sees groups it cannot act on - Cap consolidated confidence at source maximum to prevent inflation; reject results below fact_confidence_threshold - Double-consume protection prevents a fact from being merged into multiple consolidation targets - Feature-gated at both prompt and apply time with per-cycle safety caps - Add 26 tests covering candidate selection, normalization, apply guardrails, and prompt integration * fix(memory): address consolidation correctness issues from PR review Six fixes based on maintainer review of #3996: 1. Deduplicate sourceIds in normalization — ["f1","f1"] previously bypassed the ≥2-distinct-sources check; dict.fromkeys collapses it to ["f1"] which is correctly rejected. 2. Run consolidation after max_facts trim — previously, sources were deleted then the merged fact could be evicted by the trim, leaving no record of either. Moving consolidation last ensures source facts exist in the post-trim index before removal. 3. Fix count= attribute in consolidation prompt — advertised the full category size but listed only max_sources IDs; now uses min(len(group), max_sources) to match what the LLM can act on. 4. Exempt staleness_protected_categories from consolidation candidates — mirrors the existing staleness-review contract so correction facts are never surfaced for merging. 5. Strip and default category in consolidation normalization — " " or " preference " are now normalised, matching _normalize_memory_update_fact. 6. Propagate sourceError from source facts into consolidated fact — correction context is no longer silently lost on merge. * fix(memory): add apply-time guardrails and tests for consolidation P1: mirror the staleness-pass defense-in-depth pattern — build allowed_source_ids from _select_consolidation_candidates at apply time so a protected-category or below-threshold fact proposed by the LLM is rejected regardless of model behavior. P2a: test that LLM-returned confidence is capped at max source confidence and that a capped result below fact_confidence_threshold is rejected. P2b: test that factsToConsolidate with consolidation_enabled=False is a no-op at apply time (35 tests, all pass). * fix(memory): address three correctness issues from second review round 1. Default consolidation_enabled=False — consolidation is lossy (source content is permanently replaced, only consolidatedFrom IDs preserved); new lossy features default to off. config.example.yaml updated to match. 2. Unify confidence coercion between prompt and apply — _build_consolidation_section now calls _coerce_source_confidence(fact) instead of an inline 0.0-default coercion, so a null-confidence fact renders with 0.50 in the LLM prompt and is capped at 0.50 at apply time (same value, same function). 3. Preserve staleness clock on merge — consolidated fact now carries the newest source's createdAt (not now) so aged information does not gain a fresh staleness-review window just by being consolidated; consolidatedAt is added as an explicit audit field. Three regression tests added (default=false, null-confidence consistency, createdAt policy); all guardrail tests now set consolidation_enabled=True explicitly so they test the guardrail, not the feature flag. 38 tests pass. * fix(memory): harden createdAt comparison and confidence handling 1. createdAt max via _parse_fact_datetime — replaces string max() which crashes on non-string createdAt (numeric unix timestamps) and sorts Z/+00:00 mixed formats incorrectly. Mirrors how staleness computes age. 2. Remove dead min(..., 1.0) — _coerce_source_confidence already clamps each source confidence to [0, 1], so max(source_confidences) ≤ 1.0 by contract; the outer min could never bind. 3. Clamp raw_llm_conf to [0, 1] before applying the source cap — out-of- range values like 1.5 are safe today (pinned by the cap) but defensively clamped first so the invariant holds even if the cap is ever loosened. 4. Doc: expand the apply-time guardrails comment to call out the protected- category exclusion via allowed_source_ids — this is the central safety property ("explicit user feedback is never silently merged away"). 5. Test: add test_confidence_fallback_to_max_source_when_llm_omits_field covering the else-branch (LLM omits confidence → uses max_source_conf). 6. Fix lint: reorder imports in test file (stdlib before third-party). 39 tests, all pass. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-10 11:31:08 +08:00
hataa	266883b3dd	fix(subagents): inherit summarization middleware and harden step capture (#3875 Phase 3) (#4009) ... Phase 3 of #3875 — subagents previously inherited none of the lead's context-compaction, so a deep-research subagent (max_turns up to 150) could accumulate >1M cumulative input before max_turns/timeout/token_budget engaged, even after Phase 2's budget capped the pathological tail. - Gate the subagent runtime chain on the SAME ``app_config.summarization.enabled`` switch the lead reads (per maintainer guidance in #3875), via the shared ``create_summarization_middleware`` factory. One config covers both chains; no separate ``subagents.summarization`` field. No-op when summarization is off (factory returns None). - ``skip_memory_flush=True`` on the subagent path: the factory otherwise attaches ``memory_flush_hook`` (when memory.enabled), which flushes pre-compaction messages into durable memory keyed by thread_id. Subagents share the parent's thread_id, so without skipping the hook a subagent's internal turns would pollute the PARENT thread's durable memory (#3875 Phase 3 review point). - Harden ``capture_new_step_messages`` to tolerate history contraction: summarization rewrites the messages channel via ``RemoveMessage(id=REMOVE_ALL_MESSAGES)``, shrinking len(messages) below the step-capture cursor. Without a reset, every step appended after the compaction point was dropped until length overtook the stale cursor (#3845 interaction, maintainer validation point (a)). Cursor now resets to the new tail; id/content dedup prevents re-emitting pre-compaction steps. - Couple the DEFAULT token-budget ceiling to ``summarization.enabled`` (#3875 Phase 3 review point): 1M when compaction is on, 2M when off (preserves Phase 2's deliberate headroom for summarization-off deep-research runs that can exceed 1M). A user-set budget (global or per-agent) always wins regardless of the switch. Flagged tunable. The summarization middleware does not implement ``consume_stop_reason``, so the Phase 2 guard-cap stop-reason channel is unaffected. Refs: https://github.com/bytedance/deer-flow/issues/3875	2026-07-10 11:17:35 +08:00
Ryker_Feng	ebc09ce130	feat(mcp): auto-promote deferred MCP tools from routing hints (#4019) ... * feat(mcp): auto-promote deferred MCP tools from routing hints When tool_search.enabled=true defers MCP tool schemas, PR1 routing hints still require the model to spend a tool_search discovery round trip before it can call the tool the routing metadata already points at. This adds a McpRoutingMiddleware that matches the latest user message against PR1 routing keywords and promotes the matching deferred schemas before the model call, removing that round trip. Design (soft routing, opt-in, additive): - Matches only the latest real HumanMessage (shared is_real_user_message helper, reused by SkillActivationMiddleware so the two cannot drift); case-insensitive substring match, no tokenizer dependency. - Ordering: priority desc, then tool name asc; capped by the new global tool_search.auto_promote_top_k (default 3, clamped 1..5). Does not add or consume a per-tool auto_promote_top_k (PR1 schema unchanged); a per-tool value is ignored with a DEBUG note. - Returns a plain {"promoted": ...} state update (not a Command) and relies on ThreadState.merge_promoted for union/dedupe, so auto-promote and a model-triggered tool_search converge on the same catalog hash. - Installed before DeferredToolFilterMiddleware on every deferred-tool path (lead agent, subagent, embedded client, webhook via shared builders); a construction-time assert rejects the reversed order. catalog_hash is None / no routing index is a complete no-op, so bootstrap and ACP skip it. - Privacy: never executes tools, never promotes policy-filtered tools, adds no routing keywords or matched tool names to trace metadata or INFO/WARN logs. No behavior change when tool_search.enabled=false. Tests: index construction, matching semantics, middleware state updates, same-cycle deferred-filter interaction, lead/subagent/embedded-client builder wiring + order invariant, config clamping, config.example.yaml parseability, and privacy assertions. * refactor(mcp): address auto-promote review nits - executor: access app_config.tool_search.auto_promote_top_k directly to match the lead-agent and embedded-client paths (drop the over-defensive getattr that masked missing config); update the subagent test mock to carry tool_search. - tool_search / mcp_routing_middleware: cross-reference the duplicated routing priority/keyword normalization between the builder and the middleware's defensive _normalize_index so they cannot silently drift. - MCP_SERVER.md: document that auto-promote keyword matching is a case-insensitive substring test (not word-boundary), advising distinctive keywords.	2026-07-10 07:54:36 +08:00
Vanzeren	dc38d2d003	perf(boxlite): add benchmark-driven warm-pool reclaim tuning (#4001) ... * bench: add provider-agnostic sandbox benchmark with BoxLite warm pool results - scripts/bench_sandbox_provider.py: CLI for measuring acquire/run/release across providers, scenarios, workloads, and concurrency levels - scripts/summarize_bench.py: JSONL aggregation with p50/p95/p99 tables - bench_results.jsonl: 110 turns across 7 scenarios on real BoxLite 0.9.7 Key findings: cold acquire: ~860ms warm reclaim: ~14ms (60x speedup) release: ~0ms warm_hit_rate: 95% (warm_same_thread) * perf(boxlite): skip health check for recently-released warm pool boxes Boxes released within health_check_skip_seconds (default 5.0s) are promoted directly without the ~14ms echo-ok round-trip. A VM alive seconds ago is overwhelmingly likely to still be alive. Add sandbox.health_check_skip_seconds config option. Set to 0 to always health-check (old behaviour). Benchmark (warm_same_thread, noop, 20 iters): acquire p50: 14.9ms → 0.0ms total p50: 29.9ms → 14.0ms * chore: move benchmark scripts into backend/scripts/benchmark/ * fix: address BoxLite benchmark review findings * fix(boxlite): only skip warm reclaim checks for released boxes * fix(benchmark): keep BoxLite shim workaround off the event loop * fix(boxlite): invalidate dead boxes from command path * test(boxlite): cover skip window and invalidation edge cases * fix(boxlite): treat sandbox-has-been-closed as terminal in _exec * fix(boxlite): harden warm-pool reclaim and benchmark accounting * fix(boxlite): validate warm-pool reclaims by default * fix(config): expose boxlite health-check skip setting * fix(boxlite): tighten failure classification and benchmark workaround * Update config_version to 21 in values.yaml --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-09 17:21:25 +08:00
Ryker_Feng	5ba25b06ec	feat(mcp): add MCP routing hints (#4004) ... * feat: add MCP routing hints * test: isolate mcp routing prompt config * fix: address mcp routing review feedback	2026-07-09 16:26:31 +08:00
hataa	c9fb9768d4	fix(subagents): unify guardrail caps on additive stop_reason + add token_budget (#3875 Phase 2) (#3980) ... Phase 2 of #3875. Two guardrail axes can end a subagent run early — the turn budget (GraphRecursionError) and the token budget (TokenBudgetMiddleware) — and both now surface *why* through one additive `subagent_stop_reason` field instead of a status enum. This completes and course-corrects Phase 1 (#3949), which shipped the turn-budget cap as a `max_turns_reached` status enum. The agreed Phase 2 design replaces that enum with an optional `stop_reason` field (token_capped | turn_capped | loop_capped): a new enum value would break v1 consumers, while an additive field is ignored by older frontends and ledger readers. `max_turns_reached` and SubagentStatus.MAX_TURNS_REACHED are removed. - subagents.token_budget config (default enabled, 2,000,000 tokens, warn 0.7) with per-agent override; TokenBudgetMiddleware is now attached in build_subagent_runtime_middlewares so the cost-ceiling backstop engages for every subagent. The hard-stop does not raise — it strips tool_calls and lets the run finish with a final answer, recording the cap on a per-run consume_stop_reason() accessor. - executor.py: on normal completion it reads consume_stop_reason() and stamps completed + token_capped when the budget fired; on GraphRecursionError it recovers the last AIMessage partial (completed + turn_capped) or, if nothing usable survived, failed + turn_capped. SubagentResult gains stop_reason. - status_contract.py / contracts/subagent_status_contract.json (v2) / frontend subtask-result.ts: additive subagent_stop_reason field, pinned by test_status_values_match_contract / test_stop_reason_values_match_contract. - task_tool.py + delegation_ledger.py: drop the max_turns_reached paths; the ledger captures stop_reason and renders model-facing "capped" guidance so the lead reuses a capped completion knowingly. The 2,000,000-token default is deliberately loose (tighten to taste) — it would have roughly halved the reported 4.4M burn while leaving legitimate deep-research runs (max_turns=150) room. Subagent summarization is a follow-up.	2026-07-08 22:26:06 +08:00
Ryker_Feng	01dc067997	feat: add composer input polishing (#3986) ... * feat: add composer input polishing * Revert "Merge branch 'main' into feat/input-polish" This reverts commit 5b6ceccf0db3092bc62fde3b05e7816829601756, reversing changes made to 45fbc57fef5fa5fd878cf0176c37f3e3bc7ebef6. * Merge main into feat/input-polish * style(frontend): format input helper polish guard * fix(input-polish): address composer polish review findings Frontend - Add a cancel affordance to the in-flight polish status pill that calls abortInputPolishRequest(), so a slow/hung provider no longer hard-locks the composer for up to stream_chunk_timeout with a page reload (and draft loss) as the only escape. - Reset promptHistoryIndexRef/promptHistoryDraftRef when a rewrite is applied (and on undo), so a stale history-browse index can no longer let the next ArrowDown silently overwrite the polished draft. - Disable polishing while an open human-input card is present, matching the frontend/AGENTS.md rule that composer entry points defer to the card so card-reply metadata is preserved. - canPolishInput now reuses parseGoalCommand/parseCompactCommand instead of a third hardcoded reserved-command regex, and drops the phantom /help entry (no /help parser exists in the composer), so future builtins only need to be taught to the existing parsers. Backend - Extract the non-graph one-shot LLM path (build model + inject Langfuse metadata + system/user invoke + text extract) into deerflow.utils.oneshot_llm.run_oneshot_llm, shared by the input-polish and suggestions routers so tracing-metadata and invocation shape cannot drift between the two copies. - strip_think_blocks gains truncate_unclosed (default True, preserving the suggestions/goal JSON-prep behavior); input polish passes False so a draft that legitimately contains a literal <think> substring is no longer truncated into a partial rewrite or a spurious 503. - Validate the empty-check and max_chars boundary against the same stripped view of the draft that is sent to the model, so the user-facing length boundary and the model input can no longer disagree. Tests / docs - Backend: literal-<think> preservation, whitespace-only rejection, and normalized-length/model-input agreement cases; suggestions tests repoint the create_chat_model patch to the shared helper module. - Frontend: helper unit tests updated for the /help/reserved-command change; a new Playwright case covers cancelling an in-flight polish request. - backend/AGENTS.md documents the shared one-shot helper and the polish normalization/think-tag behavior. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-08 17:10:27 +08:00
AochenShen99	658c39ccf7	feat(skills): Add native SkillScan phase 1 for skills (#3033) ... * Add phase 1 skill static scanning * Rework SkillScan phase 1 as native scanner * refactor(skillscan): align phase 1 with trimmed RFC contract - SecurityFinding: 7 fields (rule_id, severity, file, line, message, remediation, evidence); category/analyzer derive from the rule_id prefix, confidence/column/fingerprint/metadata removed - scan_archive_preflight()/scan_skill_dir() are pure functions: no ScanContext, no policy schema; CRITICAL-blocks is a code constant and skill_scan.enabled is applied by enforce_static_scan()/callers - secret-* evidence is redacted before findings leave the scanner - de-dup keys on (rule_id, file, line) so repeated occurrences keep distinct locations for agent self-correction - cloud-metadata detection consolidated into network-cloud-metadata - nested zip members get a one-level stdlib magic-byte peek; an executable member escalates package-nested-archive to CRITICAL - install metadata sidecar removed (Phase 7 decides if it is needed) - rule specs moved next to their analyzers; skillscan/rules/ removed - tests updated + new anchors: redaction, dedup lines, nested-zip escalation, single cloud-metadata rule, bundled-skill zero-CRITICAL * fix(skillscan): tighten reverse-shell/secret/archive scan rules from review Address PR #3033 review feedback on the native SkillScan analyzers: - Reverse-shell false positives: split shell detection by signal strength (/dev/tcp/, nc -e stay CRITICAL; bash -i, mkfifo -> new HIGH shell-reverse-shell-heuristic, warn->LLM). The Python check is now AST-anchored on real socket.socket/os.dup2/subprocess call sites instead of raw-text substring matching, so prose/docstrings no longer hard-block. - Secret evidence: _redact_secret_evidence returns [redacted] with no secret bytes (was value[:6], which leaked 2 real token bytes past the prefix). - Archive DoS: cap outer archive member count (_MAX_ARCHIVE_MEMBERS=4096); scan_archive_preflight early-aborts with a package-too-many-members CRITICAL finding (routes through the existing blocked->400 fail-closed path). - shell-destructive-command: broaden the rm -rf matcher to sensitive system roots (/home, /usr, /*, --no-preserve-root /) while leaving safe subpaths unflagged. - Dead code: collapse _decode_text_for_analysis to a single decode path and drop the unused _TEXT_SUFFIXES set and _has_text_shebang helper. - local_skill_storage: document why the host_path branch keeps app_config possibly-None (lazy kill-switch resolution; avoids eager get_app_config in config-free environments such as CI). Tests: new negative/positive coverage in test_skillscan_native.py. Full backend suite 6616 passed, 26 skipped.	2026-07-07 21:44:28 +08:00
He Wang	fd41fdb065	feat(middleware): add structured tool result meta and tool-progress state machine (#3601) ... * feat(middleware): add structured tool result meta and tool-progress state machine feat: - Add tool_result_meta.py: ToolResultMeta dataclass (status/error_type/retryable/ recoverable_by_model/recommended_next_action/source) + normalize_tool_result and stamp_exception_meta utilities; classifies every ToolMessage regardless of path - Add ToolProgressMiddleware: per-(thread_id, tool_name) state machine ACTIVE → WARNED (hint injected as HumanMessage) → BLOCKED (call short-circuited); Jaccard near-duplicate detection for repeated successful results; auth/config/internal errors bypass WARNED and go directly to BLOCKED; LRU-bounded thread state store - Add ToolProgressConfig: all thresholds configurable (stagnation_threshold, warn_escalation_count, jaccard_similarity_threshold, exempt_tools, etc.); disabled by default (enabled: false) - Wire ToolProgressMiddleware as outer wrapper around ToolErrorHandlingMiddleware in _build_runtime_middlewares so it receives results already carrying deerflow_tool_meta fix: - ToolErrorHandlingMiddleware now calls stamp_exception_meta on exception path and normalize_tool_result on success path so every ToolMessage carries deerflow_tool_meta test: - Add test_tool_result_meta.py: 26 cases covering all classification paths, stamp_exception_meta, and normalize_tool_result Command passthrough - Add test_tool_progress_middleware.py: 27 cases including full async paths, Jaccard duplicate detection, LRU eviction, hint injection, and malformed meta passthrough - Extend test_tool_error_handling_middleware.py: middleware ordering invariant and meta stamping on exception docs: - Add tool_progress section to config.example.yaml with all fields and descriptions - Update CLAUDE.md middleware chain documentation (entries 8-9) Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * fix(middleware): recoverable errors stay WARNED; fix auth keyword shadowing fix: - WARNED is terminal for recoverable_by_model=True errors (no_results, not_found, permission); hint re-injected on each problem call instead of escalating to BLOCKED, so the model can retry with different parameters (e.g. fresh query, new URL) without being hard-blocked by a prior stagnation count. Non-recoverable (rate_limited, transient) still escalate WARNED → BLOCKED after warn_escalation_count more problems; auth/config/internal remain immediately BLOCKED. - Remove bare "api key" keyword from auth classification rule so "no api key configured" correctly classifies as config (not auth), producing the accurate block-reason text for the model. docs: - CLAUDE.md: document all three ToolProgressMiddleware transition paths - config.example.yaml: update inline state-machine comment to match new paths test: - test_recoverable_errors_stay_warned_indefinitely: WARNED never escalates for recoverable errors regardless of how many problem calls accumulate - test_recoverable_error_re_injects_hint_past_escalation: hints continue past the escalation zone for recoverable errors - test_no_api_key_is_config_not_auth: regression guard for keyword shadowing fix Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * fix(tool_result_meta): add JSON error extraction and fix source classification fix: - Fix non-standard error path: source was "exception" but should be "tool_return" - Add _extract_json_error_text to isolate JSON error fields from noisy JSON bodies (e.g. Brave Search {"error": "...", "query": "..."} — query keywords no longer pollute error classification) - Add success-path JSON extraction to catch tools that return HTTP 200 with a JSON error body (status="success" but {"error": "API key not configured"}) - Add _SEMANTIC_ZERO_ERROR_STRINGS frozenset to suppress false positives from tools that use {"error": "none"} / {"error": "null"} / {"error": "ok"} as success signals - Document that stamp_exception_meta always overwrites existing TOOL_META_KEY (exception-derived classification is authoritative over tool return-time stamps) test: - Add parametrized regression tests for all semantic-zero error strings - Add tests for non-standard error path source field - Add tests for JSON error extraction (nonstd, success-path, numeric, falsy values) - Correct test comment for test_no_api_key_is_config_not_auth Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * fix(tool_progress_middleware): fix 6 bugs, add terminal guard and structured logging fix: - H1: fix exempt_tools empty-set silently ignored — use `is not None` instead of truthiness check so ToolProgressConfig(exempt_tools=set()) correctly disables all exemptions - Fix _get_block_reason creating phantom LRU entries via _get_state (write path); now uses dict.get + explicit move_to_end on read path only - Fix _pending memory leak: LRU eviction of _phase_states now synchronously removes all (evicted_thread, *) keys from _pending - Fix _assess_and_transition missing terminal guard for blocked state — a recoverable error result could silently demote blocked → warned in concurrent-race scenarios; early return preserves terminal semantics - Fix recent_word_sets window: stored [-5:] but is_near_duplicate only compared [-3:]; align to [-3:] and change type list→tuple (prevents accidental in-place mutation across dataclasses.replace shallow copies) - Fix _format_hint missing "success" key and "continue" action: Jaccard near-duplicate results produced the generic fallback instead of a specific actionable message feat: - Add structured state-transition logging (ACTIVE/WARNED/BLOCKED transitions, blocked intercepts, hint injection debug log) test: - Add regression tests for all 6 bug fixes (H1, phantom LRU, pending leak, terminal guard, window alignment, format_hint near-dup) - Add Jaccard near-threshold boundary test (7/9 vs 8/9 Jaccard) - Add production min_words=10 skip test for short content - Add exempt_tools empty-set and None round-trip tests - Add _augment_request deduplication test - Add before_agent current-run preservation test - Add structured logging tests (WARNED/BLOCKED/ACTIVE/intercepted/debug) Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * chore(config): remove unused backward-compat fields from ToolProgressConfig Remove max_calls_per_intent and window_size fields that were marked "Retained for backward compatibility; not used by the current state machine" when the state machine was introduced. Pydantic v2 ignores unknown fields by default, so existing config.yaml files with these keys remain valid. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * fix(tool_progress): address PR review and multi-agent review findings fix: - Remove <80-char length gate for partial_success; only _PARTIAL_MARKERS now - Add word-boundary regex for numeric HTTP codes (401/403/404/500) to avoid false positives like "500ms" or "4010 rows" triggering hard-block - Add "task" to default exempt_tools (delegation primitive, not a search tool) - Remove move_to_end() from _get_block_reason read path; blocked threads were permanently warm in LRU, starving active threads of eviction slots - Add _reset_blocked_states in before_agent: scope BLOCKED and WARNED states to a single run; clear recent_word_sets so stale Jaccard windows don't cause false near-duplicate detections in the next run - Compute word_set() lazily (only for success results); cap content at 8192 chars to bound memory and CPU cost on large tool results - Remove unused retryable field from ToolResultMeta (no consumer existed) - Add isinstance-based ordering guard and warning log for missing meta - Fix JSON-without-error-key fallback: use _UNKNOWN_ERROR instead of classifying incidental field values (e.g. {"user_id": 401} → auth → stop) - Fix _extract_json_error_text: use json.dumps for dict/list error fields instead of str() which produced Python repr matching config rules spuriously - Add "no results found"/"no content found"/"no images found" to _PARTIAL_MARKERS so success responses with empty results trigger stagnation detection - Fix immediate-block path to increment consecutive_problems (was left at 0) - Fix _queue_assessment: skip phantom _pending entries for evicted threads - Bump config_version 13→16 (upstream added 14/15; tool_progress is additive) test: - Update test_short_content_is_partial → test_short_terse_success_is_not_partial - Add parametrized test_numeric_keyword_word_boundary (8 positive + negative cases) - Add test_before_agent_resets_blocked_states_for_new_run (strengthened assertions) - Add test_before_agent_resets_warned_states_for_new_run - Add test_missing_meta_on_non_exempt_tool_emits_warning - Add test_middleware_ordering_guard_raises_when_progress_is_inner - Add test_auth_error_immediately_blocked asserts consecutive_problems == 1 - Add tests for JSON-without-error-key, dict error field, no-results partial_success Co-Authored-By: Claude <noreply@anthropic.com> * fix(tool_progress): address second PR review — perf, architecture doc, concurrency note fix: - Extract content.lower() once before _PARTIAL_MARKERS check in normalize_tool_message; previously computed up to 7× per call inside the generator (once per marker) docs: - Add division-of-labor paragraph to ToolProgressMiddleware module docstring explaining coexistence with LoopDetectionMiddleware: result-quality guard (per-tool BLOCK) vs call-pattern guard (whole-turn hard-stop); no shared state, no double-stop risk - Add threading.Lock comment explaining why asyncio.Lock is not used (short critical sections, must also protect sync wrap_tool_call path from subagent executor threads) - Update backend/CLAUDE.md entry 8 with division-of-labor summary; fix entry 9 (remove stale retryable field reference, add missing recoverable_by_model/source) test: - Add test_tool_progress_and_loop_detection_coexist_without_interfering: drives both middlewares to WARNED state simultaneously, verifies independent state, independent hint queues, and no cross-contamination; uses snapshot copy for final assertion Co-Authored-By: Claude <noreply@anthropic.com> * fix(tool_progress): reset all tool states at run boundary; fix semantic-zero test validity fix: - _reset_run_states (formerly _reset_blocked_states) drops the phase filter and resets all tracked (thread, tool) pairs unconditionally at before_agent; ACTIVE tools with sub-threshold consecutive_problems or cached recent_word_sets no longer bleed into the next run, preventing spurious WARNED transitions on clean R2 calls - test_normalize_json_semantic_zero_error_string_not_treated_as_error: replace {error_value!r} f-string (produces invalid JSON with single quotes) with json.dumps so _extract_json_error_text actually parses the payload and the _SEMANTIC_ZERO_ERROR_STRINGS guard is exercised, not bypassed at json.loads test: - add test_before_agent_resets_active_state_consecutive_problems_and_word_sets to lock the ACTIVE-phase run-boundary reset: drives tool to active/cp=1/ws≠() in R1, asserts both fields are zero/empty after before_agent fires for R2 * docs(tool_progress): document intentional per-run reset vs LoopDetection thread-scoped retention Addresses reviewer observation in PR #3601 that _reset_run_states diverges from LoopDetectionMiddleware's cross-run scoping policy without explanation. Expands the _reset_run_states docstring to record the intentional design choice: ToolProgressMiddleware resets per-run because result-quality errors (rate_limited, transient) are time-bound and may resolve between turns — retaining stale counters would risk false-positive BLOCKED calls. LoopDetectionMiddleware retains history across runs because call-pattern loops are time-invariant. The divergence is by design, not oversight. Co-Authored-By: Claude <noreply@anthropic.com> * fix(middleware): restore ReadBeforeWriteMiddleware as outermost write gate A merge conflict resolution had accidentally placed ReadBeforeWriteMiddleware after ToolErrorHandlingMiddleware (inner), reversing the original intent from b81334cc where it was the outermost write gate before ToolErrorHandling. fix: - Restore ReadBeforeWriteMiddleware to outer position: ReadBeforeWrite → ToolProgress → ToolErrorHandling. Blocked writes now return immediately without consuming a ToolProgress slot. - Add normalize_tool_result call on blocked ToolMessages so they carry deerflow_tool_meta (recoverable_by_model=True) even though they bypass ToolErrorHandlingMiddleware. test: - Add test_blocked_write_has_deerflow_tool_meta (sync + async) to lock the normalize_tool_result behavior on blocked writes. - Fix chain order assertions in TestChainWiring and test_build_lead_runtime_middlewares_chain_order_matches_agents_md. docs: - Renumber AGENTS.md items: 10→ReadBeforeWrite, 11→ToolProgress, 12→ToolErrorHandling; update descriptions to reflect outermost-gate design. - Fix stale cross-reference: LoopDetectionMiddleware (item 23) → (item 25). Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-06 20:57:41 +08:00
Tianye Song	28f2b07b79	feat(memory): add staleness review to prune silently-outdated facts (#3860) ... * feat(memory): add staleness review to prune silently-outdated facts Facts created long ago may become outdated without any future conversation explicitly contradicting them ("Silent Staleness"). This adds a staleness review mechanism that surfaces aged facts to the LLM during the normal memory-update call so it can semantically judge whether each is still valid. - New MemoryConfig fields: staleness_review_enabled, staleness_age_days, staleness_min_candidates, staleness_max_removals_per_cycle, staleness_protected_categories - New STALENESS_REVIEW_PROMPT section injected into MEMORY_UPDATE_PROMPT when enough stale candidates exist - New staleFactsToRemove output field in the LLM response schema - Safety cap limits max removals per cycle, keeping lowest-confidence entries when the LLM returns more than the cap - Correction facts (category=correction) are protected by default - Observability via structured logging of each removal with reason - 32 unit tests covering parsing, selection, triggers, formatting, normalization, safety cap, and integration * fix(memory): add deterministic guardrail for staleness removals _apply_updates previously removed any fact id the LLM returned in staleFactsToRemove without verifying it was in the actual staleness candidate set. An LLM slip could silently delete protected-category facts (e.g. correction) or fresh facts, defeating the stated guarantee. Now intersect stale_ids_to_remove with _select_stale_candidates before the safety cap, making the protection independent of both model behavior and the staleness_review_enabled flag. Add three regression tests: - test_protected_category_fact_refused_at_apply - test_non_aged_fact_refused_at_apply - test_guardrail_runs_when_staleness_review_disabled * docs(memory): sync AGENTS.md staleness config + simplify datetime parsing Address reviewer feedback from PR #3860: - Add staleness workflow step and 5 new config fields to backend/AGENTS.md - Simplify _parse_fact_datetime: drop manual Z→+00:00 replace, Python 3.12+ fromisoformat handles Z natively --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-06 15:09:56 +08:00
Vanzeren	67c8ade375	feat(sandbox): warm pool for boxlite (#3951) ... * feat: add boxlite SDK as harness dependency * test: add fake SimpleBox fixtures for BoxLite warm pool tests * feat: add deterministic sandbox_id and warm pool fields to BoxliteProvider * feat: pass deterministic sandbox_id to SimpleBox name * feat: warm pool lifecycle — park on release, reclaim on acquire - release(): parks VMs in _warm_pool with timestamp instead of closing - _reclaim_warm_pool(): health checks warm boxes via echo ok - acquire(): tries warm pool reclaim before creating new boxes - Deterministic sandbox_id ensures thread isolation * feat(boxlite): idle reaper, replica enforcement, warm-pool shutdown/reset - Task 6: idle reaper daemon thread destroys expired warm-pool boxes - Task 7: replica enforcement evicts oldest warm-pool box when at capacity - Task 8: shutdown() stops idle checker first, destroys all boxes (active+warm); reset() clears warm pool Tests: 17 passed (4 new: idle reaper, replica enforcement, shutdown, reset) * fix(boxlite): harden warm pool lifecycle races * docs(boxlite): document warm pool configuration * fix(sandbox): log warning when evicting oldest warm box is failed Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(boxlite): stop provider lifecycle on reset * fix(boxlite): make runtime an optional dependency * test(boxlite): remove unused warm pool lookup * docs(boxlite): clarify optional runtime support * refactor: extract shared WarmPoolLifecycleMixin for sandbox warm-pool lifecycle Introduce deerflow.community.warm_pool_lifecycle.WarmPoolLifecycleMixin owning idle-checker loop, warm-pool expiry, oldest-warm eviction, replica counting, and soft-cap logging. Move AioSandboxProvider and BoxliteProvider onto the mixin; keep AIO active-idle cleanup local and delegate only warm-pool expiry to the shared helper. BoxliteProvider also gains: - Prefixed box names (deer-flow-boxlite-*) for startup orphan reconciliation - _reconcile_orphans() adopting surviving boxes from a prior process - Pinned timeout forwarding: command timeout now bounds both BoxLite SDK exec(timeout=...) and the loop bridge .result(timeout) - reset() reworked as lightweight registry clear (boxes -> warm pool, no close, no idle-reaper stop, no loop close) so reset_sandbox_provider() config switches are safe; shutdown() remains the teardown path Backward-compatible: AIO DEFAULT_IDLE_TIMEOUT/DEFAULT_REPLICAS/ IDLE_CHECK_INTERVAL stay importable; Boxlite IDLE_CHECK_INTERVAL stays monkeypatchable. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-07-06 07:21:12 +08:00
Tianye Song	15454b6fec	feat(skills): deferred skill discovery via describe_skill tool (#3775) ... Replace the full-metadata <available_skills> system-prompt block with a compact <skill_index> (names only) and an on-demand describe_skill tool when skills.deferred_discovery: true (default: false / backward compat). New modules: - skills/catalog.py — SkillCatalog (immutable, searchable; select: has no cap, keyword/prefix search caps at MAX_RESULTS=5) - skills/describe.py — build_describe_skill_tool(catalog) closure; build_skill_search_setup() wires SkillSearchSetup into both the LangGraph agent factory (agent.py) and DeerFlowClient (client.py) Changes: - Skill @dataclass(frozen=True); allowed_tools/required_secrets list→tuple - Skill First prompt line gated on skill_names (deferred vs legacy wording) - get_skills_prompt_section: short-circuit storage on deferred path; merge user_id (upstream) + skill_names (this PR) params - describe_skill tool parameter named "name" (matches prompt wording) - select: branch removes [:MAX_RESULTS] cap (exact request, not ranking) - AGENTS.md: document deferred_discovery config field + new modules Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-04 23:09:29 +08:00
Zheng Feng	dcb2e687d5	feat(channels): add GitHub as a webhook-driven channel (#3754) ... * feat(channels): add GitHub event-driven agents (#3754) Add a webhook-driven GitHub channel with fail-closed webhook routing, deterministic per-agent PR/issue threads, mention-gated trigger fan-out, GitHub App token injection for sandboxed gh/git commands, and backend/AGENTS.md documentation. * fix(llm-middleware): classify bare IndexError as transient Upstream chat providers occasionally return 200 OK with an empty generations list (observed against Volces "coding" on ark.cn-beijing.volces.com). When that happens, langchain_core.language_models.chat_models.ainvoke raises ``IndexError: list index out of range`` at ``llm_result.generations[0][0].message`` and kills the run. Treat a bare IndexError reaching the middleware as a transient upstream-payload glitch and route it through the existing retry/backoff path instead of failing the whole agent run. The retry budget and backoff schedule are unchanged. Adds three regression tests covering the classifier and both the recover-on-retry and exhausted-retries paths. * fix(runtime): ignore stale LLM fallback markers from prior runs When a run on a thread ends with the LLM-error-handling middleware emitting a `deerflow_error_fallback`-marked AIMessage (e.g. after the IndexError empty-generations classification fix lands), that message is persisted to the thread's checkpoint as part of the messages channel. LangGraph replays the full message history in `stream_mode="values"` chunks, so every subsequent run on the same thread re-streams the stale fallback marker — and the worker's chunk scanner faithfully picks it up, flipping `RunStatus.success` to `RunStatus.error` for runs that themselves had no LLM failure at all. Snapshot the set of pre-existing message ids from the pre-run checkpoint and thread it through `_extract_llm_error_fallback_message` / `_try_extract_from_message` as a filter. Markers on history messages are ignored; markers on fresh messages produced during this run still trip the error path. Falls back to an empty set when the checkpointer is absent or the snapshot can't be captured, preserving the prior behavior on first-run / no-state paths. Adds unit tests for the new filter (helper-level and `_collect_pre_existing_message_ids`) plus an integration test exercising the full `run_agent` path with a stale history checkpointer. * fix(channels): make github channel fire-and-forget to avoid httpx.ReadTimeout on long runs GitHub agent runs (clone -> edit -> test -> push -> PR) routinely exceed the langgraph_sdk default 300s read deadline. The manager's runs.wait call kept an HTTP stream open for the entire run lifetime, so the long run blew up with httpx.ReadTimeout and the outer except branch then released the dedupe key and emitted a false 'internal error' outbound. The GitHub channel's outbound send is log-only by design: agents post to the issue/PR via the gh CLI in the sandbox when they choose to comment or create a PR. There is nothing for the manager to ferry back, so the long-poll was pure overhead. This change adds ChannelRunPolicy.fire_and_forget (default False) and sets it True for the github channel. When fire_and_forget is True, _handle_chat dispatches via client.runs.create (short POST, returns once the run is pending) instead of client.runs.wait, and skips the response-extraction + outbound-publish block. ConflictError on a busy thread still trips the standard THREAD_BUSY_MESSAGE path so behavior on the busy case is preserved for any future non-github fire-and-forget channel. Other (non-github) channels are unchanged: their policy defaults fire_and_forget=False and they continue to dispatch via runs.wait. Adds 6 regression tests in tests/test_channels.py::TestGithubFireAndForget: - Default ChannelRunPolicy.fire_and_forget is False. - The github policy registers fire_and_forget=True. - github inbound calls runs.create, not runs.wait, with the right kwargs. - github inbound publishes no outbound on success. - ConflictError from runs.create still emits THREAD_BUSY_MESSAGE. - Non-github channels (slack) still dispatch via runs.wait. * test(lead-agent): accept user_id kwarg in skill-policy test stubs The two GitHub-channel tests added in #3754 stubbed _load_enabled_skills_for_tool_policy with a lambda that only accepted `available_skills` and `app_config`, but the real function (and its call site in agent.py) also passes `user_id`. This raised TypeError on every run, failing backend-unit-tests. Add `user_id=None` to match the three sibling stubs in the same file. * refactor(gateway): disambiguate context-key set names The two frozensets _INTERNAL_ONLY_CONTEXT_KEYS and _CONTEXT_ONLY_KEYS shared a confusable "CONTEXT_ONLY" token in different orders, and the first broke the _CONTEXT_<X>_KEYS pattern of its sibling _CONTEXT_CONFIGURABLE_KEYS. Rename to make the distinct axes explicit: _CONTEXT_INTERNAL_CALLER_KEYS - WHO: internal callers (scheduler) only _CONTEXT_RUNTIME_ONLY_KEYS - WHERE: runtime context only, never configurable Pure rename, no behavior change.	2026-07-04 22:56:24 +08:00
Xinmin Zeng	4fc08b4f15	feat: add scheduled tasks MVP (#3898) ... * feat: add scheduled tasks MVP * fix: harden scheduled task execution semantics * feat(scheduled-tasks): preset-driven schedule form with timezone and live preview Replace the raw cron input with a preset Select (hourly/daily/weekly/monthly/custom) plus structured inputs (time picker, weekday toggles, day-of-month), datetime-local for one-time tasks, a timezone selector defaulting to the browser timezone, and a live human-readable preview. Reuses one ScheduledTaskScheduleInput for create and edit; backend contract unchanged; zero new deps (pure Intl + DST-safe offset helpers). * feat(scheduled-tasks): full-page i18n + recipe templates + E2E locale pin Localize the rest of the scheduled-tasks page (filters, detail pane, actions, edit form, run list, enum values) via t.scheduledTasks.* in en/zh. Add four built-in recipe templates (GitHub Trending, news digest, issue triage, weekly report) exposed as a chip row that pre-fills title + prompt + schedule. Pin Playwright locale to en-US so E2E selectors stay stable against i18n. No backend change, no new deps. * fix(scheduled-tasks): idempotent 0003 migration, update head constants, future-date once test Merge with main surfaced three CI failures: - 0003_scheduled_tasks create_table collided with legacy test seeds that build from full metadata; guard with inspector.has_table so the revision no-ops when the table already exists (0004/0005 are already idempotent via _helpers.py). - persistence bootstrap concurrency/regression tests pinned HEAD to main's 0002_runs_token_usage; bump to the new head 0005_scheduled_task_thread_nullable. - once-task router test used a fixed past run_at and tripped the must-be-in-the-future validation; use a future date. * address review: ok-check, 502 for trigger failure, mock fields, migration filename, doc fences - fetchThreadScheduledTasks now checks response.ok like the other fetchers. - trigger endpoint returns 502 (not 409) when dispatch fails outright, so clients can distinguish a real conflict from a server-side failure. - E2E mock normalizes scheduled-task objects with context_mode/last_thread_id and nullable thread_id, matching the backend contract the UI renders against. - Rename 0002_scheduled_tasks.py -> 0003_scheduled_tasks.py to match its revision id (file was renamed in spirit already; filename now follows). - CONFIGURATION.md: close the Tool Groups yaml fence and drop the stray fence after the Scheduler notes so the sections render correctly. * fix(scheduled-tasks): harden lease, poller, config, and frontend UX after review * fix(scheduled-tasks): harden run lifecycle, overlap skip, non_interactive gating, and DST conversion after review - defer a once task's terminal status to the run-completion hook; the task stays running until the real outcome, and a startup sweep cancels once tasks orphaned by a crash (launch-time 'completed' could stick forever) - record interrupted runs as a distinct 'interrupted' run status with a readable message; an interrupted once task ends 'cancelled', not 'failed' - enforce overlap_policy=skip for fresh_thread_per_run via an active-run pre-check (same-thread ConflictError can never fire across fresh threads) - protect terminal run statuses from the late launch-path 'running' write - honor context.non_interactive only for internally-authenticated callers; arbitrary clients can no longer strip ask_clarification - fix DST-stale timezone offset in zonedLocalToUtcIso by re-deriving the offset at the resolved instant (once tasks fired an hour late around spring-forward and the create->edit round-trip diverged) - drop dead ScheduledTaskRunRepository.update_by_run_id; share one Gateway API error helper between channels and scheduled-tasks frontends * fix(scheduled-tasks): close review round-3 gaps in guards, concurrency, and API ergonomics - scrub internal-only context keys (non_interactive) from the assembled run config for non-internal callers: gating body.context alone left the same key smuggle-able through the free-form body.config copied verbatim by build_run_config - guard update_after_launch with protect_terminal so the launch bookkeeping write cannot clobber a once task already finalized by a fast-failing run's completion hook (parent-row sibling of the run-row guard) - reject a manual trigger while the task has an active run (409) instead of launching a duplicate concurrent run on fresh_thread_per_run - re-arm a terminal once task to enabled when PATCH pushes run_at into the future; previously the endpoint returned 200 with a next_run_at that could never be claimed - make max_concurrent_runs a real global cap: each poll claims only into the remaining budget of active (queued/running) scheduled runs - paginate GET /scheduled-tasks/{id}/runs (limit<=200, offset) and push the thread filter of /threads/{id}/scheduled-tasks into SQL - stamp context.user_id on scheduler-launched runs, matching IM channels, so user-scoped guardrail providers see the owning user --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-04 21:51:57 +08:00
Janlay	38342b15a3	fix(redis): stream retention recovery (#3933) ... * fix(gateway): retain redis streams safely * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-07-04 16:07:42 +08:00
Zhipeng Zheng	53a80d3ad1	feat(skills): per-user custom skill isolation with sandbox mounting (#3889) ... * feat(skills): per-user skill isolation (#2905) Implement user-scoped skill storage that isolates custom skills between users while sharing public skills globally. Key changes: - Add UserScopedSkillStorage class for per-user custom skill directories - Introduce get_or_new_user_skill_storage() factory with user_id context - Auth middleware sets effective_user_id for request-scoped storage - Agent/prompt/middleware now use user-scoped storage and prompt cache - Sandbox mounts user-scoped skill directories for search/read tools - Add validate_skill_file_path() to SkillStorage for path security - Migration script supports --all-users bulk migration - Frontend: add editable field to Skill type, error check in enableSkill - All skill categories can be toggled (custom skills default to enabled) - Update skill-creator SKILL.md with isolation-aware instructions Tests: - Add test_user_scoped_skill_storage.py (new) - Update all existing skill tests for user-scoped storage - Update sandbox, client, and router tests * fix(skills): address second-round PR review feedback (#3889) - P1-1: restrict legacy skill mount to users without custom skills - P1-2: fail-closed for _is_disabled_skill_path (OSError → return True) - P2-1: AND-merge global extensions_config skill disabled state - P2-2: atomic write for _skill_states.json (mkstemp + replace) - P2-3: normalize X-DeerFlow-Owner-User-Id in trusted boundary - P2-4: LRU-bounded _enabled_skills_by_config_cache (OrderedDict, maxsize=256) - P2-5: clear global prompt cache on PUBLIC skill toggle - P2-6: invalidate skill caches on client.update_skill * fix(tests): correct tool policy test after merge * fix(skills): use DEFAULT_SKILLS_CONTAINER_PATH in UserScopedSkillStorage The "/mnt/skills" literal in UserScopedSkillStorage.__init__ triggers test_skill_container_path_defaults::test_mnt_skills_literal_is_owned_by_skill_constants_module on CI. Migrate the default to the existing deerflow.constants constant, matching the pattern already used by LocalSkillStorage, SkillStorage, and the durable/tool_error middlewares. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-04 13:54:04 +08:00
AochenShen99	66b9e7f212	feat: emit structured runtime metadata (follow-up#3887) (#3906) ... * feat: emit structured runtime metadata * fix: avoid subagent import cycle in replay gateway * fix: preserve legacy subtask result parsing * refactor: tighten runtime metadata contracts * fix(middleware): keep recovery hint on task exception wrapper content The structured-metadata stamp overwrote the wrapper text with the bare task-failure message, dropping the model-facing 'Continue with available context, or choose an alternative tool.' guidance that every other tool exception keeps. Append the shared hint after the formatted message. * fix(subagents): require lowercase hex for result_sha256 reader Length-only validation accepted any 64-char string; a faulty serializer or relaying wrapper could store a non-digest value in the delegation ledger. Enforce the producer's hexdigest shape with a fullmatch. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-04 11:27:19 +08:00
Janlay	72f033fbbe	feat(gateway): add redis stream bridge (#3191) ... * feat: add redis stream bridge * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(gateway): address redis stream bridge review Redis was imported eagerly through deerflow.runtime and declared as a hard dependency, which made memory-only installs load redis.asyncio at startup and left the lazy factory import ineffective. Move redis behind an optional extra, remove the public eager re-export, and keep make_stream_bridge as the only runtime import path with an actionable install hint when the extra is missing. Because Docker deployments now default the stream bridge to Redis via DEER_FLOW_STREAM_BRIDGE_REDIS_URL, install the redis extra explicitly in Docker/dev container flows and teach the local uv-extra detector to infer redis from both stream_bridge.type and the Redis URL env var. This keeps Docker working while preserving slim non-Docker installs. Harden the Redis bridge by batching XREAD replay, replacing brittle ResponseError string matching with a single fallback to 0-0 for malformed Last-Event-ID values, documenting connection/retention/fail-hard behavior, and adding fake plus opt-in real Redis coverage for XADD/XREAD, replay, invalid IDs, and MAXLEN trimming. * fix(config): bump config version for stream bridge * fix redis stream bridge terminal handling * fix: repair uv.lock, format redis.py, and align Dockerfile extras test The uv.lock file was missing a closing bracket for the redis extras section, redis.py had a formatting issue caught by ruff, and the Dockerfile extras test did not account for the hardcoded --extra redis flag. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-07-04 09:21:19 +08:00
Nan Gao	b81334ccfe	feat(middlewares): deterministic read-before-write version gate for file tools (#3911) (#3912) ... * docs(specs): read-before-write gate design for issue #3857 output layer * docs(plans): read-before-write gate implementation plan (#3857) * refactor(sandbox): extract read_current_file_content helper (#3857) * feat(middlewares): read-before-write version gate for file tools (#3857) * test(middlewares): pin async read-before-write gate paths (#3857) * feat(config): wire ReadBeforeWriteMiddleware into runtime chain, default on (#3857) * docs(sandbox): document read-before-write gate in tool docstrings and AGENTS.md (#3857) * docs(plans): align plan doc with landed config_version (17) and drop machine-specific paths Addresses Copilot review comments on #3912. * fix(middlewares): read-before-write gate — error-string sandboxes fail open; serialize gate+execution per path (#3912 review) - AIO/E2B read_file reports failures (incl. missing files) as 'Error: ...' strings instead of raising; the gate treated that string as existing file content and blocked first-write creation. Error-string reads now count as uninspectable: gate fails open, no mark is stamped. - LangGraph runs one AIMessage's tool calls concurrently, so two same-turn writes could both pass on one stale mark before either mutation landed (and a read mark could hash a version the model never saw). Gate check + tool execution (and read + mark stamping) now share a per-(thread, path) critical section, separate from the tool-internal file_operation_lock.	2026-07-03 17:21:04 +08:00
bjtolo	c9a5f23e7b	feat(mcp): add per-server tool_call_timeout for MCP tool calls (#3843) ... * feat(mcp): add per-server tool_call_timeout for MCP tool calls Add a configurable timeout for individual MCP tool calls to prevent agent runs from blocking indefinitely when an MCP server becomes unresponsive (e.g., rate-limited HTTP API, hung subprocess). Uses the MCP SDK's built-in read_timeout_seconds parameter on ClientSession.call_tool, which handles the timeout within the session's own task — avoiding cross-task cancellation issues with the session pool (ref #3379, #3203). Config field is named tool_call_timeout (not timeout) to avoid collision with langchain-mcp-adapters' existing timeout field on HTTP/SSE connections. Closes #3840 * fix(mcp): read tool_call_timeout from McpServerConfig, not connection dict The previous implementation put tool_call_timeout into the connection dict returned by build_server_params, which langchain's create_session then passed to _create_stdio_session(), causing TypeError. Now reads the timeout directly from ExtensionsConfig.mcp_servers where the wrapper is built, keeping it out of the connection dict entirely. Fixes P1 bug from review on #3843. * test(mcp): regression test for tool_call_timeout not leaking into connection dict Adds two tests: - test_build_server_params_excludes_tool_call_timeout: verifies the connection dict returned by build_server_params() does NOT contain tool_call_timeout - test_stdio_tool_call_timeout_does_not_raise_typeerror: end-to-end test that get_mcp_tools() with a stdio server having tool_call_timeout configured loads tools without TypeError from _create_stdio_session() Regression for PR #3843 P1 bug. * fix(mcp): only pass read_timeout_seconds when tool_call_timeout is set When tool_call_timeout is None, don't pass read_timeout_seconds=None to session.call_tool(). This avoids breaking existing tests that assert on exact call_tool arguments without the extra kwarg. * docs(mcp): clarify stdio tool timeout	2026-07-03 15:39:12 +08:00
AnoobFeng	e3e5c73b03	feat(observability): add trace-id correlation and enhanced logging (#3902) ... * feat(observability): add trace-id correlation and enhanced logging - add opt-in gateway request trace correlation via X-Trace-Id - enhance logging with configurable trace_id-aware formatting - propagate deerflow_trace_id into runtime context and Langfuse metadata - keep enhanced logging disabled by default to preserve existing behavior * fix: harden trace correlation wiring - Make logging enhancement a restart-required startup snapshot and remove per-request config reads from TraceMiddleware - Restrict trace ids to printable ASCII before writing them to response headers, logs, and Langfuse metadata - Gate implicit DeerFlowClient trace-id creation behind logging.enhance.enabled while preserving explicit caller opt-in - Bind embedded client trace context per stream step to avoid generator ContextVar leaks and cross-context reset errors - Rebind memory update trace ids in Timer/executor worker paths so enhanced logs keep the captured correlation id - Remove unrelated __run_journal context overwrite from the trace-correlation change set * fix(gateway): avoid eager app construction on package import * fix(gateway): avoid config load during app import Keep Gateway app construction import-safe when config.yaml is absent by disabling TraceMiddleware only for that construction-time fallback path. Startup lifespan still performs strict config loading before serving.	2026-07-03 08:01:46 +08:00
Yi Deng	4e6248f013	fix(gateway): clamp client-supplied recursion_limit to prevent runaway runs (#3903) ... build_run_config() copied every top-level request key (except configurable/context) verbatim into the LangGraph RunnableConfig, including recursion_limit. The server default of 100 was fully overridable by the caller with no upper bound, so a request like {"config": {"recursion_limit": 100000000}} could make a single run execute effectively unbounded LangGraph super-steps (each >= 1 LLM call), enabling runaway API cost / DoS. Validate the client value server-side and clamp it into a safe range: - valid positive ints are capped at a configurable ceiling (AppConfig.max_recursion_limit, default 1000 to match the existing frontend/public-skill default so legitimate deep runs are unaffected) - invalid/non-positive/bool/None values fall back to the 100 server default - applied on both the configurable and the LangGraph >= 0.6.0 context paths - WARNING logged on clamp for observability Add unit tests (including a configurable-ceiling case), expose max_recursion_limit in config.example.yaml (config_version 16), and document the ceiling/fallback in backend/docs/API.md. Co-authored-by: DengY11 <DengY11@users.noreply.github.com>	2026-07-02 11:38:21 +08:00
AochenShen99	442248dd06	feat: preserve durable context across summarization (#3887) ... * feat: preserve durable context across summarization * fix: harden durable context review gaps * style: format delegation ledger live test * chore: remove stale delegation ledger prefix * fix: address durable context review feedback	2026-07-01 22:49:17 +08:00
xiawiie	2453718acd	fix(title): avoid default LLM call before stream end (#3885) ... * fix(title): avoid default LLM call before stream end * fix(title): keep default fallback local * fix(title): harden fallback and replay e2e * docs(title): align fallback title behavior	2026-06-30 18:58:02 +08:00
DanielWalnut	fe82552023	fix(sandbox): stop blocking bash commands (e.g. servers) from hanging the turn (#3864) ... * fix(sandbox): stop blocking bash commands from hanging the turn Starting a server through the host bash tool (e.g. `python -m http.server`) could hang the whole turn for the full 600s timeout. `LocalSandbox.execute_command` used `subprocess.run(capture_output=True)`, whose captured pipes are inherited by any process the command spawns — so a backgrounded long-lived process (`server &`) keeps the read end open and blocks `communicate()` until the timeout fires, even though the foreground command already returned. Commands that read stdin blocked the same way, and on timeout only the direct child was killed, leaving orphaned process groups. Rework the POSIX path to capture stdout/stderr via temp files instead of pipes, take stdin from /dev/null, and run the command in its own session/process group: - Backgrounded long-lived processes (servers) now return immediately while the process keeps running. - A command reading stdin gets immediate EOF instead of blocking. - A genuinely blocking foreground command is bounded by a configurable wall-clock timeout; on timeout the whole process group is killed and the agent gets an explanatory notice telling it to background long-lived processes. The timeout is configurable via `sandbox.bash_command_timeout` (default 600). The Windows path is unchanged. Adds focused regression tests and updates docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): instruct the agent to background long-lived processes The code fix bounds a foreground server with a timeout, but the turn still waits the full timeout before the run continues. Add the prompt-side half: the bash tool description now tells the model to ALWAYS start long-lived processes (e.g. web servers) in the background with output redirected, so the tool returns immediately. The timeout notice points at the same readable workspace log path. Pins the guidance with a test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): make fallback-kill exception explicit and observable Address automated review: the inner `except OSError: pass` in _terminate_process_group silently swallowed the case where the direct-child fallback kill found the process already gone. Make the intent explicit with a comment and a debug log instead of a bare pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): address bash timeout review feedback * fix(sandbox): document fd cleanup races --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-30 14:55:31 +08:00
ly-wang19	b66e3253a0	perf(config): index AppConfig.get_*_config by name (O(n) -> O(1)) (#3688) ... get_model_config / get_tool_config / get_tool_group_config did a next(...) linear scan of self.models / self.tools / self.tool_groups on every call. These sit on hot paths: get_tool_config runs 2-3x per community-tool invocation (web_search etc.) and get_model_config several times per agent build, across 40+ call sites. Build name -> config dicts once in a mode="after" validator (stored as PrivateAttr), so each getter is an O(1) dict lookup. A config reload constructs a fresh AppConfig, which rebuilds the indexes; setdefault keeps first-match-wins on duplicate names, matching the prior next(...) semantics. Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-23 10:39:40 +08:00
Tianye Song	b990da785f	feat(memory): add guaranteed injection for correction facts with graceful fallback (#3592) ... * feat(memory): add guaranteed injection for correction facts with graceful fallback When the token budget is tight, high-value facts (e.g. user corrections) can be silently evicted by lower-priority regular facts. This change: - Introduces configurable 'guaranteed_categories' (default: [correction]) whose facts draw from a separate 'guaranteed_token_budget', ensuring they are never dropped due to budget pressure. - Adds a graceful fallback to confidence-only ranking when the guaranteed-category path raises an unexpected exception. - Refactors fact selection into a header-agnostic helper (_select_fact_lines) with explicit token accounting in the caller, eliminating double-counting of separators. - Emits a single 'Facts:' header regardless of whether both guaranteed and regular facts are present. - Extends the final safety truncation limit to account for the additional guaranteed budget so guaranteed facts survive end-to-end. * refactor(memory): address review feedback on guaranteed injection - Restore strict break-on-overflow in `_select_fact_lines` to preserve the caller's confidence-ordered ranking; add a regression test locking in the invariant that a shorter lower-confidence fact never slips ahead of a skipped higher-confidence one. - Account for the inter-group `\n` separator between guaranteed and regular fact blocks in the regular budget (1-token precision fix). - Clarify docstrings on `format_memory_for_injection` and `MemoryConfig.guaranteed_token_budget` to distinguish the common *displacement* case (total stays within `max_tokens`) from the rarer *additive* case (safety-truncation ceiling raised when guaranteed lines alone would overflow). * fix(memory): address P1 safety truncation + P2s from review - Structure-aware safety truncation: Facts block is now a protected suffix so guaranteed-category facts can never be silently discarded by a prefix-cut on overflow. Only the preceding (user/history) sections are eligible for truncation. - Extend the same protected-suffix treatment to the except/fallback path by returning fact lines alongside the formatted section from _fallback_format_facts, avoiding string parsing. - Single inter-section separator: facts section no longer embeds its own leading \n\n; the final "\n\n".join(sections) is the single source of truth for section-to-section spacing. - Bare string for guaranteed_categories now raises TypeError instead of silently iterating single characters. - Category-less / malformed facts no longer default-promote into the guaranteed "context" pool — only facts with an explicit category field qualify. - Lift valid_facts pre-filter outside the try so the fallback path reuses it instead of re-doing validation work. - MemoryConfigResponse + DeerFlowClient.get_memory_config now expose guaranteed_categories / guaranteed_token_budget. - config.example.yaml: document the two new fields and bump config_version from 12 to 13. - Add regression tests for every finding. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-06-23 08:10:12 +08:00
Chetan Sharma	78fff5a5e2	feat(middleware): add TokenBudgetMiddleware for per-run token budget e… (#3412) ... * eat(middleware): add TokenBudgetMiddleware for per-run token budget enforcement * address copilot comments * resolve feedback --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-06-21 22:25:15 +08:00
ly-wang19	c177d0e542	fix(config): coerce null object config sections to their defaults (#3573) ... * fix(config): coerce null object config sections to their defaults #3434 made commented-out list sections (models/tools/tool_groups) parse as [] instead of crashing, but the same scenario still crashes for object sections: commenting out every key under e.g. memory: / summarization: / guardrails: makes PyYAML parse the value as None, and AppConfig then raises "Input should be a valid dictionary" for that section — breaking the documented `cp config.example.yaml config.yaml` first-run flow. Generalize the handling: a model_validator(mode="before") drops None-valued sections so each field falls back to its default (list sections -> [], object sections -> their default config). This subsumes the previous list-only field_validator and the database special-case. Required sections without a default (sandbox) still error when null. Adds test_app_config_coerces_commented_out_object_sections; the existing list-section regression test still passes. * docs+test: address review on null-section coercion - Correct the _drop_null_config_sections docstring: it does not subsume the database special-case; _apply_database_defaults still owns `database` and applies concrete defaults beyond null-coercion. - Strengthen test_app_config_coerces_commented_out_object_sections to assert each null section falls back to its expected default config type, not just that it is non-None. - Add test_app_config_null_required_section_still_errors covering the required-section (sandbox) "still errors when null" claim. --------- Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>	2026-06-21 22:16:00 +08:00
Zheng Feng	ee8ad1bc67	feat(auth): add OIDC SSO support (#3506) ... Add provider-agnostic OIDC authentication with Keycloak-compatible configuration, frontend SSO UI support	2026-06-21 15:47:53 +08:00
Huixin615	ec16b6650d	fix(channel): force reload config on channel restart (#3619) ... * fix: force reload config on channel restart * fix: detect config content changes for reload	2026-06-17 22:57:46 +08:00
Nan Gao	0966131b31	fix(channels): require bound identity for user-owned IM messages (#3578) ... * fix(channels): require bound identity for user-owned IM messages * make format * docs: document bound identity channel config * refactor: reuse channel connection config * refactor _requires_bound_identity() * refactor from_app_config() * make format * fix: reject unbound channel chats before semaphore * security enhancement * make format * fix: enforce bound-identity admission at command entry point The bound-identity gate only ran for non-command messages in _handle_message() and as a fallback inside _handle_chat(). Commands had no equivalent boundary, so an unbound platform user could send /new and reach _create_thread() directly, creating an unowned Gateway thread and empty checkpoint. Info commands (/status, /models, /memory) likewise leaked Gateway state to unbound users. Add the same _requires_bound_identity() check at the top of _handle_command(), rejecting via _reject_unbound_channel_message() before any thread creation or Gateway query. The gate is a no-op in legacy open-bot mode (require_bound_identity=False) and auth-disabled mode. Provider-level binding flows (/connect, /start) are consumed by the provider adapter before reaching the manager, so they are unaffected. Tests: - unbound auth-enabled /new is rejected before threads.create - bound auth-enabled /new still creates the thread Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(channels): carry workspace fallback decision on inbound messages * fix(channels): recheck bound identity by normalized workspace * fix(channels): avoid duplicate bound identity checks * fix(channels): preserve verified routing for bound identity rejects * fix(channels): clarify bound identity upgrade failures --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-06-16 23:04:39 +08:00
DanielWalnut	05be7ea688	fix(subagents): raise general-purpose max_turns to 150 and default timeout to 30min (#3610) ... * fix(subagents): raise general-purpose max_turns to 150 and default timeout to 30min Deep-research subtasks failed out of the box with GraphRecursionError (Recursion limit of 100 reached): the built-in general-purpose subagent caps at max_turns=100. Raise it to 150 and bump the default subagent timeout from 900s (15min) to 1800s (30min) so the extra turns have time to run instead of shifting the failure to a timeout. The lead agent recursion_limit (100) is unchanged; the failures are subagent-only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(subagents): clarify lead recursion_limit is independent of subagent max_turns Add comments at both lead recursion_limit=100 sites (gateway services + channel manager) explaining the lead's LangGraph super-step budget is separate from subagent depth, so the two 100s are not conflated. Comment-only, no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(subagents): clarify built-in vs custom timeout scope; pin bash max_turns in test Review follow-ups: (1) clarify SubagentConfig docstring + global timeout field/comment that the 1800 default applies to built-in subagents (custom agents keep their own timeout_seconds); (2) pin bash.max_turns==60 in the defaults regression test so the config.example.yaml doc cannot drift; (3) rename test_default_timeout_preserved_when_no_config -> test_explicit_global_timeout_propagates_to_general_purpose since it intentionally exercises an explicit non-default 900. No runtime behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 19:55:04 +08:00
Chetan Sharma	d2cc991d55	make ai follow-up suggestions optional (#3591)	2026-06-15 17:59:25 +08:00
DanielWalnut	aa015462a7	feat(im): Add user-owned IM channel connections (#3487) ... * Add user-owned IM channel connections * Fix dev startup and channel connect popup * Use async channel connect flow * Harden dev service daemon startup * Support local IM channel connections * Align IM connections with local channels * Fix safe user id digest algorithm * Address Copilot IM channel feedback * Address IM channel review comments * Support all integrated IM channel connections * Format additional channel connection tests * Keep unavailable channel connect buttons clickable * Fix IM channel provider icons * Add runtime setup for enabled IM channels * Guard global shortcut key handling * Keep configured IM channels editable * Avoid password autofill for channel secrets * Make channel threads visible to connection owners * Persist IM runtime config locally * Allow disconnecting runtime IM channels * Route no-auth channel sessions to local user * Use default user for auth-disabled local mode * Show IM channel source on threads * Prefill IM channel runtime config * Reflect IM channel runtime health * Ignore Feishu message read events * Ignore Feishu non-content message events * Let setup wizard enable IM channels * Fix frontend formatting after merge * Stabilize backend tests without local config * Isolate channel runtime config tests * Address channel connection review comments * Use sha256 user buckets with legacy migration * Ensure runtime IM channels are ready after restart * Persist disconnected IM channel state * Address channel connection review comments * Address channel connection review findings Frontend connect flow: - Open the runtime-config dialog only when a provider still needs credentials; configured providers go straight to the connect flow, so the binding-code/deep-link path is reachable from the UI again. - After saving credentials, continue into the connect flow when a user binding is still required (multi-user mode) instead of stopping at a "Connected" toast. - Extract shared provider-state helpers to core/channels/provider-state and add unit + e2e coverage for the direct-connect and configure-then-connect paths. Provider status semantics: - Report connection_status from the user's newest connection row; with no binding it is not_connected, except in auth-disabled local mode where a configured running channel is effectively connected. Concurrency and event-loop correctness: - Offload ChannelRuntimeConfigStore construction and writes, channel service construction, and Slack connection replies to threads; add a tests/blocking_io/ anchor for the runtime-config handlers. - Consume binding codes with a conditional UPDATE so a code can only be used once under concurrent workers; retry upsert_connection as an update when a concurrent insert wins the unique constraint. - Serialize ensure_channel_ready per channel so concurrent provider polls cannot double-start a channel worker. Config and migration hardening: - Stop mutating the get_app_config()-cached Telegram provider config; the runtime store now owns the UI-entered bot username. - Register channel_connections in STARTUP_ONLY_FIELDS with the standardized startup-only Field description. - Match the legacy unsafe-id bucket by recomputing its exact SHA-1 name so another user's same-prefix bucket can never be migrated. - Remove the unused Telegram process_webhook_update path and document src/core/channels in the frontend docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address PR review comments on authz scoping and channel runtime Security (review feedback from ShenAC-SAC): - Scope internal-token callers to the connection owner carried in X-DeerFlow-Owner-User-Id instead of bypassing owner checks outright, in both require_permission(owner_check=True) and the stateless run endpoints. Internal callers keep access to their own and shared/legacy threads, and may claim a default-owned channel thread for its real owner, but a leaked internal token no longer grants cross-user thread access. - Require admin privileges for POST/DELETE /api/channels/{provider}/ runtime-config: runtime credentials and channel workers are instance-wide shared state (same model as the MCP config API). Read-only provider listing stays available to all users. Performance (review feedback from willem-bd): - Skip the redundant thread channel-metadata PATCH after the first successful backfill per thread. - Reuse the per-connection Slack WebClient until its token changes instead of constructing one per outbound message. - Reconcile channel readiness for all providers concurrently in GET /api/channels/providers. Also resolve the code-quality unused-import flag in the blocking-io anchor by pre-importing the channel service via importlib. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix prettier formatting in provider-state test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Reconcile UI runtime channel config with config reload on restart Main now reloads a channel's config.yaml entry on restart_channel() (#3514, issue #3497). Adapt the user-owned connection flow to coexist: - configure_channel() restarts with reload_config=False — the caller just supplied the authoritative config (browser-entered credentials that are never written to config.yaml), so a file reload must not clobber it with the stale on-disk entry. - _load_channel_config() re-applies the UI runtime-store overlay used at startup, so an operator-triggered restart keeps browser-entered credentials for channels without a config.yaml entry and does not resurrect a channel disconnected from the UI. - Offload the reload's disk IO (config.yaml + runtime store) with asyncio.to_thread, matching the blocking-IO policy on this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-06-12 15:24:58 +08:00
hataa	b3c2cc42cf	fix(agents): require config.yaml in resolve_agent_dir to skip memory-only directories (#3390) (#3481) ... When memory is enabled, the first conversation with a legacy shared agent creates a per-user agent directory containing only memory.json (no config.yaml). On the second turn, resolve_agent_dir() returned this incomplete directory, causing load_agent_config() to fail with "Agent config not found". Require config.yaml to exist alongside the directory for both the per-user and legacy paths, so that memory-only directories fall through correctly. This aligns resolve_agent_dir with the existing config.yaml check in list_custom_agents. Refs: https://github.com/bytedance/deer-flow/issues/3390	2026-06-10 23:57:17 +08:00
Ryker_Feng	167ef4512f	feat(memory): add memory.token_counting config to avoid tiktoken network dependency (#3429) (#3465) ... * feat(memory): add memory.token_counting config to avoid tiktoken network dependency (#3429) Add a `memory.token_counting` option (`tiktoken` | `char`) so deployments in network-restricted environments can opt out of tiktoken entirely. In `char` mode the memory-injection budget uses a network-free character-based estimate and never triggers the BPE download from openaipublic.blob.core.windows.net, which could otherwise block for tens of minutes (see #3402). Also harden the default `tiktoken` path: - cache an in-flight LOADING sentinel so concurrent callers fall back immediately instead of spawning more blocking get_encoding threads when the first load is still running (e.g. under the 5s startup warm-up timeout); - cache failures with a timestamp and retry after a cooldown so a transient network outage self-heals back to accurate counting without a restart; - skip startup warm-up entirely in char mode. The new config is surfaced via the memory config API and config.example.yaml (config_version bumped). Default remains `tiktoken`, so existing deployments are unaffected. * fix(memory): use CJK-aware char token estimate and address review feedback - Replace the flat len(text)//4 fallback with a CJK-aware estimate so Chinese/Japanese/Korean memory content does not over-fill the injection budget - Document the internal tiktoken retry cooldown and char-mode escape hatch - Sync CLAUDE.md / config.example.yaml / MEMORY_IMPROVEMENTS.md wording - Fix MemoryConfigResponse mocks/assertions and add CJK estimate tests	2026-06-10 23:26:15 +08:00
Lucy Shen	ae9e8bc0bf	fix(sandbox): make missing sandbox.mounts host_path a loud ERROR (#3244) (#3250) ... In Docker production deployments, LocalSandboxProvider runs inside the deer-flow-gateway container, so any `sandbox.mounts[].host_path` from config.yaml is resolved against the gateway container's filesystem — not the host machine. When the path isn't also bind-mounted into the gateway service, the mount was silently dropped with only a WARNING log, leaving agents reading an empty directory in production while the same config worked under `make dev`. Escalate the missing-host_path branch to logger.error with explicit guidance about Docker bind mounts and docker-compose, so the failure is hard to miss in default log configurations. Skip behaviour is preserved to avoid breaking existing deployments. Also clarify the misleading `VolumeMountConfig.host_path` field description so it documents reality for both providers: - LocalSandboxProvider checks host_path from inside the gateway process (host in `make dev`, container in `make up`). - AioSandboxProvider (DooD) passes host_path straight to `docker -v` for the sandbox container, where the host Docker daemon resolves it from the host machine's perspective. config.example.yaml's `sandbox.mounts` comment gets a Note: block pointing operators at the docker-compose bind-mount requirement so the Docker-mode gotcha is discoverable from the canonical template. Adds a regression test that: - confirms missing host_path is still skipped (no behaviour break); - asserts an ERROR record is emitted referencing the offending paths; - asserts the message contains actionable Docker/gateway/docker-compose keywords so future refactors can't quietly downgrade it. Refs: https://github.com/bytedance/deer-flow/issues/3244	2026-06-09 23:16:14 +08:00
ly-wang19	8db16bb3d8	fix(config): coerce null config.yaml list sections to empty list (#3434) ... Copying config.example.yaml to config.yaml and starting DeerFlow crashed with `pydantic ValidationError: models — Input should be a valid list [input_value=None]`, because the example ships every entry under `models:` commented out, so PyYAML parses the key as null. Reported in #1444. Add a field_validator(mode="before") on AppConfig that coerces null models/tools/tool_groups to [] (matching their default_factory=list), and emit an actionable warning from from_file when no models are configured (pointing to config.example.yaml / make setup). Adds regression tests. Closes #1444 Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-06-09 15:45:28 +08:00
Nan Gao	f725a963d5	fix(runtime): protect sync singleton init and reset (#3413) ... * fix(runtime): protect sync singleton init/reset with threading.Lock * fix(runtime): serialize sync singleton init and reset * make format * test(runtime): assert store reset creates new singleton * Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(runtime): load config outside singleton locks * fix(runtime): share checkpointer config loading helper --------- Co-authored-by: GODDiao <diaoshengjia@gmail.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-06-08 08:38:36 +08:00
Xinmin Zeng	befe334f10	fix(config): make the reload boundary discoverable from code (#3144) (#3153) ... * fix(config): make the reload boundary discoverable from code, not just docs Closes #3144. The hot-reload contract — per-run fields are resolved through `get_app_config()` on every request, infrastructure fields snapshot at gateway startup — landed in `backend/CLAUDE.md` as part of #3131. A maintainer reading `get_config()` or an `AppConfig` field still had to context-switch to that document to know which fields require a process restart, and there was no enforcement that the prose list stayed in sync with the code. This commit moves the boundary to a machine-readable single source of truth and surfaces it where the code lives: - New `deerflow.config.reload_boundary` module owns the registry of restart-required fields (`STARTUP_ONLY_FIELDS`) and a tiny helper API (`is_startup_only_field`, `iter_startup_only_field_paths`, `format_field_description`). The standardised `"startup-only:"` prefix is exported as `STARTUP_ONLY_PREFIX` so future scanners / lint hooks / doc generators can pivot off it without re-parsing prose. - `AppConfig`'s `database`, `checkpointer`, `run_events`, `stream_bridge`, `sandbox`, and `log_level` fields now build their `Field(description=...)` from `format_field_description(...)`. The same text shows up in IDE hover (Pydantic v2 exposes `description` via `model_fields[...]`). - `channels` is restart-required too but lives outside the AppConfig Pydantic schema (the config section is consumed directly by `start_channel_service`). The registry owns it so the boundary is not split between two places. - `get_config()` docstring points to the registry instead of leaving the reader to find `CLAUDE.md`. The `CLAUDE.md` table collapses to a one-liner pointing back at `reload_boundary.py` so the boundary has one canonical location, not two. Drift coverage in `tests/test_reload_boundary.py`: - Every registered field has a non-trivial reason. - Iterator / membership helpers stay in sync with the dict. - Every registry entry that maps to an `AppConfig` field also carries the `"startup-only:"` prefix in the schema (catches "forgot to update the schema"). - Reverse drift: any AppConfig field whose description starts with the prefix must be registered (catches "marked restart-required in the schema but forgot the registry"). - The runtime introspection that IDE hover depends on (`AppConfig.model_fields["database"].description`) is pinned, so a future Pydantic upgrade or schema swap that breaks the hover surface shows up as a test failure rather than a silent regression. Refs: bytedance/deer-flow#3138 (split summary), #3107 (origin), #3131 (prior boundary fix in prose form). * fix(config): preserve field doc and correct log_level reload reason Two follow-ups on the PR #3153 review: 1. The `log_level` STARTUP_ONLY_FIELDS reason previously claimed `apply_logging_level()` mutates the root logger level. It does not: only the `deerflow` / `app` logger levels are set, and root handler thresholds are conditionally lowered so messages from those loggers can propagate. Reword to match the actual behavior so operators reading IDE hover get accurate restart guidance. 2. `format_field_description(field_path)` was the sole `Field(description=)` for every restart-required field, which silently overwrote the original human-facing documentation — most visibly the `log_level` field that used to list debug/info/warning/error and clarify that third-party libraries are not affected. Extend the helper with a keyword-only `field_doc` parameter that composes the startup-only marker with the original prose so IDE hover documents both *why* the field is restart-required and *what* it actually accepts. Updated all six restart-required AppConfig fields (`log_level`, `database`, `sandbox`, `run_events`, `checkpointer`, `stream_bridge`) to pass their original descriptions through the helper. Tests: two new cases in `test_reload_boundary.py` pin (a) the helper composition and (b) every AppConfig restart-required field still surfaces a recognisable substring of its original documentation. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-06-07 21:27:14 +08:00
Huixin615	88e36d9686	fix(#3189): prevent write_file streaming timeout on long reports (#3195) ... * fix(#3189): prevent write_file streaming timeout on long reports Adds a layered defense against StreamChunkTimeoutError caused by oversized single-shot write_file tool calls: - factory: default stream_chunk_timeout to 240s for OpenAI-compatible clients (overridable via ModelConfig.stream_chunk_timeout in config.yaml) - sandbox/tools: server-side 80 KB length guard on non-append write_file calls (configurable via DEERFLOW_WRITE_FILE_MAX_BYTES env var, 0 disables); rejects oversized payloads with a structured error pointing the model at str_replace or append=True - middleware: classify StreamChunkTimeoutError as transient but cap retries at 1 via per-exception _RETRY_BUDGET_OVERRIDES (same-payload retry on a chunk-gap timeout buffers the same way upstream; full 3-attempt loop would stack 6-12 min of dead air) - middleware: surface an actionable user-facing message for stream-drop exceptions instead of leaking the raw langchain stack - prompts: add a routing-style File Editing Workflow hint to both lead_agent and general_purpose subagent prompts, pointing the model at str_replace for incremental edits (mirrors Claude Code's Edit / Codex's apply_patch) - tests: behavioural coverage for size guard, retry budget override, stream-drop user message, factory default injection Refs #3189 * fix(#3189): drop stream_chunk_timeout for non-OpenAI providers Address CR feedback on PR #3195: - factory: pop `stream_chunk_timeout` from kwargs for any model_use_path other than `langchain_openai:ChatOpenAI` instead of returning early. `ModelConfig.stream_chunk_timeout` is part of the shared schema, so a user-supplied value on a non-OpenAI provider would otherwise be forwarded to its constructor and raise `TypeError: unexpected keyword argument`. - factory: rewrite docstring to describe the actual `exclude_none=True` behaviour (explicit null is excluded and falls back to the default) instead of the misleading "None falling out via exclude_none=True keeps its value". - tests: add regression coverage asserting the kwarg is stripped before reaching a non-OpenAI provider's constructor. Refs: bytedance#3189 * fix(#3189): restrict stream-drop user copy to StreamChunkTimeoutError only Per CR on #3195: narrow _STREAM_DROP_EXCEPTIONS to StreamChunkTimeoutError. Generic httpx RemoteProtocolError / ReadError fall back to the standard 'temporarily unavailable' copy, since they routinely fire on transient network blips where the 'split the output' guidance is misleading. Retry/backoff classification is unchanged — both remain transient/retriable. Tests updated to reflect new copy, plus a symmetric regression test for ReadError. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-06-07 17:47:11 +08:00
Ryker_Feng	8fca56cf43	fix(mcp): accept transport field as alias for type (#3238) (#3243) ... The official MCP configuration schema uses `transport` to specify the transport mechanism (stdio/sse/http), but `McpServerConfig` only honored `type` and defaulted to `stdio`. Remote MCP servers configured with just `transport: sse` were therefore misidentified as stdio and failed with "with stdio transport requires 'command' field". Add a model validator that promotes `transport` to `type` when only `transport` is provided, while keeping `type` authoritative when both are set. This matches the MCP-spec field name without breaking existing configurations. Fixes #3238	2026-06-03 18:11:38 +08:00
zhongli-sz	3ae82dc663	fix(mcp): add auth interceptor with channel user_id and keep header propagation to mcp tools (#3294) ... * 修复channel中的user_id传递到interceptor中的bug, mcp可通过header传递user_id到mcp工具 Co-authored-by: Cursor <cursoragent@cursor.com> * fix(channel,mcp,gateway): normalize channel user_id and add regression tests Normalize external channel user ids into filesystem-safe runtime context while preserving raw channel_user_id, and document gateway user_id propagation semantics. Add regression coverage for channel user_id context mapping, gateway user_id precedence/internal-role behavior, and MCP interceptor header forwarding via meta.headers. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auth,mcp): harden user id normalization and header handling Increase sanitized user-id digest suffix to 16 hex chars, replace internal system role magic string with a shared constant, and harden MCP header forwarding with Mapping type checks. Add regression tests for empty channel user_id handling, unsupported header types, and updated digest length behavior. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: zhongli <335302680@qq.com> Co-authored-by: Cursor <cursoragent@cursor.com>	2026-06-03 15:48:19 +08:00
Xinmin Zeng	ca487578a4	feat(agent): add ToolOutputBudgetMiddleware for oversized tool output protection (#3303) ... * feat(agent): add ToolOutputBudgetMiddleware for oversized tool output protection Closes #3289. Adds a unified middleware that enforces per-result budgets on ALL tool outputs (MCP, sandbox, community, custom), preventing oversized external tool results from blowing the model context window. Design informed by claude-code (persistToolResult), hermes-agent (tool_result_storage), and pi (OutputAccumulator) — the three most mature implementations in production coding-agent frameworks. Key features: - Disk externalization: oversized outputs written to thread-local .tool-results/ directory, replaced with compact preview + file reference. Model can read full output via read_file with offset/limit. - Fallback truncation: head+tail truncation when disk is unavailable (no thread_data, write failure), ensuring the context is always protected. - read_file exemption: prevents persist-read-persist infinite loops (independently discovered by claude-code, hermes-agent, and pi). - Per-tool threshold overrides via config. - Line-boundary-aware truncation (no partial lines in previews). - Multimodal content passthrough (images/structured blocks skip budget). - Historical ToolMessage patching in wrap_model_call for checkpoint recovery scenarios. Related: #3222 (design RFC), #1844 (comprehensive context management), #3137 (write_file args compaction), #1677 (sandbox tool truncation). * test: add MCP content_and_artifact format coverage Add 5 tests for MCP tool output format (list of content blocks): - text content blocks are extracted and budgeted - multiple text blocks are joined and budgeted - image content blocks are skipped (multimodal passthrough) - mixed text+image blocks are skipped - small text blocks pass through unchanged Total test count: 59 (was 54). * fix(agent): address Codex review findings for ToolOutputBudgetMiddleware Three issues identified by Codex code review, all fixed: 1. `enabled` config field was unused — middleware now checks `config.enabled` and skips all processing when disabled. 2. `_build_fallback` could exceed `fallback_max_chars` — the marker text itself (~139 chars) was not deducted from the budget. Now pre-computes marker overhead and falls back to hard slice when max_chars is smaller than the marker. 3. Sync file I/O in async path — `awrap_tool_call` now delegates `_patch_result` to `asyncio.to_thread` to avoid blocking the event loop during disk writes. Tests updated to use realistic fallback_max_chars values (500+) that can accommodate the marker overhead, plus two new tests: - `test_result_never_exceeds_max_chars` (parametric across sizes) - `test_very_small_max_chars_does_not_crash` * fix(agent): address Copilot review — path traversal, async perf, shared config 1. Path traversal defense: sanitize tool_name via _sanitize_tool_name() (strips separators, .., absolute paths), validate storage_subdir is relative, and verify resolved filepath stays inside storage_dir. 2. Async hot-path optimization: add _needs_budget() cheap check before asyncio.to_thread offload — small outputs (99% of calls) skip the thread overhead entirely. 3. Replace shared module-level _DEFAULT_CONFIG with _default_config() factory to prevent cross-instance mutation of mutable fields. 12 new tests: TestSanitizeToolName (5), TestExternalizePathTraversal (3), TestNeedsBudget (4). * fix(agent): correct preview hint to match read_file actual API read_file uses start_line/end_line (1-indexed line numbers), not offset/limit. The previous wording was copied from hermes-agent which has a different read_file interface. * perf(agent): hoist hot-path imports, add model-call pre-scan (review #3303) Address maintainer review feedback: 1. Hoist inline imports to module level — `import asyncio` (was in awrap_tool_call hot path) and `from dataclasses import replace` (was in _patch_result) now live at module top. 2. Add a cheap pre-scan to _patch_model_messages so the historical message list is not rebuilt on every model call when nothing is oversized (the common case once results are budgeted at tool-call time). Also adds the same _needs_budget gate to the sync wrap_tool_call for symmetry with awrap_tool_call. The pre-scan is refactored into per-tool-aware helpers (_effective_trigger / _tool_message_over_budget) that mirror the exact trigger conditions in _budget_content — including tool_overrides — so the fast-path can never produce a false negative (silently skipping budgeting for a tool with a low per-tool threshold). 7 new regression tests lock the per-tool-override-through-pre-scan path and the model-call early return. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-05-29 22:59:26 +08:00
Xinmin Zeng	be0eae9825	fix(runtime): suppress tool execution when provider safety-terminates with tool_calls (#3035) ... * fix(runtime): suppress tool execution when provider safety-terminates with tool_calls When a provider stops generation for safety reasons (OpenAI/Moonshot finish_reason=content_filter, Anthropic stop_reason=refusal, Gemini finish_reason=SAFETY/BLOCKLIST/PROHIBITED_CONTENT/SPII/RECITATION/ IMAGE_SAFETY/...), the response may still carry truncated tool_calls. LangChain's tool router treats any non-empty tool_calls as executable, so partial arguments (e.g. write_file with a half-finished markdown) get dispatched and the agent loops on retry. Add SafetyFinishReasonMiddleware at after_model: detect safety termination via a pluggable detector registry, clear both structured tool_calls and raw additional_kwargs.tool_calls / function_call, preserve response_metadata.finish_reason for downstream observers, stamp additional_kwargs.safety_termination for traces, append a user-facing explanation to message content (list-aware for thinking blocks), and emit a safety_termination custom stream event so SSE consumers can reconcile any "tool starting..." UI. Default detectors cover OpenAI-compatible content_filter, Anthropic refusal, and Gemini safety enums (text + image). Custom providers are added via reflection (same pattern as guardrails). Wired into both lead-agent and subagent runtimes. Closes #3028 * fix(runtime): persist safety_termination as a middleware audit event Address review on #3035: the SSE custom event is great for live consumers but invisible to post-run audit. RunEventStore should carry its own row so operators can answer "which runs were safety-suppressed today?" from a single SQL query without joining the message body. Worker now exposes the run-scoped RunJournal via runtime.context["__run_journal"] (sentinel key, internal channel). SafetyFinishReasonMiddleware calls the previously-unused RunJournal.record_middleware, which emits event_type = "middleware:safety_termination" category = "middleware" content = {name, hook, action, changes={ detector, reason_field, reason_value, suppressed_tool_call_count, suppressed_tool_call_names, suppressed_tool_call_ids, message_id, extras}} Tool *arguments* are deliberately excluded — those are the very content the provider filtered and persisting them would defeat the purpose of the safety filter (per review note in #3035). Graceful skips when journal is absent (subagent runtime, unit tests, no-event-store local dev). Journal exceptions never propagate into the agent loop. Refs #3028 * fix(runtime): satisfy ruff format + address Copilot review - ruff format on safety_finish_reason_config.py and e2e demo (CI lint failed on ruff format --check; backend Makefile lint target runs ruff check AND ruff format --check). - Docstring on SafetyFinishReasonConfig now says resolve_variable to match the actual loader used in from_config (the wording was resolve_class previously; behavior is unchanged — resolve_variable mirrors how guardrails.provider is loaded). - Switch the AIMessage type check in SafetyFinishReasonMiddleware._apply from getattr(last, "type") == "ai" to isinstance(last, AIMessage), matching TokenUsageMiddleware / TodoMiddleware / ViewImageMiddleware / SummarizationMiddleware which are the dominant pattern. Refs #3028	2026-05-22 21:20:28 +08:00
Xinmin Zeng	df95154282	fix(tracing): propagate session_id and user_id into Langfuse traces (#2944) ... * fix(tracing): propagate session_id and user_id into Langfuse traces Adds Langfuse v4 reserved trace attributes (langfuse_session_id, langfuse_user_id, langfuse_trace_name, langfuse_tags) to RunnableConfig.metadata inside the run worker, so the langchain CallbackHandler can lift them onto the root trace. - New deerflow.tracing.metadata.build_langfuse_trace_metadata() returns the reserved keys when Langfuse is in the enabled providers, else {}. - worker.run_agent merges them with setdefault so caller-supplied keys win, allowing per-request overrides from upstream metadata. - session_id mirrors the LangGraph thread_id; user_id reads get_effective_user_id() (falls back to "default" in no-auth mode). - trace_name defaults to "lead-agent"; tags carry env and model name when DEER_FLOW_ENV (or ENVIRONMENT) and a model name are present. Closes #2930 * fix(tracing): attach Langfuse callback at graph root so metadata propagates The first commit injected ``langfuse_session_id`` / ``langfuse_user_id`` / ``langfuse_trace_name`` / ``langfuse_tags`` into ``RunnableConfig.metadata``, but on ``main`` the Langfuse callback is attached at *model* level (``models/factory.py``). LangChain still threads ``parent_run_id`` through the contextvar, so the handler sees the model as a nested observation and ``__on_llm_action`` strips the ``langfuse_*`` keys (``keep_langfuse_trace_attributes=False``). The trace's top-level ``sessionId`` / ``userId`` therefore stayed empty in deer-flow's LangGraph runtime — confirmed live against a real Langfuse instance. This commit moves the callback to the **graph invocation root** so the handler fires ``on_chain_start(parent_run_id=None)`` and runs the ``propagate_attributes`` path that actually lifts ``session_id`` / ``user_id`` onto the trace: - ``models/factory.py``: add ``attach_tracing`` keyword (default ``True``) so standalone callers (``MemoryUpdater``, etc.) keep their direct model-level tracing. - ``agents/lead_agent/agent.py``: call ``build_tracing_callbacks()`` once inside ``_make_lead_agent`` and append the result to ``config["callbacks"]``; the four in-graph ``create_chat_model`` sites (bootstrap, default agent, sync + async summarization) pass ``attach_tracing=False`` to avoid duplicate spans. - ``agents/middlewares/title_middleware.py``: same ``attach_tracing=False`` for the title-generation model, since it inherits the graph's RunnableConfig via ``_get_runnable_config``. Test updates: - ``tests/test_lead_agent_model_resolution.py`` and ``tests/test_title_middleware_core_logic.py``: extend the fake ``create_chat_model`` signatures / mock assertions to accept the new ``attach_tracing`` kwarg. - ``tests/test_worker_langfuse_metadata.py``: switch the no-user fallback test from direct ContextVar mutation to ``monkeypatch.setattr`` on ``get_effective_user_id`` to avoid pollution across the langfuse OTel global tracer provider. - ``tests/conftest.py``: add an autouse fixture that resets ``deerflow.config.title_config._title_config`` to its pristine default after every test. Any test that loads the real ``config.yaml`` (via ``get_app_config()``) calls ``load_title_config_from_dict`` and mutates the module-level singleton, which previously poisoned the title-middleware suite when run after, e.g., the new ``test_worker_langfuse_metadata.py`` cases. The fixture is independent of this PR's main change but unblocks the cross-file test run. Live verification (same Langfuse instance as before): - Drove ``worker.run_agent`` against the real ``make_lead_agent`` + ``gpt-4o-mini`` for three distinct ``user_context`` identities (``fancy-engineer``, ``alice-pm``, ``bob-designer``). - Each run produced one ``lead-agent`` trace whose top-level ``sessionId`` / ``userId`` / ``tags`` carry the expected values, e.g. ``session=e2e-2930-8f347c-alice-pm user=alice-pm name='lead-agent' tags=['model:gpt-4o-mini']``. Refs #2930. * fix(tracing): extend root-callback + metadata injection to the embedded client Addresses Copilot review on PR #2944. Commit 2 disabled model-level tracing for ``TitleMiddleware`` and ``_create_summarization_middleware`` because ``_make_lead_agent`` now attaches the tracing callbacks at the graph invocation root. But the embedded ``DeerFlowClient`` does not call ``_make_lead_agent`` — it calls ``_build_middlewares`` directly and never appends the tracing handlers to its ``RunnableConfig``. So under the embedded path, title-generation and summarization LLM calls were left untraced — a regression introduced by this PR. This commit mirrors the gateway worker's injection in ``DeerFlowClient.stream``: - Append ``build_tracing_callbacks()`` to ``config["callbacks"]`` so the Langfuse handler sees ``on_chain_start(parent_run_id=None)`` at the graph root and runs the ``propagate_attributes`` path. - Merge ``build_langfuse_trace_metadata(...)`` into ``config["metadata"]`` with ``setdefault`` so caller-supplied keys still win. - ``_ensure_agent`` now creates its main model with ``attach_tracing=False`` to avoid duplicate spans now that the callback lives at the graph root. Docs: - ``backend/CLAUDE.md`` Tracing section rewritten to describe the graph-root attachment model (replacing the inaccurate "at model-creation time" wording). - ``README.md`` Langfuse section now lists both injection points (worker + client) instead of only the worker path. Tests: - ``tests/test_client_langfuse_metadata.py`` (new, 3 cases): callbacks + metadata are injected when Langfuse is enabled, caller-supplied metadata overrides win via ``setdefault``, and the injection is inert when Langfuse is disabled. Live verification on the real Langfuse instance: === user=fancy-client === id=cbd22847.. session=client-2930-6b9491-fancy-client user=fancy-client name='lead-agent' === user=alice-client === id=b4f6f576.. session=client-2930-6b9491-alice-client user=alice-client name='lead-agent' Refs #2930. * refactor(tracing): address maintainer review on PR #2944 Addresses @WillemJiang's 5 comments. 1. Duplicated metadata-injection code between worker.py and client.py New ``deerflow.tracing.inject_langfuse_metadata(config, ...)`` helper takes the 10-line build + merge + setdefault logic that was duplicated in ``runtime/runs/worker.py`` and ``client.py``. Both callers now share a single source of truth, so the two paths cannot drift. 2. Direct private-attribute mutation in conftest.py and tests Added public ``reset_tracing_config()`` / ``reset_title_config()`` functions. ``tests/conftest.py`` and every test that previously did ``tracing_module._tracing_config = None`` or ``title_module._title_config = TitleConfig()`` now goes through the public API. A future internal rename will surface as an ImportError instead of a silent no-op. 3. client.py reading os.environ directly ``DeerFlowClient.__init__`` grows an optional ``environment`` parameter so programmatic callers can pass the deployment label explicitly. ``stream()`` consults ``self._environment`` first and only falls back to ``DEER_FLOW_ENV`` / ``ENVIRONMENT`` env vars when nothing was passed in. Backwards compatible — env-var behaviour preserved for callers that opt to keep using it. 4. build_tracing_callbacks() cached on hot path Not implemented. Inspected the langfuse v4 ``langchain.CallbackHandler`` constructor: it only resolves the module-level singleton client via ``get_client()`` and initialises a few dicts (no I/O, no env parsing at construction time). The build is essentially free. Caching would trade a non-measurable speedup for two real risks: handler instances carry per-run state internally (``_run_states``, ``_root_run_states``, ``last_trace_id``), and tracing config can be reloaded by env-var changes between runs. Will revisit if profiling ever shows it as a hot spot. 5. attach_tracing=False easy to forget at new in-graph call sites - Module docstring at the top of ``lead_agent/agent.py`` documents the invariant ("every in-graph ``create_chat_model`` MUST pass ``attach_tracing=False``") and enumerates the current sites. - New regression test ``test_make_lead_agent_attaches_tracing_callbacks_at_graph_root`` in ``tests/test_lead_agent_model_resolution.py`` locks both halves of the invariant: ``config["callbacks"]`` carries the tracing handler after ``_make_lead_agent``, AND every ``create_chat_model`` call captured by the test passes ``attach_tracing=False``. A future in-graph site that forgets the flag will fail this test. Lint clean. Full touched-suite bundle: 246 passed. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-05-21 16:49:31 +08:00