* feat(context): record effective memory identity per run
* fix(context): address memory identity review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(mcp): auto-promote deferred MCP tools from routing hints
When tool_search.enabled=true defers MCP tool schemas, PR1 routing hints
still require the model to spend a tool_search discovery round trip before
it can call the tool the routing metadata already points at. This adds a
McpRoutingMiddleware that matches the latest user message against PR1
routing keywords and promotes the matching deferred schemas before the
model call, removing that round trip.
Design (soft routing, opt-in, additive):
- Matches only the latest real HumanMessage (shared is_real_user_message
helper, reused by SkillActivationMiddleware so the two cannot drift);
case-insensitive substring match, no tokenizer dependency.
- Ordering: priority desc, then tool name asc; capped by the new global
tool_search.auto_promote_top_k (default 3, clamped 1..5). Does not add or
consume a per-tool auto_promote_top_k (PR1 schema unchanged); a per-tool
value is ignored with a DEBUG note.
- Returns a plain {"promoted": ...} state update (not a Command) and relies
on ThreadState.merge_promoted for union/dedupe, so auto-promote and a
model-triggered tool_search converge on the same catalog hash.
- Installed before DeferredToolFilterMiddleware on every deferred-tool path
(lead agent, subagent, embedded client, webhook via shared builders);
a construction-time assert rejects the reversed order. catalog_hash is
None / no routing index is a complete no-op, so bootstrap and ACP skip it.
- Privacy: never executes tools, never promotes policy-filtered tools, adds
no routing keywords or matched tool names to trace metadata or INFO/WARN
logs.
No behavior change when tool_search.enabled=false.
Tests: index construction, matching semantics, middleware state updates,
same-cycle deferred-filter interaction, lead/subagent/embedded-client
builder wiring + order invariant, config clamping, config.example.yaml
parseability, and privacy assertions.
* refactor(mcp): address auto-promote review nits
- executor: access app_config.tool_search.auto_promote_top_k directly to match
the lead-agent and embedded-client paths (drop the over-defensive getattr that
masked missing config); update the subagent test mock to carry tool_search.
- tool_search / mcp_routing_middleware: cross-reference the duplicated routing
priority/keyword normalization between the builder and the middleware's
defensive _normalize_index so they cannot silently drift.
- MCP_SERVER.md: document that auto-promote keyword matching is a case-insensitive
substring test (not word-boundary), advising distinctive keywords.
* Add phase 1 skill static scanning
* Rework SkillScan phase 1 as native scanner
* refactor(skillscan): align phase 1 with trimmed RFC contract
- SecurityFinding: 7 fields (rule_id, severity, file, line, message,
remediation, evidence); category/analyzer derive from the rule_id
prefix, confidence/column/fingerprint/metadata removed
- scan_archive_preflight()/scan_skill_dir() are pure functions: no
ScanContext, no policy schema; CRITICAL-blocks is a code constant and
skill_scan.enabled is applied by enforce_static_scan()/callers
- secret-* evidence is redacted before findings leave the scanner
- de-dup keys on (rule_id, file, line) so repeated occurrences keep
distinct locations for agent self-correction
- cloud-metadata detection consolidated into network-cloud-metadata
- nested zip members get a one-level stdlib magic-byte peek; an
executable member escalates package-nested-archive to CRITICAL
- install metadata sidecar removed (Phase 7 decides if it is needed)
- rule specs moved next to their analyzers; skillscan/rules/ removed
- tests updated + new anchors: redaction, dedup lines, nested-zip
escalation, single cloud-metadata rule, bundled-skill zero-CRITICAL
* fix(skillscan): tighten reverse-shell/secret/archive scan rules from review
Address PR #3033 review feedback on the native SkillScan analyzers:
- Reverse-shell false positives: split shell detection by signal strength
(/dev/tcp/, nc -e stay CRITICAL; bash -i, mkfifo -> new HIGH
shell-reverse-shell-heuristic, warn->LLM). The Python check is now
AST-anchored on real socket.socket/os.dup2/subprocess call sites instead
of raw-text substring matching, so prose/docstrings no longer hard-block.
- Secret evidence: _redact_secret_evidence returns [redacted] with no secret
bytes (was value[:6], which leaked 2 real token bytes past the prefix).
- Archive DoS: cap outer archive member count (_MAX_ARCHIVE_MEMBERS=4096);
scan_archive_preflight early-aborts with a package-too-many-members CRITICAL
finding (routes through the existing blocked->400 fail-closed path).
- shell-destructive-command: broaden the rm -rf matcher to sensitive system
roots (/home, /usr, /*, --no-preserve-root /) while leaving safe subpaths
unflagged.
- Dead code: collapse _decode_text_for_analysis to a single decode path and
drop the unused _TEXT_SUFFIXES set and _has_text_shebang helper.
- local_skill_storage: document why the host_path branch keeps app_config
possibly-None (lazy kill-switch resolution; avoids eager get_app_config in
config-free environments such as CI).
Tests: new negative/positive coverage in test_skillscan_native.py. Full
backend suite 6616 passed, 26 skipped.
* feat(community): add Crawl4AI web_fetch provider
Crawl4AI is a self-hosted, no-API-key web fetcher: it runs headless
Chromium and returns server-cleaned "fit" markdown directly via its
POST /md endpoint, so no client-side readability extraction is needed.
It sits alongside the existing self-hosted Browserless provider.
- deerflow.community.crawl4ai: async Crawl4AiClient + web_fetch_tool
(reads base_url/timeout_s/token/filter from config; "Error:" string
convention; 4096-char cap), mirroring the browserless provider
- tests: 17 unit cases (success, HTTP error, success:false, empty,
timeout, request error, token header, truncation, config reads)
- config.example.yaml: commented web_fetch example
- doctor: register as a no-key (free) web_fetch provider
- setup wizard: add to WEB_FETCH_PROVIDERS (no API key)
- docs: README, CONTRIBUTING, CONFIGURATION, AGENTS provider lists
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(community): address Crawl4AI provider review feedback
- timeout: robust _coerce_timeout (bool / non-numeric -> default) mirroring
jina, so 'timeout: off' no longer becomes 0.0 and times out every request
- read web_fetch config once per invocation and pass values into the client,
so a concurrent hot-reload can't split base_url from filter
- rename config key timeout_s -> timeout to match jina/infoquest (the
default providers); update config.example.yaml + setup wizard
- validate + normalize the markdown filter against {fit,raw,bm25,llm};
unknown values fall back to fit with a warning instead of an opaque HTTP 400
- client: a non-JSON 200 body (reverse proxy / auth wall) now reports the
content-type + snippet instead of a generic JSONDecodeError
- tests: 22 cases (added non-JSON-200, _coerce_timeout, _coerce_filter,
invalid-filter fallback, read-config-once)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: DanielWalnut <45447813+hetaoBackend@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(tui): add Hermes-like terminal workbench backed by DeerFlowClient
Implements the `deerflow` TUI from RFC #3540: a terminal-native, embedded
workbench over the existing harness (no Gateway/frontend/nginx/Docker), built
Python-native with Textual and learning UX patterns from tao-pi.
Architecture — every layer except the Textual app is pure and unit-tested:
- view_state.py: ViewState + reduce(state, action), the testable heart
- runtime.py: StreamEvent -> reducer actions (pure translate + threaded driver)
- message_format / command_registry / input_history / render / theme: pure
- app.py: Textual App; runs the sync DeerFlowClient.stream() on a worker thread
and marshals actions back to the UI thread. Slash command palette, model and
thread modal pickers, ↑/↓ history, Ctrl+C interrupt, TTY-aware fallback.
- cli.py: pure launch-mode planning + headless --print/--json + `deerflow`
console script (textual is an optional [tui] extra; degrades to headless help)
Web UI visibility (the RFC's key decision): persistence.py writes a threads_meta
row under the local default user into the same DB the Gateway reads, so terminal
sessions appear in the Web UI sidebar without running the Gateway. Best-effort,
no-op on the memory backend; all DB work on one long-lived background loop.
Tests: 95 TUI tests — pure layers via pytest, app/palette/overlays via Textual's
pilot harness with a fake session, and a threads_meta read/write round-trip.
ruff clean; respects the harness->app import boundary. Docs: backend/docs/TUI.md
plus CLAUDE.md/README updates and preview screenshots.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tui): de-duplicate streamed assistant text and tool cards; keep Tab in composer
Self-test surfaced three issues, all root-caused to consuming non-strict
streaming from DeerFlowClient (proven by the client's own
test_dedup_requires_messages_before_values_invariant, which shows the client can
re-emit a message id's full content twice):
- Assistant text was doubled (e.g. "answer answer") because the reducer blindly
concatenated same-id deltas. Now merges by content: a re-send or cumulative
snapshot replaces; only genuine increments append.
- Tool activity showed duplicate and empty "gear" cards from partial/re-emitted
tool-call chunks. ToolStarted now de-dupes by tool_call_id, drops id-less
noise chunks, and fills the name on a later chunk; a tool result with no prior
card still surfaces as a completed card.
- Tab moved focus off the composer to the scroll region (felt like broken cursor
logic). Tab is now consumed by the composer (completes a command when the
palette is open, no-op otherwise).
Adds reducer tests for each case plus a Tab-focus test; 102 TUI tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tui): make Esc interrupt an active run (matches the status hint)
The status line advertised "esc interrupt" but Esc was only wired to close the
slash palette, so it did nothing during a run. Esc now: closes the palette when
open, interrupts the active run when streaming, and is a no-op when idle. The
interrupt logic is shared with Ctrl+C via _interrupt_run(). Adds a regression
test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tui): stop prior answers duplicating on threads with history
On a thread with history, DeerFlowClient re-emits every prior message on each
new turn (its streamed_ids dedup is per-stream-call), and a re-emitted older
message can arrive after a newer message has already started. The reducer only
matched the *most recent* assistant row by id and otherwise appended, so each
re-emitted older answer was duplicated verbatim at the end of the transcript.
Match an assistant row by id anywhere in the transcript and merge in place.
Tool cards already de-dupe by call id globally, so they were unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tui): correct CJK cursor drift in the composer
Confirmed a Textual Input bug (latest 8.2.7): Input._cursor_offset adds an
unconditional +1 at the end of the value, overshooting by one cell after
double-width (CJK) characters. That misplaces the hardware/IME cursor — the
drift seen when typing Chinese in iTerm2 (the on-screen block cursor, drawn
separately in render_line, is fine; English doesn't use an IME so it looks
correct). Reproduced with a bare Input, so it's upstream, not our layout.
Add ComposerInput(Input) overriding _cursor_offset to the true cell position and
use it for the composer. Numeric tests pin the CJK end/mid and ASCII cases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tui): render finalized assistant messages as Markdown
The transcript showed raw Markdown (literal **bold**, ## headings, - lists,
links). Finalized assistant messages now render as Rich Markdown — headings,
bold/italic, lists, inline code + code blocks, blockquotes, horizontal rules and
links — with the ● speaker marker aligned to the top of the body.
The actively-streaming message stays plain text so partial Markdown doesn't
reflow/jump, then snaps to its rendered form when the run ends. Transcript
re-renders are coalesced on a ~60ms timer (dirty flag) so per-token Markdown
re-parsing stays smooth on long threads. Tests cover both the rendered and the
streaming-plain paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* style(tui): apply ruff format
CI lint runs `ruff format --check` via uvx (latest ruff); apply the formatter so
the lint-backend job passes. No behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(tui): address code-quality review comments
From github-code-quality[bot] on #3760:
- runtime.py: give the `_ClientLike` Protocol method a docstring body instead of
a bare `...` (flagged as a no-effect statement), matching the harness
convention for Protocol stubs (e.g. SafetyTerminationDetector).
- test_tui_cli_main.py: drop the unnecessary `lambda: _FakeSession()` wrappers in
monkeypatch.setattr; pass `_FakeSession` directly (same behavior).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tui): keep history Markdown-rendered when a follow-up run starts
Previously the transcript rendered "the last assistant row" as plain text while
streaming. But when a follow-up turn starts, the last assistant row is the
*previous, finalized* answer until the new message begins — and the client
re-emits prior messages early in the turn — so sending a follow-up reverted the
previous answer from rendered Markdown back to raw text.
Track the actively-streaming message id in ViewState instead: it's reset on
RunStarted, set only when an AssistantDelta actually adds new content (history
re-emits are no-ops and don't mark it), and cleared on RunEnded. The renderer
keeps only that one message plain; all history stays Markdown.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(readme): add Terminal Workbench (TUI) section to root README
Mention the new `deerflow` TUI alongside the Embedded Python Client in the root
README.md and README_zh.md (install, launch/headless commands, feature summary,
Web UI visibility), with a ToC entry and a preview screenshot. Links to
backend/docs/TUI.md for the full guide.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tui): address review feedback (willem-bd)
Ten findings from the TUI code review:
1. /resume was dead-ended — registered + in /help + tested as a builtin, but no
dispatch branch. Wired it to thread resolution / the switcher.
2. --resume <title> was forwarded raw into the checkpointer (blank thread).
Added Session.resolve_ref() to resolve id-or-title via list_threads; used by
--resume and /resume.
3. str(get("id","")) returned "None" for an explicit id:None (truthy), defeating
the empty-id guard so unrelated null-id tool calls collapsed into one card.
Coerce via a None-safe helper.
4. Headless --print/--json no longer spin up the persistence loop/engine/pool
(open_session(persistence=False)).
5. _LoopThread + engine are now closed: Session.close() (dispose engine + stop
loop) called from a try/finally around app.run().
6. --cli --continue (and piped --cli) now run headless instead of erroring.
7. Cancelled runs no longer persist a truncated title (guard on _cancelled).
8. Palette highlight resets to the top when the filter set changes.
9. Dropped the never-populated tools count from the header.
10. Documented the `not row.error` merge guard.
Adds regression tests for each; 126 TUI tests pass, ruff check + format clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(persistence): wire alembic migrations + bootstrap schema on startup
Closes#3682. Pre-#3658 DBs lack the `runs.token_usage_by_model` column
because alembic was never wired up — startup only ran `create_all`,
which never ALTERs existing tables.
Adds a hybrid bootstrap in FastAPI lifespan (replaces bare `create_all`):
- empty DB → create_all + stamp head
- legacy DB → stamp 0001_baseline + upgrade head
- versioned DB → upgrade head
Concurrency: Postgres `pg_advisory_lock` (cross-process); SQLite
per-engine `asyncio.Lock` + 30s `PRAGMA busy_timeout` on both prod and
alembic engines. Column revisions use `safe_add_column` /
`safe_drop_column` idempotent helpers as fallback.
Other bits:
- 0001 baseline (chain root) + 0002 add `runs.token_usage_by_model`
- `include_object` filter so alembic ignores LangGraph checkpointer tables
- `make migrate-rev MSG="..."` for authoring new revisions
(no migrate/stamp targets — startup is the only execution path)
- Tests: three-branch decision, concurrency, #3682 regression, env
filter, blocking-IO gate anchor
- CLAUDE.md: new "Schema migrations" section
* fix(style): fix lint error
* perf(persistence): address review feedback on alembic bootstrap
Behavioural fixes
- _SQLITE_LOCKS now keyed via WeakKeyDictionary so id-reuse after GC
cannot return a stale, loop-bound lock and the cache cannot leak one
entry per disposed engine.
- safe_add_column compares nullable / server_default against the desired
column when the name already exists and emits a warning on drift,
surfacing manual-ALTER workarounds instead of silently no-op'ing.
- _postgres_lock issues SET LOCAL idle_in_transaction_session_timeout=0
before pg_advisory_lock, so managed Postgres cannot kill the idle
lock-holding session mid-upgrade and silently release the advisory
lock.
- legacy branch now backfills missing baseline tables via a restricted
create_all (Base.metadata.create_all scoped to _BASELINE_TABLE_NAMES).
Restores pre-#1930 upgraders whose channel_* tables were never
provisioned, without pre-empting future create_table revisions for
newly-added models.
Schema parity
- runs.token_usage_by_model gains server_default=text("'{}'") in both
the ORM model and the 0001_baseline create_table, matching what 0002
adds via ALTER. create_all and alembic-upgrade paths now produce
identical column definitions.
- New parity test compares Base.metadata.create_all output against a
pure alembic upgrade base->head, asserting column-set, nullable, and
server_default agree across all tables (normalized through the same
helper safe_add_column's drift check uses).
Guards
- test_baseline_table_names_constant_matches_0001 pins
_BASELINE_TABLE_NAMES to 0001_baseline.upgrade()'s actual output --
the constant cannot drift silently when someone edits 0001.
- test_legacy_backfill_skips_non_baseline_tables verifies the restricted
backfill does not create a phantom table on Base.metadata, modelling
a future revision that would otherwise collide on op.create_table.
Doc residuals
- Three-branch decision table is now consistent across bootstrap.py
top docstring, engine.py comment, test module docstring, and
CLAUDE.md.
- Stale test anchor in blocking_io/test_persistence_engine_sqlite.py
docstring now points at the real file.
* fix(style): fix lint error
* fix(persistence): close drift detection holes
- _check_column_drift compares column type via a family equivalence
allowlist ({JSON, JSONB}). Catches the wrong-type workaround
`TEXT NOT NULL DEFAULT '{}'` that previously slipped through silently,
while keeping Postgres JSON/JSONB dialect reflection quiet. Reflected
and desired type are also echoed in every drift warning's payload for
operator triage.
- Extract _escape_url_for_alembic so bootstrap._alembic_safe_url and
scripts/_autogen_revision share the ConfigParser % escape rule
instead of duplicating it.
- backend/README.md: add `make migrate-rev MSG=...` to Commands and a
Schema Migrations section per the repo's README/CLAUDE.md sync policy.
- test_base_to_dict.py: scope the test ORM class to an isolated MetaData
so the create_all-vs-alembic parity test (added in the previous
commit) is not polluted by the phantom table on the full pytest
session.
* Fix stale AIO sandbox cache reuse
* Address AIO sandbox review feedback
* Distinguish sandbox health check failures
* Keep local discovery recoverable when the runtime check fails
LocalContainerBackend.discover() shares _is_container_running, which now
raises on transient daemon errors instead of returning False. Discovery has
no exception handling in _discover_or_create_with_lock(_async), so a brief
Docker hiccup turned a recoverable "could not verify, create instead" into a
hard acquire failure. Catch the check failure inside discover() and return
None so an unverifiable container is simply not adopted, restoring the
pre-change fall-through while keeping raise-on-unknown semantics protecting
the destroy path.
Reported by fancy-agent on PR #3494.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Narrow the not-found match in container inspect error handling
A bare "not found" substring also matches transient failures like "command
not found" or "context not found", which would misclassify a check error as
"container definitely gone" and bypass the raise-on-unknown contract. Keep
Docker's specific "No such object"/"No such container" phrases, and only
trust a generic "not found" (Apple Container) when the message names the
inspected container or refers to a container/object.
Reported by WillemJiang on PR #3494.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(detectors): add static blocking IO inventory
* refactor(detectors): drop superseded runtime probe; clarify static report path
- Remove the #2924 custom runtime blocking IO probe entirely:
backend/tests/support/detectors/blocking_io.py,
backend/tests/test_blocking_io_detector.py,
backend/tests/test_blocking_io_probe_integration.py, and the
pytest_addoption / pytest_runtest_call / pytest_runtest_teardown /
pytest_sessionfinish / pytest_terminal_summary hooks plus the
blocking_io_detector fixture from backend/tests/conftest.py.
Its narrow DEFAULT_BLOCKING_CALL_SPECS (time.sleep, requests, httpx,
os.walk, Path.resolve, Path.read_text, Path.write_text) cannot serve
as a CI gate; a Blockbuster-backed runtime detector will land in a
separate follow-up PR. Leaving the half-coverage probe alongside
the static inventory in this PR added a redundant detect path with
no production value.
- Address Copilot review comments on backend/README.md and
backend/CLAUDE.md by stating explicitly that the JSON report writes
to .deer-flow/blocking-io-findings.json at the repository root,
whether the target is invoked from the repo root or from backend/.
Verified: pytest tests/test_detect_blocking_io_static.py (18 passed),
ruff check + format on touched files (passed), make detect-blocking-io
from both repo root and backend/ produce the same 105-finding report
at <repo-root>/.deer-flow/blocking-io-findings.json.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(nginx): defer cors to gateway allowlist
Remove proxy-level wildcard CORS handling so browser origins are controlled by the Gateway allowlist and stay aligned with CSRF origin checks.
* docs: document gateway cors allowlist
Clarify that same-origin nginx access needs no CORS headers while split-origin or port-forwarded browser clients must opt in with GATEWAY_CORS_ORIGINS.
* docs(gateway): record cors source of truth
Document that Gateway CORSMiddleware and CSRFMiddleware share GATEWAY_CORS_ORIGINS as the split-origin source of truth.
* fix(gateway): align cors origin normalization
* docs: clarify gateway langgraph routing
* docs(gateway): update runtime routing note
* fix: make tool argument behavior discoverable
The write_file tool already supported append=false by default with append=true for end-of-file writes, but the parsed docstring did not describe append in the model-facing schema. This records the overwrite default and append path in the tool description, adds resilient schema regression coverage, and keeps backend sandbox docs aligned.
The regression now also checks that every public parameter in the existing tool schema test matrix has a description. Enabling docstring parsing on setup_agent and update_agent fills the two existing gaps with their existing Args docs instead of duplicating descriptions elsewhere.
Constraint: Issue #2831 asks for a small docstring/schema discoverability fix without changing runtime file-writing behavior
Rejected: Changing write_file defaults | would alter existing overwrite semantics and broaden the fix beyond schema discoverability
Rejected: Exact phrase assertions | too brittle for future docstring rewording while testing the same behavior
Confidence: high
Scope-risk: narrow
Directive: Keep model-facing tool parameters documented through parsed docstrings or equivalent schema descriptions
Tested: cd backend && uv run pytest tests/test_setup_agent_tool.py tests/test_update_agent_tool.py tests/test_tool_args_schema_no_pydantic_warning.py tests/test_sandbox_tools_security.py::test_str_replace_and_append_on_same_path_should_preserve_both_updates -q
Tested: cd backend && uv run ruff check packages/harness/deerflow/sandbox/tools.py packages/harness/deerflow/tools/builtins/setup_agent_tool.py packages/harness/deerflow/tools/builtins/update_agent_tool.py tests/test_tool_args_schema_no_pydantic_warning.py
Not-tested: Full backend test suite
Co-authored-by: OmX <omx@oh-my-codex.dev>
* Fix the lint error
---------
Co-authored-by: OmX <omx@oh-my-codex.dev>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: prevent concurrent subagent file write conflicts
Serialize same-path str_replace operations in sandbox tools
Guard AioSandbox write_file/update_file with the existing sandbox lock
Add regression tests for concurrent str_replace and append races
Verify with backend full tests and ruff lint checks
* fix(sandbox): Fix the concurrency issue of file operations on the same path in isolated sandboxes.
Ensure that different sandbox instances use independent locks for file operations on the same virtual path to avoid concurrency conflicts. Change the lock key from a single path to a composite key of (sandbox.id, path), and add tests to verify the concurrent safety of isolated sandboxes.
* feat(sandbox): Extract file operation lock logic to standalone module and fix concurrency issues
Extract file operation lock related logic from tools.py into a separate file_operation_lock.py module.
Fix data race issues during concurrent str_replace and write_file operations.
* fix(security): disable host bash by default in local sandbox
* fix(security): address review feedback for local bash hardening
* fix(ci): sort live test imports for lint
* style: apply backend formatter
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Add LangSmith tracing setup instructions across the project:
- .env.example: add LANGSMITH_* env vars (commented out)
- README.md + translations (zh/ja/fr/ru): add LangSmith Tracing section
under Advanced with setup steps and env var reference
- backend/README.md: add detailed LangSmith Tracing section with setup,
env var table, how-it-works explanation, and Docker notes
- docker-compose.yaml: update LANGCHAIN_TRACING_V2 to LANGSMITH_TRACING
for naming consistency with the rest of the project
Made-with: Cursor
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(threads): clean up local thread data after thread deletion
Delete DeerFlow-managed thread directories after the web UI removes a LangGraph thread.
This keeps local thread data in sync with conversation deletion and adds regression coverage for the cleanup flow.
* fix(threads): address thread cleanup review feedback
Encode thread cleanup URLs in the web client, keep cache updates explicit when no thread search data is cached, and return a generic 500 response from the cleanup endpoint while documenting the sanitized error behavior.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(feishu): stream updates on a single card
* fix(feishu): ensure final message on stream error and warn on missing card ID
- Wrap streaming loop in try/except/finally so a is_final=True outbound
message is always published, even when the LangGraph stream breaks
mid-way. This prevents _running_card_ids memory leaks and ensures the
Feishu card shows a DONE reaction instead of hanging on "Working on it".
- Log a warning when _ensure_running_card gets no message_id back from
the Feishu reply API, making silent fallback to new-card behavior
visible in logs.
- Add test_handle_feishu_stream_error_still_sends_final to cover the
error path.
- Reformat service.py dict comprehension (ruff format, no logic change).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Avoid blocking inbound on Feishu card creation
---------
Co-authored-by: songyaolun <songyaolun@bytedance.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
add oauth schema to MCP server config (extensions_config.json)
support client_credentials and refresh_token grants
implement token manager with caching and pre-expiry refresh
inject OAuth Authorization header for MCP tool discovery and tool calls
extend MCP gateway config models to read/write OAuth settings
update docs and examples for OAuth configuration
add unit tests for token fetch/cache and header injection
Updated documentation to accurately cover all backend subsystems including
subagents, memory, middleware chain, sandbox, MCP, skills, and gateway API.
Fixed broken MCP_SETUP.md link in root README.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add README.md with project overview, quick start, and API reference
- Add CONTRIBUTING.md with development setup and contribution guidelines
- Add docs/ARCHITECTURE.md with detailed system architecture diagrams
- Add docs/API.md with complete API reference for LangGraph and Gateway
- Add docs/README.md as documentation index
- Update CLAUDE.md with improved structure and new features
- Update docs/TODO.md to reflect current status
- Update pyproject.toml description
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
