1875 Commits

Author SHA1 Message Date
Evan
797e093ddc
fix(sandbox): require a path start before translating virtual and host roots (#7582)
* fix(sandbox): require a path start before translating virtual and host roots

Path translation only checked the right-hand segment boundary (#4035,
#4053), so a root embedded in a longer path or URL was rewritten too.
DeerFlow's own artifact URLs contain /mnt/user-data, so
curl .../artifacts/mnt/user-data/outputs/x.html had the thread's host
directory spliced into the URL, and a mount at /data turned an
unrelated /var/data/app.log into /var/mnt/data/app.log in model-visible
output.

Add a shared left-hand PATH_START_BOUNDARY: a root must not follow a
word character, '.', '~' or '-'. '/' is still allowed so
file:///mnt/user-data keeps mapping. It applies to the bash command
rewrite, LocalSandbox command/content resolution, and both reverse
masking paths.

* refactor(sandbox): derive the path-start lookbehind from one class

Build PATH_START_BOUNDARY from _PATH_START_BLOCKERS so the regex and
scanner paths cannot drift, and document the attached short-option
value (-o/data/x) as a known false negative of the start rule.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-27 08:44:29 +08:00
SumEn
f7d02a7515
fix(checkpoints): validate cached delta history before replay (#7548)
* fix(checkpoints): reject malformed Redis history write records

Validate pending-write tuples before optional Redis history entries reach
delta materialization. Treat the whole malformed entry as a miss so the
authoritative saver rebuilds it without dropping or inventing state.

Cover actual checkpoint reads, whole-entry replacement, valid tuple/list
records, neighbor preservation and warm-cache rereads.

* fix(checkpoints): bind cached history writes to their channel

* fix(checkpoints): keep cache warning failures best-effort

Guard warning handlers across decode, ingestion, serialization, Redis outages and purge. Preserve authoritative rebuilds and valid neighboring writes, with native graph regression coverage.

* docs(gateway): keep guidance below the hard size budget

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-27 00:28:03 +08:00
zenghao
c5193d26c7
fix(docker): bundle Lark broker runtime helpers (#7542)
* fix(docker): bundle Lark broker runtime helpers

* docs(lark): document standalone broker import dependencies
2026-10-27 00:19:23 +08:00
spud
aabfdeebc9
fix(streaming): make Redis stream END publication idempotent (#7572)
* fix(streaming): make Redis stream END publication idempotent

* fix(streaming): align Redis test fake with Lua END checks

Remove the unreachable owned missing-stream branch, mirror tail-only data checks in both fake script paths, and pin legacy END and owned missing-buffer behavior against real Redis.

* fix(streaming): gate legacy END scans after writer rollout

* fix(helm): synchronize embedded config version with schema

* test(runtime): use real stream bridge config in close coverage

Keep new configuration defaults in lifecycle fixtures and surface owner-task failures instead of masking them as event timeouts. Drain every helper task and cover context entry/close failures.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-27 00:16:53 +08:00
Hyeonsang Cho
c46906aa41
fix(community): resolve async tool config off the event loop (#7580)
* fix(community): resolve async tool config off the event loop

Agent runs execute on the Gateway event loop, and the async community
tools resolved their tool config through get_app_config() inline. That
call stats and hashes config.yaml and extensions_config.json on every
call and reparses them after an edit, so each tool call held the loop
for it, up to three times per call: Tavily, SearXNG and Firecrawl read
the config again to build their client, InfoQuest read it three times,
and the browser tools read the launch config for every action.

Resolve it through asyncio.to_thread once per call, as image_search and
groundroute already do, and build the client from that same read. This
covers jina_ai, crawl4ai, browserless, tavily, searxng, firecrawl,
infoquest, ragflow, lightrag and browser_automation.

* docs(changelog): reference #7580 for the off-loop community tool config reads

* docs(changelog): use the existing community-tools label in the zh entry for #7580

* fix(community): resolve the jsonify dataset_rows config off the event loop

dataset_rows_tool is async and called _load_settings() inline, so every
call ran get_app_config()'s stat + sha256 revalidation on the loop.
Read it through asyncio.to_thread like the other async community tools,
and add the tool to the off-loop anchors.

* docs(changelog): include Jsonify in #7580 and drop trailing whitespace

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-26 23:40:32 +08:00
llb555
19768d73cc
fix(setup-agent): keep a failed discard from replacing the publish outcome (#7564)
* fix(setup-agent): keep a failed discard from replacing the publish outcome

* test(setup-agent): pin the post-publish discard and assert the real temp name

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-26 23:33:49 +08:00
chaodidiao
6e4bcc16aa
fix(subagents): drain final cleanup across caller cancellation (#7546)
* fix(subagents): drain final cleanup across caller cancellation

* fix(subagents): preserve stop outcomes during cancellation

Follow up on PR #7546 discussion_r4374235193 and Issue #7535.

Capture the unwinding cancellation before the drained cleanup tail and use it
only while the result remains non-terminal. Preserve completed, failed and
timed-out decisions. Keep cancellation propagation and cleanup ownership.

Share the synchronous result holder and publish its existing FAILED outcome
before requesting cancellation. This necessary timeout boundary overlaps #7483;
no active-cancel terminal write is added to the cleanup tail.

Validation: 12 targeted regressions pass on Windows and Linux; make test
37876 passed / 808 skipped, make test-blocking-io 292 passed / 1 skipped.
Format, lint, diff checks and four-file incremental OCR delegated review pass.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-26 23:04:45 +08:00
haotianqi666
4f6ee59535
fix(gateway): preserve null input for static continuation (#7578)
* fix(gateway): preserve null input for static continuation

* fix(runtime): preserve continuation inputs during MCP projection

---------

Co-authored-by: haotianqi666 <250384991+haotianqi666@users.noreply.github.com>
2026-10-26 21:32:25 +08:00
Starchaser1024
8d3af33f8e
fix(skills): keep a required secret when an optional duplicate precedes it (#7569)
* fix(skills): keep a required secret when an optional duplicate precedes it

parse_required_secrets dropped every declaration of a name after the first,
so an earlier 'optional: true' entry made a later required declaration
unreachable and the skill ran without the secret. Duplicates now keep the
first position but a required declaration upgrades the entry, and the export
manifest applies the same rule.

* test(skills): cover duplicate secret order and whitespace

Port the three parameterized ordering and whitespace cases from PR #7570 while retaining #7569's full-YAML, optional-only, and export regressions.

* refactor(skills): keep the export optional read fail-closed

export.py read `optional` with `bool(secret.get("optional", False))`, so a
truthy string would have become optional=True — the opposite of
parse_required_secrets, which treats a non-boolean value as required
(#5738). validate_skill_frontmatter_text already refuses such a package
before the dedup loop (installation applies the same rule), so that
coercion is unreachable today; read the flag as `is True` anyway so the
two surfaces cannot drift if the gate ever moves.

Pin the parity from both sides: the secrets suite asserts that the same
document is rejected by validation and read as required by the parser, and
the export suite asserts a manifest with skill_export_invalid_frontmatter
and no requirements instead of a coerced optional=True.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-26 21:25:46 +08:00
Nan Gao
a684ae2fe3
fix(sandbox): leave SIGTERM/SIGINT teardown to the Gateway lifespan (#7560)
* fix(sandbox): leave SIGTERM/SIGINT teardown to the Gateway lifespan

AIO and E2B providers built on the event-loop thread installed SIGTERM/SIGINT
handlers that force-destroyed every sandbox synchronously, before uvicorn
started the run drain. The Gateway lifespan now holds sandbox shutdown
ownership from its first step, so providers skip those handlers and the
existing hook closes the provider after run drain and MCP teardown. SIGHUP
keeps the provider fallback. AIO acquire now refuses once shutdown began.

* docs(sandbox): document Gateway-owned sandbox shutdown and AIO admission fence

* test(gateway): inspect the lifespan body now that lifespan wraps it

* docs(changelog): reference #7560 for the sandbox SIGTERM drain-order fix

* ci: retrigger sandbox image smoke after a registry pull timeout
2026-10-26 20:38:32 +08:00
Cordis798
09a98e5f7b
fix(memory): honor Mem0 read policies during context deduplication (#7561)
Co-authored-by: Cordis798 <284918518+Cordis798@users.noreply.github.com>
2026-10-26 19:44:01 +08:00
Felix Shi
d53ec767af
fix(persistence): keep LF bytes in skill and agent text writes (#7501)
* fix(persistence): keep LF bytes in skill and agent text writes

Text mode translates a newline into os.linesep on write, so on Windows every
caller-supplied LF reached disk as CRLF while readers use universal newlines or
decode bytes explicitly. The difference stayed invisible: the API returned LF,
the file held CRLF, and a skill history record's prev_content described a
revision no API view could return (#7174).

Pass newline="" at the ten writers that persist or judge caller text: the local
and user-scoped skill writers, the skill history JSONL, the user skill-states
JSON, the SOUL.md/config.yaml staging in FileAgentStore, the managed-subagent
store, setup_agent's SOUL.md publish, the USER.md route, and the two temp copies
the frontmatter validator and the static scanner judge.

The tests emulate the Windows translation on io.open, so the regression runs on
the Linux CI shards instead of only on a Windows host: 10 failed before this
commit, 10 pass after.

* fix(skills): preserve candidate newlines during static scan

---------

Co-authored-by: Felix Shi <204130665+shi-felix@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-26 18:45:49 +08:00
Hyeonsang Cho
ddfa45eaaf
fix(gateway): read the route authorization config off the event loop (#7557)
* fix(gateway): read the route authorization config off the event loop

AuthMiddleware resolves route permissions for every authenticated
request, and resolve_route_permissions read the hot-reloaded
authorization config inline. get_app_config() stats and hashes
config.yaml and extensions_config.json on every call and reparses them
after an edit, so that file IO ran on the event loop per request.

Read it through asyncio.to_thread, like the plugin authorization path
already does. The context copy keeps runtime config overrides visible.

* docs(changelog): reference #7557 for the off-loop route authorization config read

* docs(config): note the get_app_config read cost where it is documented

Move the off-loop note from gateway/AGENTS.md, which is 12 bytes under
its hard limit, to the Config Caching section that documents what each
get_app_config() read costs.
2026-10-26 18:44:09 +08:00
lilei
04dc9e5339
fix(subagents): report active cancellation to lifecycle observers (#7483)
* fix(subagents): report active cancellation to lifecycle observers

* fix(subagents): preserve synchronous timeout lifecycle failures
2026-10-26 17:37:49 +08:00
llb555
794be98704
fix(client): keep a decided batch outcome through capture cleanup (#7517) 2026-10-26 17:03:45 +08:00
Xinmin Zeng
d48ea2a9a3
refactor: remove unused proxy policy and private constants (#7554)
* refactor: remove unused proxy policy and private constants

* refactor(sandbox): share provisioner runtime mount path
2026-10-26 17:02:39 +08:00
Qiu
370da43119
fix(utils): construct one-shot models off the event loop (#7484)
* fix(utils): construct one-shot models off the event loop

* docs(utils): document all one-shot callers and offload scope

Project-file summaries use the shared one-shot helper as well as input
polishing and follow-up suggestions. Include that third caller in the
README and utility guidance, and distinguish the separate skill scanner
and goal evaluator constructors.

Runtime and tests are unchanged. The existing exact-head strict and caller
regressions pass together: 80 passed on Windows with Python 3.12.14.
2026-10-26 15:41:24 +08:00
Hyeonsang Cho
30b3a012cb
fix(memory): flush pre-compaction memory off the event loop (#7530)
* fix(memory): flush pre-compaction memory off the event loop

The async compaction path fired the before-summarization hooks inline.
memory_flush_hook hands the summarized messages to add_nowait, which is
sync HTTP for mem0 and Honcho (inherited default -> add) and OpenViking
(its own override), so every automatic compaction with an HTTP memory
backend blocked the Gateway event loop for the whole request.

acompact_state now runs the hooks through asyncio.to_thread, as
MemoryMiddleware.aafter_agent already does for its writes. Ordering,
failure isolation, summary-first timing and context propagation are
unchanged; the sync compact_state path is untouched.

* docs(changelog): reference #7530 for the pre-compaction memory flush fix

* test(memory): anchor the OpenViking pre-compaction flush off the event loop

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-26 15:13:00 +08:00
jianyx1
4b1e06bbf9
feat(runtime): honor the web UI language in replies and DeerMem updates (#7294)
* feat(runtime): honor the web UI language in replies and DeerMem updates

Carry Settings > Appearance > Language as request-only run data: the
Gateway validates context.locale against a finite allowlist, the lead
agent prefers it for replies, and bundled DeerMem writes new facts and
summary prose in it. Explicit user language requests still take
precedence; checkpoint configurable, static prompts, and memory backend
APIs are unchanged.

Refs #7287

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(runtime): guard language context and pin locale maps

* test(runtime): pin UI locales across frontend and Gateway

Add the shared locale fixture and document the intentional language fallback for internal runs. Clarify live-locale precedence in the run context contract.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-26 13:24:30 +08:00
Xinmin Zeng
5010a74a03
chore: remove unused frontend components and RAGFlow helper (#7543) 2026-10-26 13:17:06 +08:00
Willem Jiang
3d5107dc0d
fix(ci): keep runtime guidance within the chain budget (#7522)
Move checkpoint modes, replay, module maps and terminal-ownership details intact into the existing runtime reference. Keep concise invariants and explicit links in AGENTS.md, reducing the effective chain from 99,004 to 80,817 bytes. The existing guidance regression now passes without changing budget thresholds.
2026-10-26 10:19:30 +08:00
Aling33
01815467aa
fix(projects): delete read-only documents when purging trash (#7499)
* fix(projects): delete read-only documents when purging trash

A permanently deleted project document could never be purged on Windows:
the read-only file attribute makes `Path.unlink` fail with `PermissionError`
(`WinError 5`), where POSIX needs only a writable parent. A restored archive
or a mode-preserving sync sets that attribute, and because the purge re-raises,
the row rolls back and the trashed document keeps coming back with the same
error on every retry.

Clear the attribute and retry the unlink once, mirroring the retry that
`config/paths.py::_clear_read_only_tree` already performs for thread
directories. Any other unlink error still propagates, so the row stays
retryable.

* fix(projects): harden trash purge permission retries

---------

Co-authored-by: Aling33 <along308@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-26 08:31:47 +08:00
Luck
ce0fa4c114
feat(batch): add browser review for persisted results (#7006)
* feat(batch): add browser review for persisted results

* fix(batch): address browser review feedback

* test(batch): cover persisted result review regressions

Exercise running-to-terminal detail refresh, scoped citation dialogs, omitted evidence, translated page labels, and legacy evidence bounds. Fix component lint and formatting and synchronize the review contract documentation.

* docs(frontend): keep batch review guidance within budget

* fix(batch): refresh retried reviews and account for omitted evidence

* fix(ci): keep batch review guidance within size budget

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 23:27:35 +08:00
Hyeonsang Cho
e64bc8a3b5
fix(sandbox): derive the AIO sandbox ID off the event loop in async acquire (#7507)
* fix(sandbox): derive the AIO sandbox ID off the event loop in async acquire

_acquire_internal_async offloads every acquire layer except the sandbox ID
derivation, which ran inline. _sandbox_id_for_thread checks the thread's
skills_view directory with exists(), and resolving it goes through
Paths.base_dir, so each async thread acquire made filesystem syscalls on the
event loop. The strict blocking-I/O gate raises on it.

Run the derivation through the same owned-worker helper as the reuse and
warm-pool layers around it, and anchor it in the blocking-I/O suite for both
the default and the policy-scoped ID.

* docs(changelog): reference #7507 for the AIO async acquire sandbox ID fix

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 23:22:24 +08:00
spud
33f7b43400
fix(runtime): backfill run.terminal after confirmed recovery takeover (#7500)
* fix(runtime): backfill run.terminal on confirmed recovery takeover

* fix(runtime): isolate recovery terminal publication failures

* fix(runtime): fence recovery terminal publication lifecycle

* fix(runtime): unify recovery receipt publication

* docs(runtime): document lifecycle guard lock constraints

* fix(runtime): reuse event transaction for recovery guards

Read Run lifecycle through the active event session when SQL stores share an engine, preserving deletion fencing without a second pool checkout. Keep custom lookups and different engines on their ordinary read path. Scope memory recovery guards and deletions to per-thread locks. Add SQLite/PostgreSQL pool pressure, stale identity, fallback, and independent-thread regressions, and document the contracts.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 23:22:02 +08:00
jay
f6da662c68
fix(projects): make read_file_chunks cancellation-safe for worker handoffs (#7496)
* fix(projects): make read_file_chunks cancellation-safe for worker handoffs

Cancelling a project document read had three unsafe windows around the
dedicated file-I/O worker: (1) a cancellation landing between the open
worker completing and the result reaching the coroutine leaked the
handle (the cleanup path never saw it); (2) the cleanup close could
start while an admitted read was still running; (3) a repeated
cancellation let the caller escape before the close worker settled
(#7491).

Gate every admitted operation with the existing await_drained
primitive: cancellation is delivered only after the operation settles,
the open result is retained in a list cell the moment the worker
returns it (so the cleanup close always has the handle), and waiting
for the next chunk via the yield stays immediately cancellable.

Regression tests gate the real executor with asyncio events at the
worker boundary (real files, no mocks): all three windows are red on
unmodified main and green with the fix; the happy path is unchanged.
The dispatch-spy promotion tests that import app.gateway.routers need
the full backend dependency set and could not run in the minimal local
env; they fail identically on unmodified main (verified).

Fixes #7491

Co-authored-by: Jay Feng <jayfeng77@outlook.com>

* test: fix _gated_offload return annotation to the unpacked tuple

Review nit: the helper returns (calls, gated) and every call site
unpacks it as a pair, but the annotation claimed list[str]. Type it as
the tuple it actually returns.

Co-authored-by: Jay Feng <jayfeng77@outlook.com>

* test: pin missing-path FileNotFoundError in read_file_chunks

Convergence item from the duplicate-pair verdict (#7493): the
open-failure path must surface FileNotFoundError as-is — the cleanup
finally skips the close because retained is empty (no handle was ever
opened), rather than raising AttributeError from closing None.

Co-authored-by: Jay Feng <jayfeng77@outlook.com>

---------

Co-authored-by: Jay Feng <jayfeng77@outlook.com>
2026-10-25 23:02:38 +08:00
Dragon ze
9a92d2e9b7
fix(persistence): treat a peer's database creation as success (#7489)
Two Gateways booting the same missing database both reach CREATE DATABASE and
only one wins; the loser aborted startup with duplicate_database (42P04) even
though the winner's catalog row was already committed and the target was
usable. Follow the SQLSTATE, like the missing-database check beside it does, so
the loser falls through to its retry. Every other failure -- permission denied
included -- still propagates untouched.
2026-10-25 23:00:40 +08:00
llb555
885062e197
fix(gateway,integrations): keep temp cleanup from replacing the caller's outcome (#7505)
* fix(gateway,integrations): keep temp cleanup from replacing the caller's outcome

* fix(gateway): adopt the family helper so a published edit stays silent
2026-10-25 22:13:26 +08:00
huyengiang101086-spec
79399d0c22
fix(uploads): invalidate edited markdown companions (#7348)
* fix(uploads): include markdown modification time in ownership records

* test(uploads): reject same-size edited companions on Windows

* fix(uploads): distinguish legacy companion records

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 22:08:31 +08:00
SumEn
d9e6341e5b
fix(checkpoints): isolate cached history payloads from state readers (#7259)
* fix(checkpoints): isolate cached history payloads from state readers

Copy nested snapshot seeds and pending-write payloads at both cache boundaries.
Keep user copy callbacks outside the registry lock so readers cannot poison
later checkpoint materialization or deadlock by re-entering the cache.

Cover sync/async state accessor reads, nested message payloads, disabled-cache
controls and callback lock ownership.

* docs(checkpoints): document synchronous history copy costs

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 22:00:14 +08:00
lester076
9d0509925e
fix(sandbox): reject write_file payloads that are the elision placeholder (#7374)
* fix(sandbox): reject write_file payloads that are the elision placeholder

The tool-output budget middleware replaces a superseded write_file payload
with a deterministic placeholder in the model-bound request only. That
placeholder is text the model sees in its own history, so it can imitate it
and submit it back as real write_file content: #7319 reports a delivered
file that was 194 bytes of placeholder.

write_file now rejects a payload that is the placeholder (whole payload,
allowing only surrounding whitespace) with an actionable error telling the
model to read_file the target path and write the real content; nothing is
written. The match is whole-payload on purpose: a substring guard would
block legitimate edits to this repository, whose constants.py and
persistence docs quote the template verbatim.

The placeholder wording moves to one shared constant,
WRITE_ELISION_PLACEHOLDER_TEMPLATE in deerflow.constants, which the
middleware renders from and the guard matches on, so the two cannot drift.

Regression tests fail on base (7 failed, 6 passed) and pass with the fix;
near-miss payloads that merely quote the placeholder still write.

Refs #7319

* fix(sandbox): match complete write elision placeholders

* docs: clarify write placeholder validation scope

* chore: re-trigger CLA status check

* fix(sandbox): preflight async writes and trim guidance

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 21:15:14 +08:00
llb555
95ffb29e50
fix(config): keep atomic-write cleanup from replacing the caller's outcome (#7279)
* fix(config): keep atomic-write cleanup from replacing the caller's outcome

* docs(config): state the cleanup invariant for every exit path
2026-10-25 19:23:44 +08:00
llb555
6b918bbe9a
fix(persistence): keep staged-write cleanup best-effort (#7485) 2026-10-25 19:23:13 +08:00
Kevin Dong
6c705f73e7
fix(middleware): preserve mounted output permissions on blob restore (#7390) 2026-10-25 19:22:11 +08:00
lukyturnflow
aa6fb7b67f
fix(models): preserve Moonshot cache_write_tokens in usage metadata (#7428)
* fix(models): preserve Moonshot cache_write_tokens in usage metadata

langchain-openai 1.2.1's _create_usage_metadata reads
prompt_tokens_details.cached_tokens but not cache_write_tokens, so the
PatchedChatDeepSeek conversion dropped the cache-creation count that Kimi K3.1
reports, and DeerFlow's usage journal never saw it (#7426).

PatchedChatDeepSeek now overrides _create_chat_result to restore the raw SDK
cache_write_tokens as input_token_details["cache_creation"].

Offline regression: a real OpenAI ChatCompletion carrying
prompt_tokens_details {cached_tokens: 100, cache_write_tokens: 256} converts to
input_token_details {cache_read: 100, cache_creation: 256} (red before, green
after); the existing 14 patched_deepseek tests and the managed-provider suites
stay green.

* fix(models): restore cache_write_tokens on the streaming path too

Streamed runs (web UI messages-tuple) build usage metadata in
_convert_chunk_to_generation_chunk -> _create_usage_metadata, which ignores
cache_write_tokens, so the non-streaming-only override still lost the count on
the path streamed runs actually take (review follow-up on #7428).

_convert_chunk_to_generation_chunk now mirrors _create_chat_result's restore
for the final usage chunk. Note chunk["usage"] is JSON null on non-final
chunks, so the lookup uses `or {}` instead of a .get default (a None usage
would otherwise raise AttributeError inside every chunk and break
test_model's connection probe).
2026-10-25 19:04:13 +08:00
PerryLink
d04ee1d131
fix(utils):Keep the timestamped output-name fallbacks distinct on a coarse clock (#7071)
* fix(harness): keep timestamped output-name fallbacks distinct

`output_name_candidates()` sampled `datetime.now(UTC)` inside the yield
loop, so its five timestamped fallbacks were only distinct when the host
clock advanced between two generator steps.  On a coarse clock (Windows
advances in sub-millisecond steps) all five produced the same name, and
the capture tools lost retries before failing with
`could not reserve an output filename`.

Sample the clock once when the first fallback is reached and offset each
attempt by one microsecond, so distinctness no longer depends on the host
clock.  Add a regression test that freezes the clock, and update the
exhaustion test to hold all five distinct stamps.

* docs: describe the coarse clock accurately

---------

Co-authored-by: PerryLink <255665900+PerryLink@users.noreply.github.com>
2026-10-25 16:17:15 +08:00
PerryLink
338cd26dc2
fix(memory): drop unusable numbers in stale extensions and consolidation confidence (#7442)
* fix(memory): drop unusable numbers in stale extensions and consolidation confidence

* fix(memory): log defaulted consolidation confidence and pin the int passthrough
2026-10-25 16:16:09 +08:00
Yufeng He
92e1372d5a
fix(skills): honor skills.container_path in user-scoped skill storage (#6770)
* fix(skills): honor skills.container_path in user-scoped skill storage

The user-scoped factory never passed container_path, so
UserScopedSkillStorage kept the default /mnt/skills while public
storage honored the configured root, and consumers reading the root
from the storage (container registry, skill activation) built paths
that do not exist once an operator moves the mount root.

Seed container_path from the resolved app config, matching the
public-storage factory; an explicit caller override still wins.

Fixes #6767

* fix(skills): seed container_path in the legacy-visibility kwargs branch

willem-bd review nit on #6770: user_should_see_legacy_skills constructs
a one-off UserScopedSkillStorage when callers pass kwargs, and that
branch skipped the app_config container_path seeding, so a future caller
combining kwargs with a custom skills root would read the default
/mnt/skills instead. Seed it the same way the factory does; an explicit
caller override still wins.
2026-10-25 15:30:20 +08:00
Starchaser1024
228913b6b7
fix(sandbox): keep document bytes when write_file translates paths (#7462)
* fix(sandbox): keep document bytes when write_file translates paths

LocalSandbox.write_file rewrote container paths inside every file it stored,
so a dictated README.md ended up holding a host path while read_file masked
the rewrite for agent-written files: the read-back showed the virtual path,
and the agent confirmed content the file did not contain (#7461).

Translation now applies only to executable content, classified with the
shared is_code_file rule from deerflow.skills.package_files that the skill
installer, the export guard, and skill_manage_tool already use. Code suffixes
and extensionless files that open with a shebang keep the host-path rewrite
they need to run under a host interpreter (PR #1935); documents, data files,
and configs are stored verbatim and read back as stored.

* fix(sandbox): translate paths for interpreter scripts and stop reading pipes

The classifier added for #7461 asked only the skill-package rule, which
does not list `.awk` (nor `.sed`, `.lua`, `.r`, `.tcl`), so a written AWK
program kept container paths that the host awk cannot open when it runs
`awk -f`. Those suffixes are now the sandbox's own set: the shared rule
also decides what SkillScan and the installer treat as package code, and
widening it there is a different call.

Classification no longer touches the destination unless the path is
extensionless. A suffix answers the question by itself, so an append to
`events.py` no longer opens the file first, and the extensionless
shebang sniff goes through `os.open(O_NONBLOCK)` plus an `fstat` regular
file check, which is what keeps a FIFO append from blocking.

* test(sandbox): pin the non-blocking sniff with a suffixless pipe append
2026-10-25 15:13:41 +08:00
lilei
ccb5fbfa8c
fix(runtime): reject delta resumes that discard pending tools (#7441)
* fix(runtime): reject delta resumes that discard pending tools

* fix(runtime): preserve pending head work on rejected delta replay

* fix(runtime): preserve non-Send checkpoint tasks on resume

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 14:44:32 +08:00
Lym308758
346ff7b253
fix(memory): rank capacity eviction per record, not per fact_id (#7454)
* fix(memory): rank capacity eviction per record, not per fact_id

select_facts_for_capacity keyed every score by fact_id, so an imported
batch that repeats an id collapsed into a single dict entry: the last
record's score won, and both the ranking pass and the evicted audit
read that one value for every position. A 0.9-confidence record could
therefore be evicted while its own 0.1-confidence duplicate survived.
The import path keeps caller-supplied ids without checking them for
uniqueness, and any over-cap import lands here.

Score by position instead: a parallel list next to the indexed facts,
so ranking and the audit read each record's own score. The public
``scores`` dict stays keyed by fact_id with last-write-wins
semantics, keeping the DeerMem benchmark contract and the callers
that never read it unchanged.

Red on main: test_confidence_policy_scores_duplicate_fact_ids_per_record
fails with the highest-confidence record evicted; green on this branch.

* fix(utils): keep share-less UNC roots out of the extended-length prefix

extended_length_path prefixed every absolute UNC path with
\\?\UNC\, but Win32 only accepts \\?\UNC\server\share\... — a
share-less root (\\server, \\server\, or a bare \) is not a valid
device path, so every Win32 call handed the prefixed spelling fails.
Only prefix when both the server and the share are present;
otherwise return the ordinary spelling unchanged.

Red on main: the \\server, \\server\, //, and //server cases of
test_windows_extended_spelling assert the forged prefix; green on
this branch.
2026-10-25 13:20:15 +08:00
Willem Jiang
2ce9bdc003
fix(sandbox): close pipe fds once after post-spawn failures (#7430)
* fix(sandbox): close pipe fds once after post-spawn failures

* fix(sandbox): reap children after startup failures
2026-10-25 11:49:44 +08:00
haochen
f7f9a7971b
fix(sandbox): normalize mount container_path trailing slashes at config load (#7393)
* fix(sandbox): normalize mount container_path trailing slashes at config load

A container_path declared with a trailing slash (e.g. /mnt/reports/) was
accepted by VolumeMountConfig and normalized by LocalSandboxProvider when
building path mappings, but the sandbox tool path gates compare the raw
configured value, so every ls/read/write/glob/grep/bash access to the
mount failed with a confusing PermissionError. Normalize trailing
slashes in the config model so both layers agree, and reject the
container root '/' which the gates cannot express.

* fix(sandbox): normalize skills.container_path and pin empty-value rejection

Review follow-up on #7393:
- Share the container-path normalizer with SkillsConfig, whose prefix
  feeds the same as-is comparison in _is_skills_path and had the same
  trailing-slash dead-path failure mode.
- Pin the empty-string rejection: on main an empty container_path made
  the gates match every absolute path, applying one mount's read_only
  flag to all paths.
- Cover both config fields in CONFIGURATION.md.

---------

Co-authored-by: Zhangchenhao-111 <285911435+Zhangchenhao-111@users.noreply.github.com>
2026-10-25 11:44:36 +08:00
Felix Shi
737d50f625
fix(channels): retry the runtime config file lock instead of failing after ~10s on Windows (#6888)
* fix(channels): retry the runtime config file lock instead of failing after ~10s on Windows

The runtime channel config sidecar lock (#6844) used Windows' blocking msvcrt LK_LOCK mode, which gives up after roughly ten seconds and surfaces cross-worker contention as Permission denied instead of serializing the read-modify-write cycles it exists to serialize, while fcntl.flock(LOCK_EX) waits on POSIX. Retry the non-blocking acquisition (LK_NBLCK + 0.2s) on the Windows fallback, mirroring the extensions manager's lock (#6020), and bind fcntl/msvcrt at module level so the fallback branch is testable.

Tests: new backend/tests/test_channel_runtime_config_store_lock.py (5 tests: retry cadence via fake msvcrt, sidecar byte creation, POSIX flock preservation, direct Windows regression through the sys.modules seam, real cross-thread contention). 3 of them fail on the unmodified base (one with OSError 36 raised by the blocking call itself) and all pass with the fix. Related suites: store peers + connections router 86 passed / 1 skipped; blocking_io + encoding + channels 498 passed / 2 skipped. Ruff format and check clean on both files. Windows 11, Python 3.12 (uv-managed venv).

* fix(channels): share contention retries and diagnose prolonged lock waits

* docs(channels): shorten guidance to satisfy lint budget

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 11:05:04 +08:00
Aling33
cb6ee72132
fix(config): delete read-only thread directories on Windows (#7446)
`Paths.delete_thread_dir()` called `shutil.rmtree()` directly. Windows
refuses to unlink a file that carries the read-only attribute
(`WinError 5`) where POSIX only needs a writable parent, so one read-only
file under `user-data/` made the whole thread undeletable: `DELETE
/api/threads/{id}` answered 500 `Failed to delete local thread data`,
`rmtree` had already removed everything it walked before that entry, and
every later retry failed on the same file.

Retry at the call site: clear the read-only attribute tree-wide and call
`rmtree()` once more. Only `PermissionError` is retried, the retry still
raises when it cannot help, so a file another process holds open stays a
real error instead of becoming a silent skip. A `onexc` handler cannot do
this portably: POSIX `_rmtree_safe_fd()` catches the handler's own
`PermissionError`, re-dispatches it as a `scandir` error and leaves the
deletion to fail later as `ENOTEMPTY`.

Co-authored-by: Aling33 <along308@users.noreply.github.com>
2026-10-25 11:02:49 +08:00
Xuehao Xu
8237bc6c9e
fix(subagents): preserve test summaries before evidence truncation (#7407)
* fix(subagents): preserve test summaries before evidence truncation

* fix(subagents): honor Node counters and cache harvested summaries

* fix(subagents): retain warning counts and Go case evidence

* fix(subagents): retain long pytest sibling summaries

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 10:25:04 +08:00
jay
63211d5da6
fix(readability): preserve <a> links when Readability.js silently degrades on Windows (#7412)
* fix(readability): preserve <a> links when Readability.js silently degrades on Windows

On Windows, simple_json_from_html_string(use_readability=True) invokes the
Readability.js subprocess, which succeeds without error but returns the
unextracted document instead of the extracted article. The pure-Python
fallback (use_readability=False) also drops every <a> destination. The net
effect is that web_fetch returns link-free Markdown on Windows.

Fix: after extraction, check whether the output retained the <a> elements
that the resolved HTML carried. When they are all gone, fall back to
converting the resolved HTML directly with markdownify — which preserves
<a> as [text](href) — so links survive regardless of the Readability.js
outcome.

Measured: 17 of 22 test_web_fetch_relative_links.py failures resolved;
43/48 pass on Windows (5 remaining are pre-existing malformed-markup
extraction tests, unrelated to this change).

Closes #6459

* test(readability): add degraded-extraction and scope-pinning tests

* test(readability): detect unwanted article fallback

* test(readability): classify HTML body in authority-tag census

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 08:56:21 +08:00
spud
e9074b7126
feat(runtime): publish owner-confirmed run.terminal evidence (#7134)
* feat(runtime): publish authoritative run terminal events

* fix(runtime): preserve terminal attribution diagnostics

* docs(runtime): consolidate terminal event docs and trim agent guidance

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-25 00:12:26 +08:00
DDT
afe244e7f9
fix(config): pin explicit app config path across get_app_config (#7092)
* fix(config): pin explicit app config path across get_app_config

Fixes #7086. reload_app_config(config_path) now records an
_app_config_source pin (mirroring _extensions_config_source) so
get_app_config and DeerFlowClient(config_path=...) keep following the
explicit file instead of ambient default resolution.

* fix(config): synchronize app config cache and absolute pins

Serialize singleton reads, reloads, resets and injections so a concurrent call cannot publish a config under another file's pin. Keep explicit paths absolute across working-directory changes while preserving ambient discovery and failed-reload behavior.

Add deterministic concurrency and relative-path regressions. Validation: 33,875 offline tests and 262 blocking-I/O tests passed; frontend check, backend formatting and agent guidance passed.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-24 22:04:48 +08:00
llb555
8984093552
fix(uploads): make companion record removal best-effort (#7409)
* fix(uploads): make companion record removal best-effort

* style(uploads): format the new test and log the markdown-reference removal

* fix(uploads): report the orphaned record through the module that owns it
2026-10-24 21:56:38 +08:00