* fix(sandbox): stop list_dir from reporting failures as empty
Remote providers swallowed find/client errors as [] and 2>/dev/null
missing paths as empty stdout. ls_tool then told the agent the
directory was (empty). Raise OSError/FileNotFoundError instead so
the tool returns Error.
* fix(sandbox): list_dir raises on missing local paths and uses find -H
Empty stdout is not a missing path when find's start point is a
symlink (E2B /mnt/acp-workspace). Dereference only the start point
with find -H. LocalSandbox now raises FileNotFoundError for a
non-directory root, matching remote providers. AIO maps a missing
result.data to OSError rather than FileNotFoundError.
* fix(sandbox): group AIO list_dir find type predicates
Without parentheses, find PATH -maxdepth N -type f -o -type d applies
-type d without maxdepth and can drop files from the listing.
* fix(sandbox): distinguish list_dir command failure from missing path
Tenki, Boxlite, and OpenSandbox treated any empty find stdout as
FileNotFoundError, so a missing find binary (exit 127) or SDK error
looked like a missing directory. Raise OSError when find status is
outside (0, 1); keep FileNotFoundError for the find-ran-but-empty case.
* fix(sandbox): apply list_dir exit-status contract to AIO and E2B
Same gap as Tenki/Boxlite/OpenSandbox: empty find stdout with exit 127
was FileNotFoundError. Raise OSError when the status is outside (0, 1).
* fix(sandbox): classify list_dir by find status not head status
find | head under sh -lc reports head's exit code, so a missing find
binary (127) became FileNotFoundError. Record find's own status after
the bounded listing, treat SIGPIPE 141 as truncation success, and add
a shell-level regression test.
* test(auth): include projects permissions in /me contract pins
#5265 added projects:read/write/delete to the registered route set.
The /auth/me tests still pinned the pre-projects list, so CI failed
after merging main.
* fix(sandbox): do not treat missing list_dir marker as success
The generated script ended on `rm -f`, so process status was 0/1 even
when find's marker never landed. Both codes are in _FIND_OK, and the
parser fallback then classified an empty listing as FileNotFoundError —
the 127 misclassification this helper was meant to close.
Exit with find's status (126 if unknown). A missing marker is now
OSError unless the process status is already a non-OK failure.
* test(sandbox): emit list_dir status marker in provider fixtures
Parser now requires __DF_FIND_STATUS__ and refuses marker-less stdout.
Update AIO/Boxlite/E2B stubs and OpenSandbox/Tenki find fakes so listings
carry :0 and missing paths carry :1 with matching exit codes.
* style(sandbox): format list dir test fixture
* style(sandbox): format remote list dir helper
* docs(sandbox): keep guidance within the tested size budget
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): mask output tails into POSIX-style virtual paths
The output maskers slice the matched path tail from the original
output. With separator-agnostic matching, a Windows-spelled nested
tail kept its backslashes and was spliced into the POSIX-style virtual
path, so glob results and masked read output showed mixed paths like
/mnt/user-data/workspace/pkg\util.py or
/mnt/skills/integrations/lark-cli\lark-doc\SKILL.md. Virtual paths are
always POSIX-style, so normalize nested tails to forward slashes the
same way depth-1 tails already end up. Depth-1 tails and the callable
replacer (LocalSandbox._reverse_resolve_path) were unaffected.
Pin the nested-tail contract in test_sandbox_path_patterns; the
previously failing glob-tool and skills-masking regressions now pass
on Windows hosts.
* refactor(sandbox): share the mask tail-splicing rule; guard it on Linux CI
Review follow-up for #5247:
- hoist the tail-splicing rule (slice off the base, strip leading
separators, normalize the rest to "/") into
path_patterns.normalize_mask_tail and import it at both call sites,
so the two maskers can only drift in their matching logic, not in
the splice;
- add test_mask_local_paths_normalizes_windows_spelled_skill_tails,
which spells the skills host root and the output with Windows-style
strings so the nested tail keeps backslashes on every platform.
Reverting the mask_local_paths_in_output-side normalization now goes
red on Linux CI too, not only on Windows hosts.
* feat(subagents): check and persist durable batch acceptance
Carry optional per-item criteria into native subagents, reuse the deterministic checker, and expose separate verdicts through item queries and exports. Preserve execution and retry semantics, renew leases during checks, and migrate existing batch rows with nullable acceptance fields.
* fix(subagents): align batch acceptance normalization and sandbox admission
* test(auth): include project permissions in the full-stack contract
* fix(tests): await future completion before asserting done()
`test_run_on_isolated_subagent_loop_survives_caller_loop_teardown`
signals from inside the coroutine:
async def deferred_work() -> None:
completed.set()
`run_on_isolated_subagent_loop` is `asyncio.run_coroutine_threadsafe`,
whose `concurrent.futures.Future` is marked done by the loop only after
the coroutine returns. The main thread can therefore wake from
`completed.wait()` while the future is still pending, and
`assert handles[0].done()` fails:
assert False
+ where False = done()
+ where done = <Future at 0x7f62241b22d0 state=pending>.done
Observed on main at a2808e82 (shard 2) and on an unrelated PR at
852a94dd (shard 4) sixteen seconds apart, so it tracks runner load
rather than any change under test.
Assert the result first — `Future.result(timeout=10)` blocks until the
future completes — then assert `done()`. Both assertions keep their
original meaning and no sleep is introduced.
Fixes#5298
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test: drop the now-redundant done() assertion
Review follow-up. Once `result(timeout=10)` has returned normally the
future is guaranteed to be FINISHED, so the `done()` assertion below it
could no longer fail — it documented intent rather than checking
anything.
`result()` alone proves both halves of what the test is about: that the
coroutine body ran after caller-loop teardown, and that the future
resolved. The `completed.wait()` guard above still covers the "work
never ran" case with a descriptive message.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(frontend): support standalone demo APIs and runtime GitHub stars
* Update API origin URL to use environment variables
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* test(frontend): align static demo tests with runtime origin
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(doctor): skip tool checks when tools block is empty
Follow-up to #5296, which fixed this for `models:`. The same defect
remains for `tools:`: `.get("tools", [])` returns None when the key is
present but empty, because the default only applies when the key is
absent. Iterating that None raises TypeError, which the surrounding
broad handler renders as a check result:
! web search configured ('NoneType' object is not iterable)
! web fetch configured ('NoneType' object is not iterable)
! web capture configured ('NoneType' object is not iterable)
! image search configured ('NoneType' object is not iterable)
✗ sandbox configured ('NoneType' object is not iterable)
Line 476 is reached by all four web/image checks through the shared
check_web_tool helper, and line 645 by check_sandbox.
Unlike the models case, a default install does not hit this: `make
config` ships ten real tool entries, so a user has to empty or comment
out that block first.
The web checks now fall through to their normal "no tool in config"
warning and the sandbox check evaluates normally. Parentheses on the
comprehension are for readability; `or` already binds correctly there.
Regression tests use the commented-out `tools:` shape that reproduces
the failure, matching the tests added in #5296.
Fixes#5300
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(doctor): skip non-mapping tool entries, tighten regression tests
Review follow-ups on the line this PR already changes.
A `tools:` list holding a scalar (`tools:\n - web_search`) reached
`t.get("name")` and raised AttributeError, which the broad handler
rendered as the check result:
! web search configured ('str' object has no attribute 'get')
That is the same leakage this PR removes for the null case, so it is
fixed here rather than deferred. `check_sandbox` already guards the same
way via `isinstance(tool, dict)`.
The empty-tools test asserted that "NoneType" was absent from the
detail, which pins the failure mode rather than the behaviour — it would
still pass if the detail became some other internal error text. Both
tests now assert the expected message directly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(doctor): assert sandbox outcome instead of the failure mode
Review follow-up, same class as the web-tool assertion fixed earlier in
this PR. The sandbox regression test still asserted that "NoneType" was
absent from the detail, which pins the failure mode rather than the
outcome — it would keep passing if some other internal error text leaked
out of the broad handler.
On this config the path is deterministic: an empty `tools:` means no
bash tool, so exactly one result. Assert the fields directly
(`CheckResult` has no `__eq__`, so whole instances cannot be compared by
value).
Verified against `main`'s scripts/doctor.py, where the same config
yields status=fail and detail="'NoneType' object is not iterable", so
the new assertions are red there and green here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
`config.example.yaml` ships a `models:` key with every entry commented
out, so it parses as None rather than an empty list and the `[]` default
in `.get("models", [])` never applies. Iterating that None raised
TypeError, which the surrounding broad handler rendered as a check
result:
✗ LLM API key check ('NoneType' object is not iterable)
✗ LLM auth check ('NoneType' object is not iterable)
✗ LLM package check ('NoneType' object is not iterable)
Every fresh install hit this before configuring a model, turning one
actionable error into four and hiding the real "models configured" hint
behind internal exception text.
Fall back on a falsy value at the three iteration sites so the checks
return no results when nothing is configured. `check_models_configured`
gets the same treatment for consistency; it was already correct because
it tests truthiness rather than iterating.
The existing tests missed this because they use `models: []`, an
explicit empty list, which iterates fine. The added regression tests use
the commented-out shape that `make config` actually produces.
`make doctor` now reports 1 error instead of 4 on a fresh clone.
Fixes#5295
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* test(checkpoint): retention deletion contract + growth baseline
Six contract scenarios x memory/sqlite/postgres pin what retention deletions
must never break (branch ancestors, explicit resume targets, pending writes,
duration-only chain links), prove the two safe shapes (leaf sibling branches,
trailing duration leaves), record the full-vs-delta growth baseline in the
normalized bench shape, and add an item 4 probe showing the default
ToolOutputBudgetMiddleware already externalizes oversized tool results.
Refs #4189
* test(checkpoint): make the retention contract load-bearing per review
Review findings from willem-bd and Ricky-7-Yan:
- scenario D pins its own row: before/after stats delta plus a serde
round-trip of the stored write, instead of an always-true > 0 check
- _delete_checkpoint now performs the joint delete the doc mandates
(checkpoint row + writes rows + blobs unreachable from surviving
checkpoints), so E1/E2 exercise the shape they prescribe
- E1 builds the real runtime duration shape via persist_run_durations
(parent dict clone, fresh id/ts, real metadata), which surfaces the
shared-version case: the leaf's blobs are the surviving parent's rows
- contract doc: blob reachability must be computed from surviving
checkpoints in a whole-thread pass; shared-version/duration-only
hazard called out explicitly; memory data model includes saver.blobs
- _stats counts memory blob rows and returns the full normalized shape
(logical byte totals included)
- probe: drops the unused middleware/outputs_dir graph parameters and
discloses the manual-harness scope limit in the module docstring
- E1/E2 assert default head resolution (protected set item 5); unused
graph_for helper and DURATION_ONLY_METADATA stand-in removed
Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
* fix(checkpoint): scope probe cleanup to owned dirs, key report by backend
Second-round review findings on #5255:
- [P1] bench_tool_result_probe.py removed the whole user-supplied
--outputs-dir (and the shared .probe-tmp) in its finally block, so
pre-existing files were deleted on success and failure alike. The run
now writes into (and removes) a fresh owned probe-run-* child beneath
the requested directory, and SQLite databases live in a unique
mkdtemp'd temp directory that is removed with the run. Regression
tests pin that unrelated pre-existing files survive both a successful
and a simulated failing run.
- [P2] the optional retention report keyed every backend's measurements
under one shared name, so a multi-backend invocation kept only the
last backend's numbers. _report() now takes the parameterized backend
explicitly (saver_env.kind); regression pins that memory and sqlite
entries coexist in one report file.
Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
---------
Signed-off-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
Co-authored-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
* feat(artifacts): preview CSV and TSV files as bounded tables
* chore: keep preview screenshots out of the PR file diff
* fix(artifacts): detect record newlines outside quoted fields
* test(auth): include project permissions in me contract expectations
* feat(projects): project workspaces with scoped chats and thread membership
Backend:
- projects table model and migration; fail-closed ProjectRepository with
ownership checks, CRUD/archive/restore/delete router, and atomic thread
move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
metadata; project-aware thread create/search with pagination bounds and
membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
at write time and dropped when invalid; serialized against project
deletion and thread assignment
- branch creation inherits the source thread's project membership (an
archived/deleted project degrades the branch to unassigned instead of
failing the request)
Frontend:
- projects data layer, thread move API, and sidebar projects section with
flat/grouped modes, archived-project threads, and stable virtual-list
offsets
- project detail page with project-scoped new chat
(/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
continuation is dropped instead of saving the goal or launching the
abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
an open project page never keeps stale titles, recency, or pagination
* fix(chats): keep archive undo toast when the sidebar row unmounts
The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.
Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.
* fix(projects): pin project thread listing contract and exclude archived chats
GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.
The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.
Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.
* docs(migrations): record the 0019/0020 chain against the bootstrap reservation
The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.
Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.
* fix(chats): invalidate project thread lists on archive/restore
useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.
An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.
Regression test asserts the project-list prefix is invalidated on success.
* fix(projects): fetch project discovery only in grouped sidebar mode
RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.
Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.
* fix(threads): fail closed on project-scoped create in memory mode
MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).
Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.
Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.
* fix(projects): window the project page thread list
ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.
Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.
* fix(projects): restore row dividers and pin them with a render test
The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.
The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.
* fix(projects): validate forward schemas and fence membership reads
* feat(authz): surface effective route permissions on GET /auth/me (Phase 4, #4063)
GET /api/v1/auth/me now returns the effective route permissions alongside
the user identity, so the frontend can hide actions the caller's role
cannot perform (RFC #4063 Phase 4).
The value reuses the AuthContext that AuthMiddleware already resolves per
request (including PAT-scope intersection and internal-caller semantics),
so /me adds zero extra provider evaluations; a middleware-less composition
falls back to the same resolution _authenticate uses.
Credential-creation responses (register/initialize) leave the field None:
they are public paths where the middleware does not run, and resolving
there would introduce fresh on-loop config loads on those routes.
* test(e2e): expect /auth/me permissions in auth-disabled contract
PR #5228 adds the effective route permissions to GET /auth/me, so the
strict toEqual against the bare AUTH_DISABLED_USER object no longer
holds: the received payload carries six extra keys (the permissions
array). Extend the expected payload with the full registered permission
set in _ALL_PERMISSIONS order — with authorization disabled the gateway
grants exactly that static list, so the pin stays deterministic.
The runtime frontend is unaffected (auth-disabled SSR never calls /me,
and userSchema strips unknown keys); only this contract pin needed the
new field.
* refactor(authz): public resolve_route_permissions_for_request wrapper
Address review nits on the middleware-less fallback: the router reached
into the private authz._is_internal_caller, so expose a thin public
wrapper pairing resolve_route_permissions with the internal-caller
heuristics, and use it from both _authenticate and the /me fallback so
the two cannot drift apart. Also drop an unused tmp_path parameter from
test_auth_disabled_me_includes_default_admin_permissions (_setup_auth
provisions its own tmp directory).
No behavior change: the wrapper delegates to the exact pair of calls the
fallback made before.
* fix: preserve actionable subagent acceptance gaps
Distinguish completed execution from acceptance in delegation guidance. Retain bounded unmet and unverified criteria after compaction and guide the lead to address remaining work within its budget.
* docs: keep acceptance guidance within instruction budget
* fix(workspace-changes): record symlink targets without the verbatim prefix
os.readlink on Windows reports absolute targets in extended-length
form (\?\C:\... or \?\UNC\server\share). The scanner stored that raw
spelling, so workspace-change events showed \?\-prefixed targets that
do not match ordinary Windows paths. Strip the prefix when recording;
POSIX readlink output is unchanged. Skills projection/review readlink
sites are untouched — they have no user-facing contract pinned on the
spelling.
* fix(workspace-changes): gate symlink target normalization to Windows
Review follow-up on #5250:
- Gate _normalize_symlink_target on os.name == "nt". readlink(2) on
POSIX returns the literal string the link was created with, and
backslash is a valid filename byte on Linux, so a target that starts
with the extended-length prefix there must be recorded verbatim. The
docstring's POSIX claim is now provably true.
- Commit the unit checks the PR body previously described as ad-hoc:
drive and UNC prefix stripping, relative and plain POSIX targets,
mid-string prefix left verbatim, and an off-Windows identity case, so
the new branch has real coverage on every platform instead of relying
on a Windows host with symlink privilege.
* test(workspace-changes): force Windows platform in prefix-strip unit tests
The os.name gate added in the previous commit makes
_normalize_symlink_target a verbatim identity off-Windows, so the two
prefix-strip assertions failed on the ubuntu-only unit CI. Force
os.name to "nt" via monkeypatch in both, mirroring the off-Windows
identity test, so every case pins exactly one platform's contract and
the suite is green on every host.
* fix(workspace-changes): strip only extended drive-letter prefixes
Review follow-up on #5250: the catch-all branch also stripped the
extended-length prefix from volume-GUID targets (\?\Volume{...}\...),
leaving a relative-looking path that loses the target's namespace.
Restrict the branch to extended drive-letter paths (letter, colon,
separator) and keep every other \?\ namespace form verbatim; add the
volume-GUID regression plus degenerate-prefix cases.
* feat(gateway): support idempotent thread runs
Accept Idempotency-Key on thread-scoped create, stream, and wait endpoints, scoped by owner and thread before durable admission.\n\nRefs #5257.
* fix(gateway): handle idempotent run reuse on wait and stream
Reused store-only records have no local task. /wait now waits on the
bridge when it can observe the stream, and otherwise returns durable
status instead of a stale checkpoint. A reused terminal stream that has
been evicted emits gap/reload_durable_state. Replay is bound to the
original input and assistant_id.
* fix(gateway): 409 reused in-flight streams on this worker
A store-only running record on a process-local bridge has no owner
stream. POST /runs/stream used to subscribe anyway, which created an
empty log and waited forever. Match join: 409 unless the run is already
terminal, so missing-stream retries can still emit gap.
* fix(gateway): keep observer joins off the idempotent stream-gap path
sse_consumer keyed missing-stream gap on the sticky
idempotency_reused flag, so a later join of a terminal run inherited
it. Gate that branch on apply_on_disconnect, which already separates
creating streams from joins. Document the retry outcomes clients have
to handle.
* fix(gateway): gate missing-stream gap on creating retry
Reuse apply_on_disconnect to pick gap vs end changed sse_consumer default path, so a missing stream started returning gap for default callers and for out-of-scope POST /api/runs/stream. Keep that branch behind emit_gap_on_missing_stream and pass it only from thread-scoped /runs/stream on this request reuse.
* fix(gateway): keep wait reuse off later checkpoints
Direct handler calls were crashing because FastAPI Header() leaked in as the Python default. Bind Idempotency-Key with Annotated so the default is None, and ignore non-str keys.
A reused completed /wait was still reading the latest thread checkpoint. After a later run on the same thread that is the later run's result. Return durable status instead of claiming the head as this run's output.
* fix(gateway): snapshot wait reuse and refresh store status
idempotency_reused lives on the shared cached record. Capture it before awaiting completion so an overlapping retry cannot suppress the original creating /wait checkpoint.
A store-only peer record still holds admission-time status after the owner publishes END. Refresh durable status/error before returning them.
* feat(community): add Sofya web search provider
Add a community provider backed by Sofya (https://sofya.co). Its search
endpoint returns the content of the result pages, not only their snippets,
and its fetch endpoint returns a page as markdown. Both are plain JSON over
HTTP, so this needs no extra Python package (uses httpx, already a
dependency).
Changes:
- backend/packages/harness/deerflow/community/sofya/__init__.py
- backend/packages/harness/deerflow/community/sofya/tools.py
Implements web_search_tool and web_fetch_tool using httpx.
API key is read from the config.yaml `api_key` field or the SOFYA_API_KEY
env var. Follows the same interface and output shape as the existing
ddg_search and serper providers, including the max_results parameter with
config override and the structured "No results found" error.
- backend/tests/test_sofya_tools.py
Unit tests covering API key resolution, config overrides, result mapping,
time range, HTTP errors, empty results, and fetch failures.
- config.example.yaml: add commented-out Sofya web_search and web_fetch
examples alongside the other providers
- .env.example: add SOFYA_API_KEY placeholder
- backend/docs/CONFIGURATION.md: list Sofya under web_search, web_fetch and
the environment variables
* fix(sofya): honor caller max_results, validate search_depth, join time_range contract test
- Caller-supplied max_results now wins; config is used only when the
argument is omitted, matching GroundRoute.
- search_depth is clamped to basic/snippets; an unsupported value logs a
warning and falls back to basic.
- Sofya added to the shared time_range schema contract test.
* fix(sofya): cap per-result content so a search stays inline
An unbounded search payload (up to 20 read pages) crossed the tool output
budget middleware's externalize_min_chars threshold, which replaces the
result list with a file reference. Cap each result's content at
contents_max_characters (default 2000, 0 disables), matching Exa's config
key. Five capped results stay under the 12000 char threshold.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* fix(sofya): list Sofya in the recency contract, coerce non-string content
_clip subscripted its input, so a non-string content or description from
the API raised TypeError instead of degrading. Coerce to text first, the
way _sofya_post and _response_results guard the shapes around it. Also add
Sofya to the Web Search Recency section in backend/AGENTS.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* fix(sofya): coerce web_fetch content, list sofya in the tools guide, add changelog
web_fetch sliced its content the same way web_search did before the last
push: a truthy non-string from the API passed the falsiness guard and then
raised TypeError. Reuse _clip, keeping the `or ""` so empty content still
reports "No content found".
Also add sofya to the community provider inventory in
packages/harness/deerflow/tools/AGENTS.md and an [Unreleased] changelog entry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* docs(zh): add the missing InfoQuest and Firecrawl web_fetch tabs
The ZH web_fetch tab list named five providers where EN names seven. Both
tabs mirror their EN counterparts, so the two locales list the same
web_fetch providers again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* test(skills): skip POSIX mode-bit assertions on Windows
Windows has no POSIX mode bits: st_mode always reports 0o777 and
Path.chmod only honors the read-only flag, so the readability
assertions in both skill-permissions tests cannot hold on Windows
hosts. Skip them there with an explicit reason; they still run on
POSIX where the chmod contract applies.
* test(skills): address review feedback on Windows skips
- correct the skip reason: Windows mode bits are observable; it is
Path.chmod() that only toggles the read-only bit, so the asserted
0o644/0o755 modes are never observable there;
- hoist the repeated skipif to a module-level requires_posix_mode_bits
decorator so the reason stays single-sourced;
- keep test_written_path_readability_is_limited_to_written_path
executing the resolve()/relative_to() traversal on Windows with
content-intact smoke assertions, skipping only the mode-bit asserts.
* test(skills): single-source the skip reason string
Follow-up to the re-review: the reason text lived verbatim in both the
module-level skipif and the inline pytest.skip() call; promote it to a
_POSIX_MODE_BITS_REASON constant used by both call sites.
* fix(sandbox): stop E2B append from overwriting on read failure
E2B has no native append, so write_file(append=True) read-modify-writes.
Treat only FileNotFoundException/FileNotFoundError as an empty file; any
other pre-read error must abort so a timeout cannot replace the original
contents with just the new fragment.
* fix(sandbox): distinguish E2B append pre-read refusal in logs
A non-not-found pre-read error now logs as a refused overwrite instead
of a write failure. Tests pin the successful read-modify-write path,
including a bytes pre-image, so dropping `existing` cannot go green.
* fix(mcp): resolve drive-qualified paths in file reference rewriting
urlparse reads a Windows drive prefix ("C:/...") as the URI scheme, so
_local_path_from_uri() returned None for every drive-qualified path and
MCP file references were never rewritten to /mnt/user-data/... virtual
paths on Windows hosts. file:// URIs were parsed with urlparse().path
alone, which also drops the drive qualifier.
- resolve file URIs through url2pathname so the /C:/... form keeps its
drive, and treat single-letter schemes as bare drive paths;
- match drive-qualified absolute paths in the free-text reference regex;
- build test URIs with Path.as_uri() and anchor absolute-path fixtures
at tmp_path so expectations are host-portable, and cover the
drive-prefix scheme quirk explicitly.
* fix(mcp): decode file URIs once and guard Windows path rejection
Review follow-up on #5242:
- url2pathname already percent-decodes on both platforms, so the extra
unquote() wrapper decoded references twice and broke filenames that
contain a literal '%'. Pass parsed.path straight through.
- On Windows, url2pathname raises OSError for paths containing a raw
'|' (e.g. file:///C:/tmp/a|b.png); catch it so one odd URI cannot
abort the whole best-effort rewrite pass.
- The relative-reference regex alternative now accepts backslash
separators, which is what Windows servers print for relative paths.
- Add Windows-only regressions driving the backslash free-text form
and a file:///C:/ URI end to end, plus the OSError rejection.
* fix(mcp): resolve file://C:/… URIs with a drive-qualified authority
Review follow-up on #5242 (two-slash Windows drive form):
- Some Windows tools emit file://C:/… without the third slash, which
puts the drive in the URI authority. Consult parsed.netloc: rebuild
the /C:/… URL path for a drive-qualified authority, keep the current
handling for empty and localhost authorities, and reject any other
host instead of silently treating its path as local.
- Extend the free-text regex so the two-slash form matches as one token
instead of the previous stray e://… mid-token match.
- Cover the two-slash form at the _local_path_from_uri unit, through
_rewrite_local_paths_in_text, and add a portable case asserting that
a remote-host file URI is ignored.
* fix(mcp): anchor the drive-qualified text alternative with a lookbehind
Review follow-up on #5242:
- [A-Za-z]:[\/] could steal a token at an earlier scan position:
for file:/tmp/… (single-slash form per RFC 8089 / Java File.toURI())
the match became e:/tmp/…, which resolves as a bare drive path and
left the reference unrewritten where /tmp/… was rewritten before.
Anchor the alternative with (?<![\w.-]) so word:/… shapes fall
through to the earlier alternatives.
- Add the missing coverage for the relative alternative's backslash
support (temp\page.yml through _rewrite_local_paths_in_text) and
a portable regression pinning the file:/… tokenization.
* fix(agents): prioritize loop hard stops across tool batches
Scan an admitted multi-tool response completely before selecting a soft warning, so any configured hard limit can reject the whole batch. Preserve warning priority and sliding-window accounting.
Add counter-level plus sync and async compiled-agent regressions proving rejected tools are not executed.
Fixes bytedance/deer-flow#5243. AI-assisted implementation and tests.
* fix(agents): rearm loop warnings after cross-tool eviction
When another tool evicts an older tool below its frequency warning threshold, clear the older suppression mark so a later burst can warn again.
Add the cross-tool sliding-window regression from the final boundary review. AI-assisted implementation and tests.
* test(agents): cover override-aware loop warning rearm
Cache the default frequency thresholds for sliding-window eviction and verify that an evicted tool uses its configured override when warning eligibility is rearmed.
Document that simultaneous frequency warnings preserve legacy first-crossing selection while hard stops remain batch-severity-first.
Addresses review on #5245. AI-assisted implementation and tests.
---------
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Continues the zh/en README drift follow-up (#5222 covered the LangGraph
Studio section). The English README documents Request Trace Correlation
as a standalone section between the slash-command table and LangSmith
Tracing; the zh README only mentioned the correlation id in passing
inside its Langfuse section.
Add the translated section in the same position (after the
slash-command table, before LangSmith 链路追踪), covering: X-Trace-Id
inheritance/generation and the response header, propagation to detached
runs / subagents / background memory threads, the logging.enhance
config, the deerflow_trace_id semantics (not a run id, not a provider
trace id, not a lookup key, ignored-and-overwritten when supplied by the
caller), terminal run.delivery receipts with orphan-recovery behavior,
and the loop-detection / deferred-tool-promotion run event records.
The Chinese README had drifted from the English one: the entire
"LangGraph Studio (Optional)" section (including the standalone
langgraph dev workflow, server-owned assistant provenance in that mode,
the persisted-store repair note, the authenticated-identity consumption
for custom agents/skills/uploads/memory, the /mnt/user-data/outputs
native-delivery enforcement, and the dual LangGraph streaming interfaces
for custom events) was never translated and had no zh counterpart.
Add the translated section in the same position it occupies in the
English README (after the local development walkthrough, before
进阶配置), keeping the en section structure aligned. Verified the
section heading now exists on both sides and that no relative links are
affected.
TestBashExecutionHarvest's shell-persistence tests report
shell_persistent=None whenever deerflow.sandbox.overwrite is not already
cached in sys.modules. _setup_executor_classes replaces the
"deerflow.sandbox" parent package with a MagicMock, so a later
`from deerflow.sandbox.overwrite import unwrap_sandbox` inside
_harvest_shell_persistence can no longer locate the submodule through
the mocked parent ("'deerflow.sandbox' is not a package") when the leaf
module is not already in sys.modules. The helper's
`except Exception: return None` silently converts that ImportError into
an UNKNOWN provenance stamp.
Whether the leaf module was cached depended on which tests ran earlier
in the session, making the outcome order-dependent: green in CI by
collection-order luck, red when the module runs alone or first.
Fix it the same way the fixture already handles audit_context and
tool_search: preload the real leaf modules (deerflow.sandbox.sandbox_provider
and deerflow.sandbox.overwrite) before installing the mocked parent
package, pin them in sys.modules for the duration of the test, and
restore the previous state afterwards.
* fix(tests): make three backend test modules runnable on Windows hosts
Follow-up to #5210 (clock-granularity fix) clearing the remaining
deterministic Windows failures in modules that are otherwise
platform-neutral. Five tests fail on Windows for root causes unrelated
to the behavior under test:
- test_pnpm_script.py::test_make_install_dry_run_does_not_invoke_bare_pnpm
shells out to `make`, which Git Bash on Windows does not bundle
(the same gap reported in #5177). Skip when make is unavailable.
- test_mcp_session_pool.py: one test asserts the injected MCP temp dir
has POSIX mode 0o700; Windows has no POSIX mode bits (ntfs reports
0o777), so the mode check now runs only on POSIX. A second test
asserted `TMP.endswith("mcp-internal/tmp")` while Windows tmp paths
use backslashes; normalize the separator before comparing.
- test_skillscan_native.py: two tests build a 3000-operand `1+1+...`
chain to exercise deep-AST resilience. CPython's C recursion limit
for ast construction is platform-dependent (~800 on Windows vs
~8000 elsewhere), so 3000 reliably overflows on Windows and the
scanner records an error instead of findings. 600 chained BinOps
stays deep for the client-analysis walk while fitting the limit on
every supported platform.
No product code is touched; on POSIX the suite behaves exactly as
before.
* fix(tests): make three backend test modules runnable on Windows hosts
Follow-up to #5210 (clock-granularity fix) clearing the remaining
deterministic Windows failures in modules that are otherwise
platform-neutral. Five tests fail on Windows for root causes unrelated
to the behavior under test:
- test_pnpm_script.py::test_make_install_dry_run_does_not_invoke_bare_pnpm
shells out to `make`, which Git Bash on Windows does not bundle
(the same gap reported in #5177). Skip when make is unavailable.
- test_mcp_session_pool.py: one test asserts the injected MCP temp dir
has POSIX mode 0o700; Windows has no POSIX mode bits (ntfs reports
0o777), so the mode check now runs only on POSIX. A second test
asserted `TMP.endswith("mcp-internal/tmp")` while Windows tmp paths
use backslashes; normalize the separator before comparing.
- test_skillscan_native.py: two tests build a 3000-operand `1+1+...`
chain to exercise deep-AST resilience. CPython's C recursion limit
for ast construction is platform-dependent (~800 on Windows vs
~8000 elsewhere), so 3000 reliably overflows on Windows and the
scanner records an error instead of findings. 600 chained BinOps
stays deep for the client-analysis walk while fitting the limit on
every supported platform.
No product code is touched; on POSIX the suite behaves exactly as
before.
Update: address review feedback (P2, recursion-recovery regression)
The 600-operand chain no longer exercises recursion exhaustion on POSIX,
so the recovery handler in _scan_python was unprotected by the renamed
test. Replace the input-based variant with a controlled RecursionError
injected via monkeypatched _find_client_handle_sink (platform-
independent); removing the handler now turns the test red again.
* fix(tests): make three backend test modules runnable on Windows hosts
Follow-up to #5210 (clock-granularity fix) clearing the remaining
deterministic Windows failures in modules that are otherwise
platform-neutral. Five tests fail on Windows for root causes unrelated
to the behavior under test:
- test_pnpm_script.py::test_make_install_dry_run_does_not_invoke_bare_pnpm
shells out to `make`, which Git Bash on Windows does not bundle
(the same gap reported in #5177). Skip when make is unavailable.
- test_mcp_session_pool.py: one test asserts the injected MCP temp dir
has POSIX mode 0o700; Windows has no POSIX mode bits (ntfs reports
0o777), so the mode check now runs only on POSIX. A second test
asserted `TMP.endswith("mcp-internal/tmp")` while Windows tmp paths
use backslashes; normalize the separator before comparing.
- test_skillscan_native.py: two tests build a 3000-operand `1+1+...`
chain to exercise deep-AST resilience. CPython's C recursion limit
for ast construction is platform-dependent (~800 on Windows vs
~8000 elsewhere), so 3000 reliably overflows on Windows and the
scanner records an error instead of findings. 600 chained BinOps
stays deep for the client-analysis walk while fitting the limit on
every supported platform.
No product code is touched; on POSIX the suite behaves exactly as
before.
Update: address review feedback (P2, recursion-recovery regression)
The 600-operand chain no longer exercises recursion exhaustion on POSIX,
so the recovery handler in _scan_python was unprotected by the renamed
test. Replace the input-based variant with a controlled RecursionError
injected via monkeypatched _find_client_handle_sink (platform-
independent); removing the handler now turns the test red again.
Update: address second review feedback (P2, early-stop regression coverage)
The 600-operand tail no longer proves the walk stops after finding a
sink (it completes inside POSIX recursion limits either way). Replace it
with the suggested instrumentation: a sentinel os.system call after the
sink plus an instrumented _walk_client_scope that records any visit to
the sentinel while analysis.found is already set, failing the test if
traversal continues past the sink. Platform-independent; the sentinel's
shell-exec finding comes from the deterministic ast.walk pass and is
irrelevant to the walk guard.
* fix(journal): dedup llm.ai.response persistence on re-fired on_llm_end
LangChain may deliver on_llm_end more than once for the same run_id.
RunJournal already dedups token accounting and the run summary
(_record_message_summary) on that premise via _counted_message_llm_run_ids,
but the durable llm.ai.response self._put() call was left unguarded.
The event store is append-only and count_messages/list_messages read raw
rows without read-time dedup, so a replayed callback persists a second
llm.ai.response row for one logical response while the run's own
message_count counts it once. This inflates count_messages, duplicates a
message in list_messages pagination, and leaves the durable feed
inconsistent with the run summary.
Gate the persistence + summary block by the existing per-run_id guard so a
replayed callback is a no-op, keeping the durable message feed and the run
summary in agreement. Distinct run_ids are unaffected.
Adds regression tests: a re-fired callback for one run_id persists exactly
one row (red on main), and distinct run_ids each still persist a message.
* fix(journal): preserve canonical response on late usage
* fix(journal): preserve late usage while deduplicating responses
* fix(journal): keep first callback response canonical
* fix(journal): snapshot canonical response summaries
---------
Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com>
test_list_by_thread creates two runs back-to-back and expects the newer
one to sort first under list_by_thread's newest-first ordering. That
relies on the wall clock advancing between the two create() calls.
On Windows, datetime.now() has a coarse granularity (~15.6 ms), so both
runs can receive an identical created_at. Python's stable sort then
keeps insertion order and the assertion fails; on this host 50
consecutive now_iso() calls return identical strings.
Drive the clock with a controlled 1 ms-per-call fake (same monkeypatch
pattern as test_list_by_thread_is_stable_when_timestamps_tie) so the
strictly-newer assumption no longer depends on the host clock. The tie
case stays covered by the existing dedicated test.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(lark): enforce private ACLs on Windows credential tree
On Windows, posix chmod(0o700/0o600) does not map to NTFS ACLs, so the
secret-bearing Lark CLI credential tree was not actually owner-restricted
and existing trees were not repaired.
Branch the permission application by platform:
- POSIX: directories 0o700, files 0o600 (behavior unchanged).
- Windows: disable inherited ACLs, grant the Gateway process user Full
Control (resolved via its SID from whoami /user /fo csv /nh so it is
locale-independent), and remove broad non-administrative grants
(Everyone, Authenticated Users, Users). Fail closed on identity or
icacls failures so a tree is never left accessible silently.
Existing-tree handling is covered by asserting every entry in the tree is
repaired, and the Windows command contract is covered by mocked tests run
in CI.
* fix(lark): harden Windows credential tree against TOCTOU and hard-link races
This replaces the path-based Windows hardening (lstat -> SetFileSecurityW(path) -> iterdir) with a handle-relative walker, so validation, the ACL update, and traversal are bound to the opened object rather than a re-resolved pathname.
Every credential object is opened no-follow; children are enumerated with GetFileInformationByHandleEx(FileFullDirectoryInfo) and opened/created relative to an already-open parent handle (NtOpenFile/NtCreateFile with OBJECT_ATTRIBUTES.RootDirectory), so a pathname swap cannot redirect the walk. Credential directories are opened exclusively (share=0): SetSecurityInfo therefore does not propagate the final owner-only OI|CI DACL into as-yet-unvalidated children, and the namespace is locked for the duration of the walk (concurrent child rename/replacement and hard-link insertion fail with sharing violations). Any file with nNumberOfLinks != 1 is rejected before its security descriptor is touched, so an NTFS hard link to an external file cannot change that file owner/DACL. POSIX keeps the lstat-before-descent walk.
Tests: native regressions for exclusive no-propagation, late hard-link insertion being blocked, mid-walk junction swap being blocked, static hard-link rejection, and both real NTFS junction rejections. Mock seams updated for the handle-relative API, and Windows portability fixes make the suite green on Windows except the known #5116 sandbox-runtime executable-bit failures.
* test(lark): keep the credential-tree symlink assertion portable
The credential-tree symlink rejection is a ValueError; POSIX reports a symlink while the Windows handle-relative walker reports a reparse point. Use a platform-dependent regex so the test passes on Linux/macOS and Windows.
* fix(lark): close remaining credential-tree hardening gaps
Review follow-up for the handle-relative credential-tree walker:
- Stage the transaction snapshot under the owner-only root, copying only config/ and data/.
- Serialize ensure() per-user across threads and processes with a dedicated lock.
- Make the walker iterative so deep trees cannot hit the recursion limit.
- Re-reject a symlinked POSIX root before mkdir; drop the over-strict ancestor-chain check.
- Soften the SetSecurityInfo failure claim; add regressions for each and carry os.SEEK_END in the os stub.
* fix(lark): anchor hardening lock under trusted base and keep POSIX untouched
Follow-up refinements to the credential-tree hardening:
- The per-user hardening lock file now lives directly under the trusted base_dir
instead of the unverified per-user chain, so it is never written through an
ancestor that has not yet passed reparse validation.
- ensure() takes the hardening lock only on the Windows branch; POSIX keeps the
original contract, so no new lock-file side effect.
- Strengthen the ancestor-junction regression (lock not written to the external
target) and fix two test docstrings to match the parent-first order and the
no-prior-broadening failure claim.
* fix(lark): anchor credential-operation lock under trusted base on Windows
The per-user credential lock (_lark_credential_lock) created its advisory lock file
under the unverified per-user chain (users/<id>/integrations/.lark-cli.credentials.lock)
before ensure() validated the ancestor chain. On Windows it is now anchored directly
under the trusted paths.base_dir (mirroring the hardening lock), so a junction at
integrations can no longer cause the credential lock to be written into an external
target before reparse validation. POSIX keeps the original location unchanged.
Tests:
- Public-flow regression (start_lark_config -> credential lock -> ensure) uses an empty
sentinel lock file to prove the old credential-lock path is never opened/written.
- CLI-write re-harden tests restore the POSIX outcome assertion (file tightened to 0600).
In tool-enabled requests MindIEChatModel._astream falls back to awaiting the full _agenerate response and re-emitting it as simulated AIMessageChunks. The full response carries usage_metadata, but none of the simulated chunks copied it, so chunk aggregation (add_ai_message_chunks) produced a final message with usage_metadata=None. Token usage therefore vanished from token accounting, run stats, persistence and the UI for every tool-enabled streamed turn.
Mirror OpenAI's terminal-usage-frame convention: attach msg.usage_metadata to exactly the last simulated chunk (the trailing tool-call chunk when present, else the last text chunk / the single tool-only chunk) so the aggregated message carries it exactly once. add_usage() is per-chunk additive, so attaching usage to every chunk would multiply the totals.
Scope: MindIEChatModel only; other providers keep native streaming and ainvoke/non-tool astream were already correct.
Tests: regression guard asserting exactly one carrier chunk equals the last one and that merged usage equals the original across all three simulated-stream branches, plus chain-level tests driving the public astream() wrapper and asserting the persisted model_dump() shape.
Closes#5192
* fix(agents): make the injected current-date timezone configurable
## Why
The date reminder injected into the lead and subagent prompts (DynamicContextMiddleware / SubagentDateContextMiddleware) was formatted with the server's local wall clock. DeerFlow containers default to UTC, so a user in Asia/Shanghai chatting in the 00:00-08:00 window was told that 'today' is the previous day - the model then reasons, plans, and date-stamps against the wrong day.
## What changed
- _format_current_date() now reads the optional DEER_FLOW_DATE_TIMEZONE env var (IANA name, e.g. Asia/Shanghai) and renders the date in that zone.
- Unset = unchanged server-local behavior; invalid names log a warning and fall back to server-local.
- Documented the knob in config.example.yaml, the module docstring, and the DynamicContext entry in agents/middlewares/AGENTS.md.
## Surface area
- [x] Agents / LangGraph - prompt-layer date context only; message shape and midnight-update behavior unchanged
- [ ] Frontend UI / Backend API / Sandbox / Skills / Dependencies
- [x] Default behavior change (opt-in via env var - no behavior change unless set)
## Bug fix verification
- New tests: test_format_current_date_honors_configured_timezone (UTC 20:30 -> 2026-09-03 in Asia/Shanghai), test_format_current_date_defaults_to_server_local_without_env, test_format_current_date_invalid_timezone_falls_back.
- Existing mocked-datetime tests pass unchanged (no env -> datetime.now() path).
## Validation
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py: 31 passed.
- blocking_io/test_dynamic_context_middleware.py: 2 pre-existing abefore_agent failures reproduce identically on clean main (blockbuster os.listdir detection on this host); the other 2 pass.
- ruff format + ruff check clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): avoid passing tz to datetime.now when no timezone is configured
CI (backend-unit-tests shard 2) failed in test_tool_error_handling_middleware.py::test_subagent_chain_injects_date_without_memory_and_coalesces_for_strict_provider because its _FrozenDateTime.now() subclass override accepts no arguments, while _format_current_date() called datetime.now(None) even when DEER_FLOW_DATE_TIMEZONE was unset.
- _format_current_date() now calls datetime.now() with no arguments unless a timezone is actually configured, preserving the exact legacy call shape for every datetime-subclass test fake.
- The configured-zone path still calls datetime.now(tz) and converts via astimezone(tz).
- Updated the no-env unit test to assert datetime.now() is called without arguments.
Validation: python -m pytest tests/test_dynamic_context_middleware.py + the previously failing strict-provider test: 32 passed. ruff clean.
* fix(agents): declare the effective current-date timezone in the assembly descriptor
## Why
Maintainer review on the DEER_FLOW_DATE_TIMEZONE change (#5154): the knob is
prompt-affecting, yet both DynamicContextMiddleware and SubagentDateContextMiddleware
were invisible to the agent assembly descriptor - describe_middleware() fell back to
{"probed": true} for unset, UTC, and Asia/Shanghai alike, so deployments that inject
different dates shared one assembly fingerprint and release observers could not
distinguish or audit the behavior change.
## What changed
- Both middlewares now implement release_policy_parameters() -> dict[str, object],
declaring {"current_date_timezone": <name>} as required by the module's middleware
self-description contract.
- The declared value is the normalized effective zone: a configured, valid
DEER_FLOW_DATE_TIMEZONE is reported by its IANA key (ZoneInfo.key); otherwise the
server-local zone is resolved to its IANA key when the platform exposes one and to
its tzname label otherwise (fixed-offset hosts), with "UTC" as the final fallback.
- Added both middlewares to _MIDDLEWARE_DECLARATIONS in
backend/tests/test_middleware_release_policy.py so the existence check and the
construct-and-canonical-hash check cover them.
## Verification
- New tests: test_date_middlewares_declare_configured_timezone (Asia/Shanghai),
test_date_middlewares_declare_utc_timezone, plus resolved-server-local assertions
for the unset and invalid-env paths; both middlewares agree in every case.
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py: 70 passed.
- Regression spot-check: tests/test_agent_assembly_descriptor.py,
tests/test_tool_error_handling_middleware.py, tests/test_system_message_coalescing_middleware.py:
102 passed.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): stabilize the declared date timezone and simplify the formatting path
## Why
Follow-up review on #5154 (willem-bd). The release-policy declaration added in
884cec4b resolved the observability gap but pinned far less identity than its
docstrings claimed, and the formatting path carried a production no-op.
## What changed
- The declared label is now stable and unambiguous: a configured, valid
DEER_FLOW_DATE_TIMEZONE is reported by its IANA key; without one, the
server-local zone is resolved to a real IANA key from the TZ env var or the
/etc/localtime symlink (Linux/macOS); when no key is recoverable (Windows,
stripped containers) the declaration falls back to a stable
`server-local(+-HH:MM)` sentinel carrying the current UTC offset. It never
reports a bare abbreviation - datetime.now().astimezone() yields only a
fixed-offset timezone whose tzname (e.g. CST, EST/EDT, CET/CEST) is
ambiguous or DST-churns, which the assembly descriptor docstring says must
not happen.
- Dropped the redundant astimezone(tz) in _format_current_date():
datetime.now(tz) already returns the instant expressed in tz. The
configured-zone test now fakes datetime.now(tz) semantics (the fixed instant
converted into the requested zone) instead of relying on that conversion.
- Documented why the knob is an env var, not a config-schema field: it is read
at runtime by both date-context middlewares so an operator can point a
container at another zone without mounting a config.yaml (module docstring +
config.example.yaml note).
- AGENTS.md: fixed the glued DynamicContext sentence (missing separator).
- Added tzdata>=2025.1 to the harness runtime dependencies (with uv.lock) so
ZoneInfo works on stripped containers / Windows without an OS zone database.
## Verification
- New tests: test_server_local_timezone_name_reads_tz_env,
test_effective_timezone_sentinel_uses_offset_when_local_zone_is_not_resolvable;
reworked test_format_current_date_honors_configured_timezone to exercise the
real datetime.now(tz) path.
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_agent_assembly_descriptor.py
tests/test_tool_error_handling_middleware.py: 140 passed.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): offload subagent date injection off the event loop
## Why
Follow-up review on #5154 (willem-bd, P2): SubagentDateContextMiddleware.abefore_agent()
called _inject() directly, so enabling DEER_FLOW_DATE_TIMEZONE could synchronously
read the OS timezone database (or the tzdata wheel) on a cold cache - filesystem
work on the async subagent execution path whenever no assembly observer resolved the
zone first.
## What changed
- SubagentDateContextMiddleware.abefore_agent() now offloads the injection via
asyncio.to_thread with the same bounded timeout DynamicContextMiddleware uses
(issue #3402); on timeout it logs and skips the date update for that run instead
of blocking the loop.
- Narrowed the exception handling in _date_timezone() and the TZ-env branch of
_server_local_timezone_name() to configuration-shaped failures
(ZoneInfoNotFoundError / ValueError / OSError). Previously a blanket
`except Exception` also swallowed BlockingError raised by the blocking-I/O
regression gate, mislabeling a loop-blocking call as an invalid timezone and
silently degrading to server-local - which made the new regression anchor
useless. Other exceptions now propagate.
## Verification
- New blocking-I/O regression anchor
(backend/tests/blocking_io/test_subagent_date_context_middleware.py): drives a
real create_agent graph under the strict Blockbuster gate with the knob enabled
and asserts the date reminder is injected. Verified it fails (BlockingError) when
the offload is reverted and passes with it in place.
- python -m pytest tests/blocking_io/test_subagent_date_context_middleware.py:
1 passed. The two pre-existing os.listdir failures in
tests/blocking_io/test_dynamic_context_middleware.py reproduce unchanged on this
host (same as clean main).
- python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_tool_error_handling_middleware.py
tests/test_agent_assembly_descriptor.py: 139 passed; the single
ToolReceiptMiddleware-ordering failure reproduces with the change stashed
(local extensions registry, unrelated to this PR).
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): read the direct /etc/localtime symlink target for the zone key
## Why
Follow-up review on #5154 (willem-bd, P2): on macOS, /etc/localtime commonly
points to /var/db/timezone/zoneinfo/<zone>, but Path.resolve() follows that
directory's own symlink and yields a versioned path such as
/private/var/db/timezone/tz/2026c.1.0/zoneinfo/Asia/Shanghai, which matched no
configured prefix. The server-local resolution then returned None and the
assembly descriptor fell back to a server-local(+HH:MM) sentinel even though
the IANA key was available - conflating zones that share an offset and making
DST-based fingerprints unstable.
## What changed
- _server_local_timezone_name() now reads the direct symlink target via
os.readlink("/etc/localtime") instead of Path.resolve(), so macOS' unversioned
zoneinfo path is seen as-is and its IANA key is preserved.
- The zone key is taken from whatever follows the last "/zoneinfo/" segment,
which also handles Apple's canonical versioned path when a direct target
already carries it, and relative targets are normalized against /etc.
- Removed the now-unused Path import and the fixed zoneinfo prefix tuple.
## Verification
- New tests: test_server_local_timezone_name_reads_direct_macos_symlink_target,
test_server_local_timezone_name_reads_apple_versioned_symlink_target, and
test_server_local_timezone_name_normalizes_relative_symlink_target.
- python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_agent_assembly_descriptor.py:
105 passed (75 after re-running the first two on the merged main). The blocking
subagent anchor still passes; the two pre-existing os.listdir blocking failures
on this host are unchanged.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* bugfix #5179
* test: cover the env-aware _pick_python fallback from #5179
Follow the test_serve_nginx_stop.py extraction pattern: drive the real
_pick_python from serve.sh against a stub-only PATH plus a mocked env.
- python3 succeeds directly but fails through env -> python selected
(red on main, green on this branch)
- env rejects every candidate -> nonzero exit (also red on main)
- healthy PATH with the real env -> python3 preferred, guarding against
over-rejection
MSYS/Git Bash hosts need the stub dir as an MSYS-style (/c/...) PATH
entry, and bash diagnostics may arrive in the console code page, so the
runner decodes output with errors="replace".
