* feat(scheduler): add interval schedule type
Allow scheduled tasks to fire every N seconds from last dispatch, not
only wall-clock cron or a single run_at. Cadence is UTC now+N with no
missed-beat catch-up, bounded by min_once_delay_seconds and 30 days.
* fix(scheduler): let interval tasks create, edit, and keep next run
Create/edit now keep every_seconds. Unchanged interval spec no longer
resets next_run_at, including timezone-only PATCH.
* fix(scheduler): keep non-minute intervals on edit
Stop rounding every_seconds to whole minutes in the form. Values that
are not whole minutes or hours now use a seconds unit so edit/duplicate
round-trips the stored cadence instead of rewriting it and resetting
next_run_at. Document that min_once_delay_seconds is also the interval
floor.
* fix(scheduler): clamp interval seconds to the default 60s floor
The new seconds unit allowed 1–59, which the API rejects under the
default min_once_delay_seconds. Clamp the form to >= 60 and show the
floor next to the preview. Also mention interval in the scheduler
field_doc, matching config.example.yaml.
* fix(scheduler): do not clamp interval amount while typing
Keystroke clamp made 90 become 9 -> 60, then 600, and backspace could
not leave 60. Keep the raw field text and apply the 60s floor on blur
and emit only.
* test(scheduler): cover interval input editing
* fix(frontend): preserve saved interval cadence until edited
* style(tests): format scheduled task router tests
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(gateway): paginate thread run history (#5282)
GET /api/threads/{thread_id}/runs stays a bare array of the newest 100
runs so LangGraph SDK clients keep working. Add GET /runs/page with a
(created_at, run_id) keyset cursor so callers can walk older history.
* fix(gateway): reject one-sided run history cursors
RunManager.list_by_thread now raises if only one of before_created_at
or before_run_id is set, matching the HTTP 422. Document the per-page
sort cost on the SQL keyset query, and add the missing CHANGELOG
[#5282] link definition.
* fix(gateway): round-trip run page cursors through query strings
Emit next_before_created_at with a Z suffix so '+' is not decoded as a
space. Accept that space, and Z, when parsing. Treat blank cursor
fields as absent and reject a non-ISO before_created_at in RunManager
so a harness caller cannot silently restart at the newest page.
* style(gateway): ruff-format run page cursor files
Collapse the one-sided cursor ValueError and the two before_created_at
asserts so ruff format --check passes at line-length 240.
* fix(sandbox): stop list_dir from reporting failures as empty
Remote providers swallowed find/client errors as [] and 2>/dev/null
missing paths as empty stdout. ls_tool then told the agent the
directory was (empty). Raise OSError/FileNotFoundError instead so
the tool returns Error.
* fix(sandbox): list_dir raises on missing local paths and uses find -H
Empty stdout is not a missing path when find's start point is a
symlink (E2B /mnt/acp-workspace). Dereference only the start point
with find -H. LocalSandbox now raises FileNotFoundError for a
non-directory root, matching remote providers. AIO maps a missing
result.data to OSError rather than FileNotFoundError.
* fix(sandbox): group AIO list_dir find type predicates
Without parentheses, find PATH -maxdepth N -type f -o -type d applies
-type d without maxdepth and can drop files from the listing.
* fix(sandbox): distinguish list_dir command failure from missing path
Tenki, Boxlite, and OpenSandbox treated any empty find stdout as
FileNotFoundError, so a missing find binary (exit 127) or SDK error
looked like a missing directory. Raise OSError when find status is
outside (0, 1); keep FileNotFoundError for the find-ran-but-empty case.
* fix(sandbox): apply list_dir exit-status contract to AIO and E2B
Same gap as Tenki/Boxlite/OpenSandbox: empty find stdout with exit 127
was FileNotFoundError. Raise OSError when the status is outside (0, 1).
* fix(sandbox): classify list_dir by find status not head status
find | head under sh -lc reports head's exit code, so a missing find
binary (127) became FileNotFoundError. Record find's own status after
the bounded listing, treat SIGPIPE 141 as truncation success, and add
a shell-level regression test.
* test(auth): include projects permissions in /me contract pins
#5265 added projects:read/write/delete to the registered route set.
The /auth/me tests still pinned the pre-projects list, so CI failed
after merging main.
* fix(sandbox): do not treat missing list_dir marker as success
The generated script ended on `rm -f`, so process status was 0/1 even
when find's marker never landed. Both codes are in _FIND_OK, and the
parser fallback then classified an empty listing as FileNotFoundError —
the 127 misclassification this helper was meant to close.
Exit with find's status (126 if unknown). A missing marker is now
OSError unless the process status is already a non-OK failure.
* test(sandbox): emit list_dir status marker in provider fixtures
Parser now requires __DF_FIND_STATUS__ and refuses marker-less stdout.
Update AIO/Boxlite/E2B stubs and OpenSandbox/Tenki find fakes so listings
carry :0 and missing paths carry :1 with matching exit codes.
* style(sandbox): format list dir test fixture
* style(sandbox): format remote list dir helper
* docs(sandbox): keep guidance within the tested size budget
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): mask output tails into POSIX-style virtual paths
The output maskers slice the matched path tail from the original
output. With separator-agnostic matching, a Windows-spelled nested
tail kept its backslashes and was spliced into the POSIX-style virtual
path, so glob results and masked read output showed mixed paths like
/mnt/user-data/workspace/pkg\util.py or
/mnt/skills/integrations/lark-cli\lark-doc\SKILL.md. Virtual paths are
always POSIX-style, so normalize nested tails to forward slashes the
same way depth-1 tails already end up. Depth-1 tails and the callable
replacer (LocalSandbox._reverse_resolve_path) were unaffected.
Pin the nested-tail contract in test_sandbox_path_patterns; the
previously failing glob-tool and skills-masking regressions now pass
on Windows hosts.
* refactor(sandbox): share the mask tail-splicing rule; guard it on Linux CI
Review follow-up for #5247:
- hoist the tail-splicing rule (slice off the base, strip leading
separators, normalize the rest to "/") into
path_patterns.normalize_mask_tail and import it at both call sites,
so the two maskers can only drift in their matching logic, not in
the splice;
- add test_mask_local_paths_normalizes_windows_spelled_skill_tails,
which spells the skills host root and the output with Windows-style
strings so the nested tail keeps backslashes on every platform.
Reverting the mask_local_paths_in_output-side normalization now goes
red on Linux CI too, not only on Windows hosts.
* feat(subagents): check and persist durable batch acceptance
Carry optional per-item criteria into native subagents, reuse the deterministic checker, and expose separate verdicts through item queries and exports. Preserve execution and retry semantics, renew leases during checks, and migrate existing batch rows with nullable acceptance fields.
* fix(subagents): align batch acceptance normalization and sandbox admission
* test(auth): include project permissions in the full-stack contract
* feat(projects): project workspaces with scoped chats and thread membership
Backend:
- projects table model and migration; fail-closed ProjectRepository with
ownership checks, CRUD/archive/restore/delete router, and atomic thread
move between projects
- threads_meta.project_id column exposed as reserved deerflow_project_id
metadata; project-aware thread create/search with pagination bounds and
membership echoed in create responses
- first-run admission assigns the project only at genuine first run, seeded
at write time and dropped when invalid; serialized against project
deletion and thread assignment
- branch creation inherits the source thread's project membership (an
archived/deleted project degrades the branch to unassigned instead of
failing the request)
Frontend:
- projects data layer, thread move API, and sidebar projects section with
flat/grouped modes, archived-project threads, and stable virtual-list
offsets
- project detail page with project-scoped new chat
(/workspace/chats/new?project=) and paginated thread list
- move-to-project thread menu, new-project dialog, archived-project gates
- project-scoped new chats pre-create the thread with membership before the
first submit or /goal set, so runs never proceed outside the project
- goal-set preparation is fenced against conversation switches: a stale
continuation is dropped instead of saving the goal or launching the
abandoned submission on the newly opened conversation
- project thread lists join thread lifecycle invalidations (stop, pin) so
an open project page never keeps stale titles, recency, or pagination
* fix(chats): keep archive undo toast when the sidebar row unmounts
The archive success toast was fired from per-mutate callbacks passed to
mutation.mutate. React Query drops those handlers when the observer
component unmounts before the mutation settles; archiving the open chat
removes its sidebar row mid-flight, so the undo toast never appeared and
the e2e archive-undo test timed out waiting for it.
Move the success/error handlers to the mutation level (useArchiveThread
options, same pattern as useMoveThreadToProject) where callbacks are
delivered even after the originating row unmounts.
* fix(projects): pin project thread listing contract and exclude archived chats
GET /api/projects/{id}/threads returned the thread store row verbatim
(list[dict], no response_model): user_id/assistant_id leaked, any future
ThreadMetaRow column would auto-leak, and the OpenAPI schema was empty.
Return a narrow ProjectThreadResponse (the exact fields ProjectThread
declares) with the same metadata secret redaction the surrounding thread
endpoints get from _MetadataRedactingResponse.
The listing also ran search() without the archived filter, so a retired
chat rendered as a normal row on the project page while the sidebar hid
it. Search archived=False to mirror the sidebar's archived:false lists;
restore stays on the global Archived tab.
Both regressions pinned by new router tests: wire-shape allowlist and
archived-member exclusion.
* docs(migrations): record the 0019/0020 chain against the bootstrap reservation
The tree now chains 0018 -> 0019_projects -> 0020_threads_meta_project_id,
so migrations/AGENTS.md was stale twice over: the revision index stopped at
0018 and the rolling-forward section still claimed the tree 'deliberately
remains at 0018'.
Document the new head and record the intentional numeric-prefix reuse of
0019: 0019_projects is in-chain while 0019_thread_incarnations stays the
reserved, allowlisted out-of-tree rollout id. The owning rollout revision
must re-parent onto this tree's head when it merges so alembic never sees
two heads off 0018; bootstrap.py now cross-references that note next to
_FORWARD_COMPATIBLE_REVISION.
* fix(chats): invalidate project thread lists on archive/restore
useArchiveThread refreshed the infinite sidebar cache, threads/search and
the per-thread metadata cache but not the project-scoped list
([...PROJECTS_QUERY_KEY, 'threads', id]) this PR adds — the one thread
mutation not wired to that key, after usePinThread, useRenameThread,
useDeleteThread, useMoveThreadToProject and invalidateStoppedThreadCaches.
An archive from a sidebar row while a project page is open therefore left
the archived chat rendered as a normal row until remount (and undo left it
missing). Invalidate the prefix in the mutation-level success handler.
Regression test asserts the project-list prefix is invalidated on success.
* fix(projects): fetch project discovery only in grouped sidebar mode
RecentChatList mounted two useProjects queries per sidebar render, but
knownProjectIds is consumed only by the grouped-mode exclusion filter; in
the default flat mode every page load paid two GET /api/projects?status=
round trips for data nothing read. Gate both queries on grouped mode —
GroupedProjectList fetches the same keys when the toggle is on and
TanStack dedupes the observers.
Also set retry: false on useProject: a deleted or foreign project 404s
deterministically, and the page renders a dedicated not-found state for
it, so the default 1s/2s/4s retry backoff kept deep links in 'loading'
for ~7s before that state appeared. Matches useThreadMetadata /
useThreadTokenUsage.
* fix(threads): fail closed on project-scoped create in memory mode
MemoryThreadMetaStore.create accepted project_id and silently ignored it,
making memory mode the one membership path that fails open: POST
/api/threads with a project id returned 200 and the run started
unassigned, violating the invariant that a run never proceeds outside the
selected project (the SQL store raises ProjectNotAssignableError inside
the insert transaction for the same request).
Raise ProjectNotAssignableError whenever project_id is present so the
router's existing 404 mapping applies, the frontend keeps the composer
text for a retry, and memory mode behaves exactly like SQL mode.
set_project already reports rejection; create now matches it.
Store-level test (raises, nothing persisted, project filter stays empty,
unscoped creates still work) plus a router-level test asserting the 404
and that no row is left behind.
* fix(projects): window the project page thread list
ProjectThreadsSection rendered every loaded page as a plain Link row, so a
long-lived project accumulated unbounded DOM on the page's scroll surface:
each load-more appended another 100 rows and every formatTimeAgo tick
re-rendered the whole list.
Reuse VirtualThreadList (now generic over any row shape with a
thread_id), pointing its scroll parent at this page's ScrollArea viewport
via the shared [data-slot="scroll-area-viewport"] selector used by
/workspace/chats; under the 60-row threshold it falls back to the plain
render, so small projects are unchanged.
* fix(projects): restore row dividers and pin them with a render test
The row class template literal concatenated transition-colors directly
with the conditional border-b token, so non-final rows rendered the
invalid class 'transition-colorsborder-b' and lost both the divider and
the transition. Compose the row classes with cn() and a boolean guard
instead.
The section moved out of page.tsx into a testable component so the row
markup finally has coverage: a DOM test asserts every row except the
final data row carries border-b (index-based, not last: — correct under
virtualization where the last mounted row is not the last data row), and
the untitled fallback plus load-more button render for a partial page.
* fix(projects): validate forward schemas and fence membership reads
* fix: preserve actionable subagent acceptance gaps
Distinguish completed execution from acceptance in delegation guidance. Retain bounded unmet and unverified criteria after compaction and guide the lead to address remaining work within its budget.
* docs: keep acceptance guidance within instruction budget
* fix(workspace-changes): record symlink targets without the verbatim prefix
os.readlink on Windows reports absolute targets in extended-length
form (\?\C:\... or \?\UNC\server\share). The scanner stored that raw
spelling, so workspace-change events showed \?\-prefixed targets that
do not match ordinary Windows paths. Strip the prefix when recording;
POSIX readlink output is unchanged. Skills projection/review readlink
sites are untouched — they have no user-facing contract pinned on the
spelling.
* fix(workspace-changes): gate symlink target normalization to Windows
Review follow-up on #5250:
- Gate _normalize_symlink_target on os.name == "nt". readlink(2) on
POSIX returns the literal string the link was created with, and
backslash is a valid filename byte on Linux, so a target that starts
with the extended-length prefix there must be recorded verbatim. The
docstring's POSIX claim is now provably true.
- Commit the unit checks the PR body previously described as ad-hoc:
drive and UNC prefix stripping, relative and plain POSIX targets,
mid-string prefix left verbatim, and an off-Windows identity case, so
the new branch has real coverage on every platform instead of relying
on a Windows host with symlink privilege.
* test(workspace-changes): force Windows platform in prefix-strip unit tests
The os.name gate added in the previous commit makes
_normalize_symlink_target a verbatim identity off-Windows, so the two
prefix-strip assertions failed on the ubuntu-only unit CI. Force
os.name to "nt" via monkeypatch in both, mirroring the off-Windows
identity test, so every case pins exactly one platform's contract and
the suite is green on every host.
* fix(workspace-changes): strip only extended drive-letter prefixes
Review follow-up on #5250: the catch-all branch also stripped the
extended-length prefix from volume-GUID targets (\?\Volume{...}\...),
leaving a relative-looking path that loses the target's namespace.
Restrict the branch to extended drive-letter paths (letter, colon,
separator) and keep every other \?\ namespace form verbatim; add the
volume-GUID regression plus degenerate-prefix cases.
* feat(community): add Sofya web search provider
Add a community provider backed by Sofya (https://sofya.co). Its search
endpoint returns the content of the result pages, not only their snippets,
and its fetch endpoint returns a page as markdown. Both are plain JSON over
HTTP, so this needs no extra Python package (uses httpx, already a
dependency).
Changes:
- backend/packages/harness/deerflow/community/sofya/__init__.py
- backend/packages/harness/deerflow/community/sofya/tools.py
Implements web_search_tool and web_fetch_tool using httpx.
API key is read from the config.yaml `api_key` field or the SOFYA_API_KEY
env var. Follows the same interface and output shape as the existing
ddg_search and serper providers, including the max_results parameter with
config override and the structured "No results found" error.
- backend/tests/test_sofya_tools.py
Unit tests covering API key resolution, config overrides, result mapping,
time range, HTTP errors, empty results, and fetch failures.
- config.example.yaml: add commented-out Sofya web_search and web_fetch
examples alongside the other providers
- .env.example: add SOFYA_API_KEY placeholder
- backend/docs/CONFIGURATION.md: list Sofya under web_search, web_fetch and
the environment variables
* fix(sofya): honor caller max_results, validate search_depth, join time_range contract test
- Caller-supplied max_results now wins; config is used only when the
argument is omitted, matching GroundRoute.
- search_depth is clamped to basic/snippets; an unsupported value logs a
warning and falls back to basic.
- Sofya added to the shared time_range schema contract test.
* fix(sofya): cap per-result content so a search stays inline
An unbounded search payload (up to 20 read pages) crossed the tool output
budget middleware's externalize_min_chars threshold, which replaces the
result list with a file reference. Cap each result's content at
contents_max_characters (default 2000, 0 disables), matching Exa's config
key. Five capped results stay under the 12000 char threshold.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* fix(sofya): list Sofya in the recency contract, coerce non-string content
_clip subscripted its input, so a non-string content or description from
the API raised TypeError instead of degrading. Coerce to text first, the
way _sofya_post and _response_results guard the shapes around it. Also add
Sofya to the Web Search Recency section in backend/AGENTS.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* fix(sofya): coerce web_fetch content, list sofya in the tools guide, add changelog
web_fetch sliced its content the same way web_search did before the last
push: a truthy non-string from the API passed the falsiness guard and then
raised TypeError. Reuse _clip, keeping the `or ""` so empty content still
reports "No content found".
Also add sofya to the community provider inventory in
packages/harness/deerflow/tools/AGENTS.md and an [Unreleased] changelog entry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
* docs(zh): add the missing InfoQuest and Firecrawl web_fetch tabs
The ZH web_fetch tab list named five providers where EN names seven. Both
tabs mirror their EN counterparts, so the two locales list the same
web_fetch providers again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): stop E2B append from overwriting on read failure
E2B has no native append, so write_file(append=True) read-modify-writes.
Treat only FileNotFoundException/FileNotFoundError as an empty file; any
other pre-read error must abort so a timeout cannot replace the original
contents with just the new fragment.
* fix(sandbox): distinguish E2B append pre-read refusal in logs
A non-not-found pre-read error now logs as a refused overwrite instead
of a write failure. Tests pin the successful read-modify-write path,
including a bytes pre-image, so dropping `existing` cannot go green.
* fix(mcp): resolve drive-qualified paths in file reference rewriting
urlparse reads a Windows drive prefix ("C:/...") as the URI scheme, so
_local_path_from_uri() returned None for every drive-qualified path and
MCP file references were never rewritten to /mnt/user-data/... virtual
paths on Windows hosts. file:// URIs were parsed with urlparse().path
alone, which also drops the drive qualifier.
- resolve file URIs through url2pathname so the /C:/... form keeps its
drive, and treat single-letter schemes as bare drive paths;
- match drive-qualified absolute paths in the free-text reference regex;
- build test URIs with Path.as_uri() and anchor absolute-path fixtures
at tmp_path so expectations are host-portable, and cover the
drive-prefix scheme quirk explicitly.
* fix(mcp): decode file URIs once and guard Windows path rejection
Review follow-up on #5242:
- url2pathname already percent-decodes on both platforms, so the extra
unquote() wrapper decoded references twice and broke filenames that
contain a literal '%'. Pass parsed.path straight through.
- On Windows, url2pathname raises OSError for paths containing a raw
'|' (e.g. file:///C:/tmp/a|b.png); catch it so one odd URI cannot
abort the whole best-effort rewrite pass.
- The relative-reference regex alternative now accepts backslash
separators, which is what Windows servers print for relative paths.
- Add Windows-only regressions driving the backslash free-text form
and a file:///C:/ URI end to end, plus the OSError rejection.
* fix(mcp): resolve file://C:/… URIs with a drive-qualified authority
Review follow-up on #5242 (two-slash Windows drive form):
- Some Windows tools emit file://C:/… without the third slash, which
puts the drive in the URI authority. Consult parsed.netloc: rebuild
the /C:/… URL path for a drive-qualified authority, keep the current
handling for empty and localhost authorities, and reject any other
host instead of silently treating its path as local.
- Extend the free-text regex so the two-slash form matches as one token
instead of the previous stray e://… mid-token match.
- Cover the two-slash form at the _local_path_from_uri unit, through
_rewrite_local_paths_in_text, and add a portable case asserting that
a remote-host file URI is ignored.
* fix(mcp): anchor the drive-qualified text alternative with a lookbehind
Review follow-up on #5242:
- [A-Za-z]:[\/] could steal a token at an earlier scan position:
for file:/tmp/… (single-slash form per RFC 8089 / Java File.toURI())
the match became e:/tmp/…, which resolves as a bare drive path and
left the reference unrewritten where /tmp/… was rewritten before.
Anchor the alternative with (?<![\w.-]) so word:/… shapes fall
through to the earlier alternatives.
- Add the missing coverage for the relative alternative's backslash
support (temp\page.yml through _rewrite_local_paths_in_text) and
a portable regression pinning the file:/… tokenization.
* fix(agents): prioritize loop hard stops across tool batches
Scan an admitted multi-tool response completely before selecting a soft warning, so any configured hard limit can reject the whole batch. Preserve warning priority and sliding-window accounting.
Add counter-level plus sync and async compiled-agent regressions proving rejected tools are not executed.
Fixes bytedance/deer-flow#5243. AI-assisted implementation and tests.
* fix(agents): rearm loop warnings after cross-tool eviction
When another tool evicts an older tool below its frequency warning threshold, clear the older suppression mark so a later burst can warn again.
Add the cross-tool sliding-window regression from the final boundary review. AI-assisted implementation and tests.
* test(agents): cover override-aware loop warning rearm
Cache the default frequency thresholds for sliding-window eviction and verify that an evicted tool uses its configured override when warning eligibility is rearmed.
Document that simultaneous frequency warnings preserve legacy first-crossing selection while hard stops remain batch-severity-first.
Addresses review on #5245. AI-assisted implementation and tests.
---------
Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(journal): dedup llm.ai.response persistence on re-fired on_llm_end
LangChain may deliver on_llm_end more than once for the same run_id.
RunJournal already dedups token accounting and the run summary
(_record_message_summary) on that premise via _counted_message_llm_run_ids,
but the durable llm.ai.response self._put() call was left unguarded.
The event store is append-only and count_messages/list_messages read raw
rows without read-time dedup, so a replayed callback persists a second
llm.ai.response row for one logical response while the run's own
message_count counts it once. This inflates count_messages, duplicates a
message in list_messages pagination, and leaves the durable feed
inconsistent with the run summary.
Gate the persistence + summary block by the existing per-run_id guard so a
replayed callback is a no-op, keeping the durable message feed and the run
summary in agreement. Distinct run_ids are unaffected.
Adds regression tests: a re-fired callback for one run_id persists exactly
one row (red on main), and distinct run_ids each still persist a message.
* fix(journal): preserve canonical response on late usage
* fix(journal): preserve late usage while deduplicating responses
* fix(journal): keep first callback response canonical
* fix(journal): snapshot canonical response summaries
---------
Co-authored-by: CorgiBoyG <CorgiBoyG@users.noreply.github.com>
* fix(lark): enforce private ACLs on Windows credential tree
On Windows, posix chmod(0o700/0o600) does not map to NTFS ACLs, so the
secret-bearing Lark CLI credential tree was not actually owner-restricted
and existing trees were not repaired.
Branch the permission application by platform:
- POSIX: directories 0o700, files 0o600 (behavior unchanged).
- Windows: disable inherited ACLs, grant the Gateway process user Full
Control (resolved via its SID from whoami /user /fo csv /nh so it is
locale-independent), and remove broad non-administrative grants
(Everyone, Authenticated Users, Users). Fail closed on identity or
icacls failures so a tree is never left accessible silently.
Existing-tree handling is covered by asserting every entry in the tree is
repaired, and the Windows command contract is covered by mocked tests run
in CI.
* fix(lark): harden Windows credential tree against TOCTOU and hard-link races
This replaces the path-based Windows hardening (lstat -> SetFileSecurityW(path) -> iterdir) with a handle-relative walker, so validation, the ACL update, and traversal are bound to the opened object rather than a re-resolved pathname.
Every credential object is opened no-follow; children are enumerated with GetFileInformationByHandleEx(FileFullDirectoryInfo) and opened/created relative to an already-open parent handle (NtOpenFile/NtCreateFile with OBJECT_ATTRIBUTES.RootDirectory), so a pathname swap cannot redirect the walk. Credential directories are opened exclusively (share=0): SetSecurityInfo therefore does not propagate the final owner-only OI|CI DACL into as-yet-unvalidated children, and the namespace is locked for the duration of the walk (concurrent child rename/replacement and hard-link insertion fail with sharing violations). Any file with nNumberOfLinks != 1 is rejected before its security descriptor is touched, so an NTFS hard link to an external file cannot change that file owner/DACL. POSIX keeps the lstat-before-descent walk.
Tests: native regressions for exclusive no-propagation, late hard-link insertion being blocked, mid-walk junction swap being blocked, static hard-link rejection, and both real NTFS junction rejections. Mock seams updated for the handle-relative API, and Windows portability fixes make the suite green on Windows except the known #5116 sandbox-runtime executable-bit failures.
* test(lark): keep the credential-tree symlink assertion portable
The credential-tree symlink rejection is a ValueError; POSIX reports a symlink while the Windows handle-relative walker reports a reparse point. Use a platform-dependent regex so the test passes on Linux/macOS and Windows.
* fix(lark): close remaining credential-tree hardening gaps
Review follow-up for the handle-relative credential-tree walker:
- Stage the transaction snapshot under the owner-only root, copying only config/ and data/.
- Serialize ensure() per-user across threads and processes with a dedicated lock.
- Make the walker iterative so deep trees cannot hit the recursion limit.
- Re-reject a symlinked POSIX root before mkdir; drop the over-strict ancestor-chain check.
- Soften the SetSecurityInfo failure claim; add regressions for each and carry os.SEEK_END in the os stub.
* fix(lark): anchor hardening lock under trusted base and keep POSIX untouched
Follow-up refinements to the credential-tree hardening:
- The per-user hardening lock file now lives directly under the trusted base_dir
instead of the unverified per-user chain, so it is never written through an
ancestor that has not yet passed reparse validation.
- ensure() takes the hardening lock only on the Windows branch; POSIX keeps the
original contract, so no new lock-file side effect.
- Strengthen the ancestor-junction regression (lock not written to the external
target) and fix two test docstrings to match the parent-first order and the
no-prior-broadening failure claim.
* fix(lark): anchor credential-operation lock under trusted base on Windows
The per-user credential lock (_lark_credential_lock) created its advisory lock file
under the unverified per-user chain (users/<id>/integrations/.lark-cli.credentials.lock)
before ensure() validated the ancestor chain. On Windows it is now anchored directly
under the trusted paths.base_dir (mirroring the hardening lock), so a junction at
integrations can no longer cause the credential lock to be written into an external
target before reparse validation. POSIX keeps the original location unchanged.
Tests:
- Public-flow regression (start_lark_config -> credential lock -> ensure) uses an empty
sentinel lock file to prove the old credential-lock path is never opened/written.
- CLI-write re-harden tests restore the POSIX outcome assertion (file tightened to 0600).
In tool-enabled requests MindIEChatModel._astream falls back to awaiting the full _agenerate response and re-emitting it as simulated AIMessageChunks. The full response carries usage_metadata, but none of the simulated chunks copied it, so chunk aggregation (add_ai_message_chunks) produced a final message with usage_metadata=None. Token usage therefore vanished from token accounting, run stats, persistence and the UI for every tool-enabled streamed turn.
Mirror OpenAI's terminal-usage-frame convention: attach msg.usage_metadata to exactly the last simulated chunk (the trailing tool-call chunk when present, else the last text chunk / the single tool-only chunk) so the aggregated message carries it exactly once. add_usage() is per-chunk additive, so attaching usage to every chunk would multiply the totals.
Scope: MindIEChatModel only; other providers keep native streaming and ainvoke/non-tool astream were already correct.
Tests: regression guard asserting exactly one carrier chunk equals the last one and that merged usage equals the original across all three simulated-stream branches, plus chain-level tests driving the public astream() wrapper and asserting the persisted model_dump() shape.
Closes#5192
* fix(agents): make the injected current-date timezone configurable
## Why
The date reminder injected into the lead and subagent prompts (DynamicContextMiddleware / SubagentDateContextMiddleware) was formatted with the server's local wall clock. DeerFlow containers default to UTC, so a user in Asia/Shanghai chatting in the 00:00-08:00 window was told that 'today' is the previous day - the model then reasons, plans, and date-stamps against the wrong day.
## What changed
- _format_current_date() now reads the optional DEER_FLOW_DATE_TIMEZONE env var (IANA name, e.g. Asia/Shanghai) and renders the date in that zone.
- Unset = unchanged server-local behavior; invalid names log a warning and fall back to server-local.
- Documented the knob in config.example.yaml, the module docstring, and the DynamicContext entry in agents/middlewares/AGENTS.md.
## Surface area
- [x] Agents / LangGraph - prompt-layer date context only; message shape and midnight-update behavior unchanged
- [ ] Frontend UI / Backend API / Sandbox / Skills / Dependencies
- [x] Default behavior change (opt-in via env var - no behavior change unless set)
## Bug fix verification
- New tests: test_format_current_date_honors_configured_timezone (UTC 20:30 -> 2026-09-03 in Asia/Shanghai), test_format_current_date_defaults_to_server_local_without_env, test_format_current_date_invalid_timezone_falls_back.
- Existing mocked-datetime tests pass unchanged (no env -> datetime.now() path).
## Validation
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py: 31 passed.
- blocking_io/test_dynamic_context_middleware.py: 2 pre-existing abefore_agent failures reproduce identically on clean main (blockbuster os.listdir detection on this host); the other 2 pass.
- ruff format + ruff check clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): avoid passing tz to datetime.now when no timezone is configured
CI (backend-unit-tests shard 2) failed in test_tool_error_handling_middleware.py::test_subagent_chain_injects_date_without_memory_and_coalesces_for_strict_provider because its _FrozenDateTime.now() subclass override accepts no arguments, while _format_current_date() called datetime.now(None) even when DEER_FLOW_DATE_TIMEZONE was unset.
- _format_current_date() now calls datetime.now() with no arguments unless a timezone is actually configured, preserving the exact legacy call shape for every datetime-subclass test fake.
- The configured-zone path still calls datetime.now(tz) and converts via astimezone(tz).
- Updated the no-env unit test to assert datetime.now() is called without arguments.
Validation: python -m pytest tests/test_dynamic_context_middleware.py + the previously failing strict-provider test: 32 passed. ruff clean.
* fix(agents): declare the effective current-date timezone in the assembly descriptor
## Why
Maintainer review on the DEER_FLOW_DATE_TIMEZONE change (#5154): the knob is
prompt-affecting, yet both DynamicContextMiddleware and SubagentDateContextMiddleware
were invisible to the agent assembly descriptor - describe_middleware() fell back to
{"probed": true} for unset, UTC, and Asia/Shanghai alike, so deployments that inject
different dates shared one assembly fingerprint and release observers could not
distinguish or audit the behavior change.
## What changed
- Both middlewares now implement release_policy_parameters() -> dict[str, object],
declaring {"current_date_timezone": <name>} as required by the module's middleware
self-description contract.
- The declared value is the normalized effective zone: a configured, valid
DEER_FLOW_DATE_TIMEZONE is reported by its IANA key (ZoneInfo.key); otherwise the
server-local zone is resolved to its IANA key when the platform exposes one and to
its tzname label otherwise (fixed-offset hosts), with "UTC" as the final fallback.
- Added both middlewares to _MIDDLEWARE_DECLARATIONS in
backend/tests/test_middleware_release_policy.py so the existence check and the
construct-and-canonical-hash check cover them.
## Verification
- New tests: test_date_middlewares_declare_configured_timezone (Asia/Shanghai),
test_date_middlewares_declare_utc_timezone, plus resolved-server-local assertions
for the unset and invalid-env paths; both middlewares agree in every case.
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py: 70 passed.
- Regression spot-check: tests/test_agent_assembly_descriptor.py,
tests/test_tool_error_handling_middleware.py, tests/test_system_message_coalescing_middleware.py:
102 passed.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): stabilize the declared date timezone and simplify the formatting path
## Why
Follow-up review on #5154 (willem-bd). The release-policy declaration added in
884cec4b resolved the observability gap but pinned far less identity than its
docstrings claimed, and the formatting path carried a production no-op.
## What changed
- The declared label is now stable and unambiguous: a configured, valid
DEER_FLOW_DATE_TIMEZONE is reported by its IANA key; without one, the
server-local zone is resolved to a real IANA key from the TZ env var or the
/etc/localtime symlink (Linux/macOS); when no key is recoverable (Windows,
stripped containers) the declaration falls back to a stable
`server-local(+-HH:MM)` sentinel carrying the current UTC offset. It never
reports a bare abbreviation - datetime.now().astimezone() yields only a
fixed-offset timezone whose tzname (e.g. CST, EST/EDT, CET/CEST) is
ambiguous or DST-churns, which the assembly descriptor docstring says must
not happen.
- Dropped the redundant astimezone(tz) in _format_current_date():
datetime.now(tz) already returns the instant expressed in tz. The
configured-zone test now fakes datetime.now(tz) semantics (the fixed instant
converted into the requested zone) instead of relying on that conversion.
- Documented why the knob is an env var, not a config-schema field: it is read
at runtime by both date-context middlewares so an operator can point a
container at another zone without mounting a config.yaml (module docstring +
config.example.yaml note).
- AGENTS.md: fixed the glued DynamicContext sentence (missing separator).
- Added tzdata>=2025.1 to the harness runtime dependencies (with uv.lock) so
ZoneInfo works on stripped containers / Windows without an OS zone database.
## Verification
- New tests: test_server_local_timezone_name_reads_tz_env,
test_effective_timezone_sentinel_uses_offset_when_local_zone_is_not_resolvable;
reworked test_format_current_date_honors_configured_timezone to exercise the
real datetime.now(tz) path.
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_agent_assembly_descriptor.py
tests/test_tool_error_handling_middleware.py: 140 passed.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): offload subagent date injection off the event loop
## Why
Follow-up review on #5154 (willem-bd, P2): SubagentDateContextMiddleware.abefore_agent()
called _inject() directly, so enabling DEER_FLOW_DATE_TIMEZONE could synchronously
read the OS timezone database (or the tzdata wheel) on a cold cache - filesystem
work on the async subagent execution path whenever no assembly observer resolved the
zone first.
## What changed
- SubagentDateContextMiddleware.abefore_agent() now offloads the injection via
asyncio.to_thread with the same bounded timeout DynamicContextMiddleware uses
(issue #3402); on timeout it logs and skips the date update for that run instead
of blocking the loop.
- Narrowed the exception handling in _date_timezone() and the TZ-env branch of
_server_local_timezone_name() to configuration-shaped failures
(ZoneInfoNotFoundError / ValueError / OSError). Previously a blanket
`except Exception` also swallowed BlockingError raised by the blocking-I/O
regression gate, mislabeling a loop-blocking call as an invalid timezone and
silently degrading to server-local - which made the new regression anchor
useless. Other exceptions now propagate.
## Verification
- New blocking-I/O regression anchor
(backend/tests/blocking_io/test_subagent_date_context_middleware.py): drives a
real create_agent graph under the strict Blockbuster gate with the knob enabled
and asserts the date reminder is injected. Verified it fails (BlockingError) when
the offload is reverted and passes with it in place.
- python -m pytest tests/blocking_io/test_subagent_date_context_middleware.py:
1 passed. The two pre-existing os.listdir failures in
tests/blocking_io/test_dynamic_context_middleware.py reproduce unchanged on this
host (same as clean main).
- python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_tool_error_handling_middleware.py
tests/test_agent_assembly_descriptor.py: 139 passed; the single
ToolReceiptMiddleware-ordering failure reproduces with the change stashed
(local extensions registry, unrelated to this PR).
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): read the direct /etc/localtime symlink target for the zone key
## Why
Follow-up review on #5154 (willem-bd, P2): on macOS, /etc/localtime commonly
points to /var/db/timezone/zoneinfo/<zone>, but Path.resolve() follows that
directory's own symlink and yields a versioned path such as
/private/var/db/timezone/tz/2026c.1.0/zoneinfo/Asia/Shanghai, which matched no
configured prefix. The server-local resolution then returned None and the
assembly descriptor fell back to a server-local(+HH:MM) sentinel even though
the IANA key was available - conflating zones that share an offset and making
DST-based fingerprints unstable.
## What changed
- _server_local_timezone_name() now reads the direct symlink target via
os.readlink("/etc/localtime") instead of Path.resolve(), so macOS' unversioned
zoneinfo path is seen as-is and its IANA key is preserved.
- The zone key is taken from whatever follows the last "/zoneinfo/" segment,
which also handles Apple's canonical versioned path when a direct target
already carries it, and relative targets are normalized against /etc.
- Removed the now-unused Path import and the fixed zoneinfo prefix tuple.
## Verification
- New tests: test_server_local_timezone_name_reads_direct_macos_symlink_target,
test_server_local_timezone_name_reads_apple_versioned_symlink_target, and
test_server_local_timezone_name_normalizes_relative_symlink_target.
- python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_agent_assembly_descriptor.py:
105 passed (75 after re-running the first two on the merged main). The blocking
subagent anchor still passes; the two pre-existing os.listdir blocking failures
on this host are unchanged.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(runtime): reclaim idle keyed locks safely
Replace the per-loop thread lock registries with a waiter-aware keyed lock table. Count holders and queued waiters before acquisition so idle entries can be reclaimed without allowing a late caller to bypass an existing waiter.
Add regression coverage for runtime call-site reclamation, goal/checkpoint domain independence, queued-waiter ordering, cancellation cleanup, high-cardinality key reclamation, and cross-event-loop isolation.
Fixes#5171
* style(runtime): format keyed lock helper
* fix(mcp): keep ToolRuntime injection for sync-wrapped MCP tools
make_sync_tool_wrapper attached an annotation-less wrapper to tool.func,
which made LangGraph's ToolNode stop detecting the coroutine's
"runtime" parameter (_get_all_injected_args falls back to func first
and its type hints are empty). Every MCP tool in a sync agent caller
then ran with runtime=None: resolve_runtime_user_id fell through to the
default user, and the background-submit wrapper lost run_id/tool_call_id
on the TaskSubmitRequest, so completion notifications launched under the
default lead agent instead of the thread's agent.
Wrap the generator and both sync_wrapper variants with functools.wraps
so get_type_hints still sees the original annotations.
Adds a regression test that drives a func-patched pooled MCP tool
through a real ToolNode and asserts the ToolRuntime is injected with
the thread's user context. It fails on main (runtime=None) and passes
with the fix.
* docs(mcp): record sync-wrapper annotation contract; extend regression coverage
Address review feedback on #5164:
- Expand the Notes block in make_sync_tool_wrapper to state the functools.wraps
contract (copies __name__/__qualname__/__doc__/__annotations__/__dict__ and
sets __wrapped__) and why that is what keeps get_type_hints resolving string
annotations from callers like
mcp/tools.py and skill_manage_tool.py. Drop the no-op wraps on the inner
run_coroutine so the wrap surface stays minimal.
- Rename the regression test to test_func_patched_mcp_tool_keeps_toolnode_runtime_injection.
- Add test_sync_wrapped_builtin_tools_still_resolve_runtime to pin that the
built-in tools (which carry runtime as a pydantic schema field) keep resolving
runtime after their func is wrapped by make_sync_tool_wrapper, so a future
wrapper refactor cannot silently regress per-user resolution for them.
* Fixes#5060: P1 snapshot config before loading, P2 RLock for sync path
P1: Config changes during initialization can permanently cache stale tools.
- Snapshot _config_path and _config_signature BEFORE await get_mcp_tools()
- Compare AFTER get_mcp_tools() completes using _current_config_state()
- If config changed during loading, discard stale result and retry
- Prevents publishing old tools with new signature, which would make
_is_cache_stale() permanently return False
P2: Module-level asyncio.Lock still fails across event loops after real contention.
- _init_lock = threading.RLock() for sync path (reentrant, prevents races)
- _async_init_lock = asyncio.Lock() for async init serialization
- reset_mcp_tools_cache() now acquires _init_lock for serialization
Also fixes test P3: removed duplicated test bodies that leaked state between tests.
* fix(mcp): make cache initialization cross-loop safe
- Replace the module-level asyncio.Lock with thread-safe generation claiming
- Snapshot config state before/after MCP loading and discard stale results
- Keep reset state changes short and non-blocking for async endpoints
- Add regression coverage for contended cross-loop init, config rewrites during load, and reset while init is in flight
* fix: release MCP init claim on cancellation
Release the in-flight generation claim from a cancellation-safe finally block so cancelling the task that owns initialization does not strand future callers. Add regression coverage for cancelling the owner and then reinitializing successfully.
* fix(mcp): retire session pool before cache reset release
Prevent a concurrent MCP cache initializer from publishing tool wrappers
bound to the session-pool singleton that reset_mcp_tools_cache() is
already retiring. Add regression coverage for that interleaving.
* fix(mcp): retire session pool on stale cache invalidation
* fix(mcp): retire pool on init discard
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(e2b-sandbox): make mount upload deadline configurable
Replace the hardcoded 120-second mount upload deadline with a
configurable `mount_upload_deadline_seconds` key read from
SandboxConfig (extra=allow). The value is validated: zero and
negative inputs are clamped to 1 second. Omitting the key
preserves the existing 120-second default.
This addresses the follow-up from PR #4842 review: operators
with large mounts or slow networks can now size the deadline to
their deployment without changing code.
* fix(e2b-sandbox): address review feedback on configurable deadline
- Remove import-time default capture from _mount_deadline_reason()
and _MountUploadBudget.deadline_seconds to prevent silent drift.
- Add warning log when mount_upload_deadline_seconds is clamped to 1
(was silent before).
- Update AGENTS.md E2B Mount Uploads section: deadline is now
configurable, not fixed 120.
- Add mount_upload_deadline_seconds to YAML examples in provider
docstring and __init__.py.
- Add config-path test that exercises SandboxConfig -> _load_config ->
_apply_mounts end-to-end.
* feat(e2b-sandbox): surface structured mount upload result on sandbox
Introduce MountUploadResult dataclass and attach it to
E2BSandbox.mount_upload_result after creation. This makes mount
truncation observable in code without re-parsing Gateway logs.
_apply_mounts() now returns MountUploadResult with truncated, reason,
and upload totals. _create_sandbox() captures the result, stores it on
the sandbox instance, and records it in a provider-level map so the
result survives warm-pool reclaim and reconnect.
MountUploadResult.truncated is True only when the upload pass was
stopped early by a resource limit (deadline, file count cap, or byte
budget). Individual mount failures (missing host path, SDK errors) are
logged but do NOT set truncated.
Tests cover: success totals, deadline truncation, file-count truncation,
byte-budget truncation, non-limit failure not reported as truncation,
missing host path not reported as truncation, create→sandbox wiring,
and create→release→warm-pool→acquire result preservation.
* fix(e2b-sandbox-provider): fix _mount_results lifecycle leak and review findings
- Add _forget_mount_result() helper and call it at all terminal sandbox
paths: _reuse_in_process_sandbox dead-evict, _reclaim_warm_pool_sandbox
reconnect/dead/bootstrap/ownership/shutdown failure branches,
_forget_local_sandbox, _kill_and_close. Prevents unbounded dict growth
over a long-running Gateway process.
- Make MountUploadResult @dataclass(frozen=True) to prevent silent mutation
of the shared reference between provider map and sandbox attribute.
- Move _mount_results insert under self._lock in _create_sandbox to match
the read discipline in _register_connected_sandbox.
- Guard _resolve_mount_upload_deadline against None (YAML explicit null)
to avoid int(None) TypeError.
- Add 5 regression tests covering each bypass path and the frozen invariant.
* fix(e2b-sandbox-provider): add _forget_mount_result to _evict_oldest_warm branches
Add _forget_mount_result() calls to all four terminal exit paths in the
E2B _evict_oldest_warm override (reconnect failure, already-gone, kill
failure, kill success). The peer-owned path already cleans up via
_forget_local_sandbox. Add test_evict_oldest_warm_cleans_mount_result to
pin the kill-success branch.
* docs: reduce agent guidance size
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
## Why
DeerFlow declares one timestamp convention in deerflow/utils/time.py: every lifecycle timestamp is UTC (now_iso / datetime.now(UTC)). SubagentResult writers in subagents/executor.py still used naive datetime.now(), so on any non-UTC host the in-memory lifecycle metadata (started_at / completed_at) was local wall-clock time. The sibling durable-batch path (subagents/batch_service.py) already stamps datetime.now(UTC), so the same run model carried two different conventions depending on which path wrote it.
## What changed
- Added an executor-local _utcnow() helper that stamps datetime.now(UTC).
- SubagentResult.completed_at default in try_set_terminal(), result.started_at in _aexecute(), and the started_at default in _aexecute_admitted() now route through _utcnow().
- Explicit caller-supplied timestamps (completed_at=...) still pass through unchanged.
- Added regression tests asserting the default writers produce UTC-aware datetimes.
## Surface area
- [x] Backend runtime (deerflow.subagents.executor) - internal dataclass lifecycle metadata, no wire format change
- [ ] Frontend UI / Backend API / Sandbox / Skills / Dependencies / Default behavior change
## Bug fix verification
- New tests: tests/test_subagent_executor.py::test_timestamp_writers_stamp_utc_aware_datetimes and test_utcnow_helper_returns_utc_aware_datetime encode the convention.
- Updated BlockingDateTime.now() in the terminal-publication-order test to mirror datetime.now's optional tz argument.
## Validation
- cd backend && python -m pytest tests/test_subagent_executor.py: 136 passed; 2 pre-existing TestBashExecutionHarvest failures reproduce identically on clean main (Windows sandbox env), unrelated to this change.
- ruff format + ruff check clean on both changed files.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis of the timestamp conventions, implementation, and regression tests authored with AI assistance; change reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(summarization): resolve fraction triggers from declared context_window, degrade instead of crashing the agent build
A fraction trigger/keep clause requires profile["max_input_tokens"], which any
third-party OpenAI-compatible model lacks, so SummarizationMiddleware
construction raised ValueError out of create_summarization_middleware and failed
the whole agent build (#3103).
- factory: translate a declared model context_window into the langchain
profile (metadata-only, never reaches the provider payload); explicit
caller/override profiles win
- summarization factory: drop unusable fraction trigger clauses (absolute
clauses survive), fall a fraction keep back to the messages default, and
disable compaction with an actionable warning only when no usable trigger
clause remains — the agent build never dies from summarization config
- docs: config.example.yaml, ModelConfig.context_window, summarization.md
* refactor(summarization): share the default keep constant with the fraction fallback
The fraction-keep degradation fallback hardcoded ("messages", 20),
duplicating SummarizationConfig.keep's default_factory literal. Move the
value to a shared DEFAULT_KEEP constant so the two cannot drift apart.
* fix(summarization): keep trigger-null + fraction-keep constructing after degradation
A trigger of None with a fraction keep hit the all-clauses-dropped branch
(has_usable_trigger=False) and disabled compaction, and the accompanying
warning claimed configured triggers were all fraction-based when none were
configured. Only report nothing-usable when trigger clauses actually
existed; trigger:null keeps constructing the never-firing middleware with
the degraded keep, matching its behavior outside the degradation path.
* fix(summarization): address review — keep manual compaction, validate ContextSize, pin wiring
Review follow-ups on #4901:
- When every configured trigger is a dropped fraction clause, keep
constructing the never-firing middleware (trigger=None) instead of
returning None: manual /compact runs with force=True and never consults
trigger clauses, so it must keep working for a profile-less model
rather than reporting 'compaction is disabled'. The warning now says
auto-compaction will not fire while manual compaction remains.
- ContextSize gains a config-load validator: fraction values must be in
(0,1] (a percent-style 80 instead of 0.8 previously produced a threshold
the context could never reach — a silently inert trigger), absolute
values must be positive.
- New un-monkeypatched integration test pins the shipped wiring
(context_window declared -> real factory attaches profile -> fraction
clause survives -> middleware constructs), which the stubbed
middleware-side tests and kwarg-capturing factory-side tests each
stopped short of.
- Docs (summarization.md + config.example.yaml) clarify that the fraction
resolves against the summary/anchor model's context_window
(summarization.model_name when set, else the run model), including the
mismatch caveat for a larger-window summary model.
* fix(summarization): reject non-finite ContextSize values at config load
YAML .nan / .inf pass pydantic's float parsing, and nan <= 0 is False,
so the positivity check alone let them through as dead thresholds
(count >= nan is always False) — the same silent-inert-trigger class the
range validator was added to close. Guard with math.isfinite first,
consistent with the existing non-finite guards on mem0 timeout_seconds
and poll_after_seconds.
* fix(summarization): merge context_window into inferred profile, require whole message counts
- construct the model first, then merge max_input_tokens into the
provider-inferred langchain profile: passing profile= to the
constructor replaced the whole inferred metadata (tool_calling,
structured_output, io capabilities, output limits) with the single
key. An explicitly configured profile is still never clobbered.
- reject non-integral ContextSize values for type=messages at config
load: langchain slices the message list with them, so a float index
raised TypeError mid-compaction.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
