* feat(extension-api): add task identity fields to CompactionEvent (0.2.10)
CompactionEvent carried the transform, its hashes and its counts, but
nothing that names the task, run or thread the compaction happened in.
An observer that wants to relate a compaction to the run that produced it
had to match hashes across sessions, which is unreliable.
Append three optional fields, task_id, run_id and thread_id (str | None,
default None). Existing keyword and positional construction is unchanged,
so already-released extensions keep working. The host fills them in the
next commit.
Bump deerflow-extension-api to 0.2.10 and the harness's exact pin with it.
* fix(extensions): hand compaction observers the live task store and its identity
The summarization middleware already recovers the live task store from
the runtime to observe its summary system call, and the registry
allocates that store precisely because a compaction observer is
registered. notify_context_compacted() then discarded it and handed every
observer a detached store, with a docstring claiming there was no live
task.
- The lead run worker and the subagent executor now seed every task store
they allocate with the task's TaskInfo, not only when a lifecycle
contributor is registered; the lifecycle notification reuses the seeded
object. A subagent without a parent run_id still gets no TaskInfo.
- compact_state() and acompact_state() pass the store from
task_store_from_runtime() to _record_compaction().
- notify_context_compacted() takes an optional task_store. A live store
goes to the observers and fills the event's task_id/run_id/thread_id
from its TaskInfo; without one (manual /compact runs outside any run)
observers still receive a detached store and the ids stay None.
For a background subagent the store keeps its public scope id (the
provider tool_call_id) while TaskInfo.task_id stays the execution id, as
lifecycle contributors already see it; the event follows TaskInfo.
* docs(extensions): document compaction event identity and seeded TaskInfo
The extensions guide and the developer manual still said compaction
observers always receive a detached store because no task is live at the
call site. Describe what the host does now: lead runs and subagents with a
parent run seed every task store with TaskInfo, compaction observers get
the live store, and CompactionEvent's task_id/run_id/thread_id come from
it; only manual /compact falls back to a detached store with None ids.
Adds the 0.2.10 row to the contract version table (PR number pending).
* docs(changelog): note compaction observers receive the live task identity
* docs(extensions): list compaction observers among task-store triggers
* docs: reference #7831 for the compaction event identity change
* docs(extensions): backfill contract history rows 0.2.5 to 0.2.9
The version table jumped from 0.2.4 to 0.2.10 although 0.2.5-0.2.9
shipped. Add those rows, attributing names merged without a bump to
the first release that carried them (#5796 in 0.2.5, #6920 and #5645
in 0.2.8, #7729 in 0.2.10). The page intro now says it documents
0.2.4 plus the 0.2.10 CompactionEvent fields, and points to the
history for the rest.
Refresh the locked lark-oapi from 1.5.5 to 1.7.3 without touching the
declared >=1.4.0 constraint, which already admits the new release.
Deployments syncing from the lockfile move off the 1.6.x line, where a
server-side protocol drift reportedly left long-connection clients
connected but receiving no event pushes. Keeping the constraint as-is
avoids shutting out downstream consumers pinned to older but working
releases (1.5.5's WS event routing, including the callback processor map
used by register_p2_card_action_trigger, is verified intact in source).
- Refresh uv.lock only: lark-oapi 1.5.5 -> 1.7.3, specifier unchanged
- Feishu channel, channel-connections, and thread-binding suites pass
on 1.7.3 (111 passed, 1 skipped); ruff clean
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(models): exclude thinking blocks from Claude cache breakpoints
* fix(models): declare Claude test SDK and clarify caching docs
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(extensions): add host-bound Agent run control
* fix(extensions): preserve internal launches and scope run control
* docs(extensions): use a plugin-local run operation key
* chore(version):update the version to 2.2.0-dev
* chore(doc):Updated the CHANGLOG with the 2.1.0 release version
* chore(doc): Updated the CHANGLOG for the latest changes
* chore(doc): Added the release SOP document
* feat(authz): add plugin resource targets, provenance, action and management enforcement
PR1 of the extension-tool-authorization series
(docs/plans/2026-09-24-extension-tool-authorization-spec.md): plugin
authorization plumbing, no new author-facing declaration.
- `authz/plugin_targets.py` is the single encoder/validator for composite
targets (`namespace/part`), with one validating composer behind three
kind-checked constructors; invalid input raises instead of degrading.
- `authz/plugin_authz.py` is the harness decision layer (`plugin_action`,
`plugin_management`, plus the batch filters the page slice will consume):
disabled is a no-op, an explicit deny raises, and provider errors follow
`fail_closed` like the sandbox gate.
- `POST /api/plugins/{ns}/actions/{name}` decides `plugin_action:invoke`
after the action resolves and before the body is streamed.
- `deerflow_extension_api.auth` publishes `require_plugin_management` /
`arequire_plugin_management` with two `app.state` resolver keys; the
Gateway installs both. Contract version 0.2.3 -> 0.2.4.
- `tools/tool_provenance.py` records host-written plugin provenance for
display (assembly descriptor, Layer-2 context later); no decision reads a
label.
- Plugin providers use a `(config signature, loop key)` cache so a
loop-affine provider is never handed to another loop or to a sync caller.
- RBAC gains `plugin_actions` / `plugin_management` aliases;
`config_version` 47 -> 48 across the three mirrors.
Tests: new plugin target/provenance/action/management suites plus a
blocking-IO anchor whose offloads were reverted to prove red->green.
* fix(authz): decide plugin resources from the request's config snapshot
Review round 1 on PR #5842 (willem-bd).
P1 — the decision layer no longer reloads the configuration. The six
`authz/plugin_authz.py` functions take a required `app_config` snapshot (the
same request-scoped config the Gateway used to resolve the provider), and
`resolve_/aresolve_plugin_authorization` return it as a third element, so
`enabled`/`fail_closed` can never come from a second read that drifted from
the one that produced the provider. A snapshot the caller could not read at
all is *unavailable*, not disabled: it fails closed as
`authz.config_unavailable`. The Gateway's `_plugin_app_config` now separates
an absent `config.yaml` (no gate, today's behavior) from a config that exists
but cannot be read or validated (deny) instead of swallowing both through
`safe_app_config`.
P2 — `_validated_decision` requires `type(decision.allow) is bool` on both
the sync `authorize` and async `aauthorize` paths, so a provider returning
`AuthzDecision(allow="false")` is a malformed verdict that follows
`fail_closed` instead of being read as an allow.
Both fixes are covered by regression tests that were verified red against the
pre-fix behavior (P1: 200 vs 403 and resolver True vs False; P2: 12 cases
across fail_closed/fail_open and sync/async), plus tests pinning that an
absent config and `authorization.enabled: false` keep today's behavior.
* fix(authz): treat a request-time config absence as unavailable
Review round 2 on PR #5842 (willem-bd).
P1 — `_plugin_app_config[_async]` no longer maps a missing `config.yaml` to
`None` ("no gate"). A Gateway cannot have started without a readable config
(`app/gateway/app.py` reads it through this same accessor in `lifespan`), so
absence *during a request* means the configuration this process was running on
became unavailable — a hot reload, an atomic replace, a ConfigMap remount. It is
unavailable, not disabled: the error propagates,
`resolve_/aresolve_plugin_authorization` raise
`_PluginAuthorizationUnavailable(fail_closed=True)`, the action dispatcher
returns 403 and the management resolvers answer `False`. The dead
`(None, None, None)` allow path is gone, the resolvers return a non-optional
`AppConfig` as their third element, and callers key "authorization disabled" on
`provider is None` alone. This matches the rule the decision layer already
documented (`authz.config_unavailable`) and the public docs.
P2 — `_deny_reason_code` is total: a `reasons` that is not iterable degrades the
code to `authz.denied` instead of raising `TypeError` outside the guarded
provider path, which turned a denial into an unhandled error. It is deliberately
not folded into `_validated_decision`: `allow` was a real `bool`, so the verdict
is a valid denial, and treating it as malformed would flip it to an allow under
`fail_closed: false`.
Both fixes are covered by regression tests verified red against the pre-fix
behavior (P1: resolver True vs False, async `PermissionError` not raised, route
200 vs 403; P2: `TypeError: 'NoneType' object is not iterable` at
`plugin_authz.py:113`). The action-route tests in
`tests/test_plugin_contributions.py` now install a readable
`authorization.enabled: false` config instead of relying on a config-less
process, and `docs/full-stack-plugins.md` plus the PR1 decision log record the
new rule.
* fix(authz): apply the running policy when the plugin config is unreadable
Follow-up to f03d2ea9 (review round 2 on PR #5842, willem-bd).
The first attempt read the P1 finding as "absence is never disabled" and failed
every config read closed unconditionally. That is too broad for this repo: a
config-less host is deliberately treated as having no policy to apply
(`_get_route_authorization_config` falls back to a disabled config,
`sandbox_authz.safe_app_config` documents the same no-op for "CI runners and
direct-call tests"), and the rule turned the config-less e2e plugin host
(`extension_test_fixtures.bookmark_plugin_gateway`) into a 403 — the three
`bookmark-plugin.spec.ts` cases lost the save action. Reproduced locally:
`POST /api/plugins/community.bookmarks/actions/save` against that fixture host
returned 403 before, 200 after this change.
The rule is now "has this process loaded a configuration?", not "was the last
read successful?":
- No loaded config (`peek_loaded_app_config()`, new read-only accessor on
`deerflow.config.app_config`) and no `config.yaml`: no policy to apply, so the
gate stays a no-op — unchanged for CI runners, direct-call tests and hosts
that mount only the plugins router.
- A loaded config whose file disappears or cannot be read: *unavailable*, never
*disabled*. The resolvers raise `_PluginAuthorizationUnavailable` carrying the
failure flag of the configuration this process runs on (authorization
disabled there -> no gate; enabled -> that policy's `fail_closed`, `True` by
default), so an enabled Gateway answers 403/False in the window instead of
letting protected plugin actions and management routes through.
- A config file that exists but cannot be read or validated right now: fail
closed when nothing is loaded (round 1's rule), otherwise the loaded policy
decides.
`peek_loaded_app_config()` is preferred over a Gateway-side startup snapshot: it
reflects the last successfully loaded config, so a hot-reloaded policy is what
applies during the window, and there is no lifecycle hook to keep in sync.
Evidence: the three no-config/lost-config branches in
`tests/test_plugin_management_guard.py` and
`tests/test_plugin_action_authorization.py`, round 1's unreadable-config tests
unchanged, `test_peek_loaded_app_config_survives_a_missing_file` in
`tests/test_app_config_reload.py`, and the fixture-host probe above. Docs
(`docs/full-stack-plugins.md`, `reference.mdx` en/zh) and the PR1 decision log
state the final rule; the log records the superseded variant.
* fix(authz): keep a denial whose reasons raise while being read
* fix(authz): intersect plugin batch filters with the candidates
The plugin_page/plugin_management batch helpers returned the provider's
`filter_resources` answer verbatim, so a custom provider could hand the
projections a target that was never asked about ("*", another namespace,
a phantom page). Every other filter call site re-projects from its own
candidate list; do the same here, state the intersection on the provider
contract, and pin it with a regression test that fails on the old
pass-through.
* docs(plugins): map a management denial to 403 in the route example
The contributed-route example called `arequire_plugin_management` and let
the documented `PermissionError` escape. The Gateway installs no
`PermissionError` handler, so a route copied from the example answered 500
on a normal policy denial instead of 403. Both example routes now translate
the denial, and the surrounding bullets say why the translation is the
route's job.
* docs(extensions): point the 0.2.4 contract row at the PR that ships it
Every other row in the version-history table links the pull request that
shipped the version; 0.2.4 linked the authorization tracking issue instead.
Both the English and Chinese tables now link this PR.
* chore(checkpoint): bump to checkpoint 4.2.0 and drop the stale saver patch
langgraph-checkpoint 4.2.0 fixes the write dropped after a full -> delta
migration (upstream langchain-ai/langgraph#8526) and keeps its own
InMemorySaver override, so the compatibility patch in checkpoint_patches.py
could neither detect the fix nor stand down: it read langgraph's version
while InMemorySaver ships in langgraph-checkpoint, and it would have kept
shadowing the upstream implementation on 4.2.0. The postgres release 3.1.2
locates plain-value delta seeds (upstream #8535).
- floor langgraph-checkpoint>=4.2.0,<5.0 in the harness and raise the postgres
extra to >=3.1.2,<3.2; langgraph's own >=4.1.0 would otherwise let a
resolver reintroduce the bug now that the patch is gone
- delete the patch, its version guard, and the three implementation-assertion
tests; test_full_to_delta_migration_replays_on_same_thread stays as the gate
and fails on 4.1.1 without the patch (verified by a one-off unpatch run)
- langgraph stays 1.2.9 and langgraph-checkpoint-sqlite stays 3.1.1
Verification: make lint; delta checkpointer/cache/worker-resume suites green
with the migration gate passing on memory and sqlite (postgres params skip
without TEST_POSTGRES_URI). The full make test run before the packaging-pin
fix below was 13 failed / 18523 passed; 12 of those failures reproduce on main
with the same environment (git stash baseline, identical failure sets), 7 of
them because the local config.yaml runs checkpoint_channel_mode: delta. The
remaining one, the postgres-extra string pin in tests/test_checkpointer.py, is
updated here.
Upstream status for the surrounding gaps: cache parity across off/cold/warm
states, postgres pagination (langchain-ai/langgraph#8448) and parallel-superstep
replay order (#8382) are still open upstream, as is the abandoned-branch fork
fix (#8548) that the delta resume linearization in runtime/runs/worker.py works
around, so that path stays.
* docs(changelog): keep checkpoint upgrade in Unreleased
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(extensions): expose incremental run evidence reader
* fix(extensions): address run evidence review feedback
* docs(extensions): clarify run deletion reconciliation
* docs(migrations): align current head documentation
* fix(extensions): isolate run evidence event reads
* test: avoid pinning run change migration to latest head
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): detect the ollama extra from configured models
make dev synced without --extra ollama, uninstalling langchain-ollama from a working setup. Same failure #2754 hit with postgres; ollama predates the detector added in #2767 and never got a rule.
* fix(deps): declare the ollama extra on backend, pin model use: matching
Review follow-ups.
`ollama` was the only extra in the detector's map that the root `backend`
project did not declare, so any consumer syncing without
`--all-packages` failed outright:
$ cd backend && uv sync --locked --extra ollama
error: Extra `ollama` is not defined in the project's
`optional-dependencies` table
`serve.sh` and `docker/dev-entrypoint.sh` both pass `--all-packages` and
were unaffected, but `backend/Dockerfile` does not, and
`config.example.yaml` documents `UV_EXTRAS` as an image build-arg — so
`UV_EXTRAS=ollama docker compose build` would have hard-failed on a
value this branch makes first-class. Declaring
`ollama = ["deerflow-harness[ollama]"]` alongside the other delegating
extras closes that, and `uv.lock` is regenerated to match.
The `use:` match also accepted any nesting depth inside `models:`, so a
`use` in a sub-mapping was read as the model's provider:
models:
- name: doubao
use: deerflow.models.patched_deepseek:PatchedChatDeepSeek
when_thinking_enabled:
use: langchain_ollama:ChatOllama
That yielded `--extra ollama` despite the model's own provider pointing
elsewhere, and `when_thinking_enabled` appears fifteen times in
config.example.yaml, so the shape is common rather than contrived. Pin
matching to the list item's own key indent, mirroring how
`section_value()` pins `child_indent` and documents deeper nesting as
ignored on purpose. Covered by a regression test that fails on the
looser parser.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore(deps): drop unrelated standard-aifc marker churn from uv.lock
Review follow-up. Regenerating the lock with a newer local uv (0.11.19)
also rewrote the `standard-aifc` entry, adding
`python_full_version >= '3.13'` markers to its `audioop-lts` and
`standard-chunk` dependencies. Unrelated to this change, so restore
upstream's lines and keep the lock diff to the `ollama` extra.
`uv lock --check` passes with these lines under both the CI- and
Dockerfile-pinned uv 0.11.1 and uv 0.11.19.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(scripts): detect ollama in unindented model lists and after nested sequences
Review follow-ups on the models parser.
`yaml.safe_dump` — used by the setup wizard (`make setup`) and
scripts/config-upgrade.sh — writes list items unindented:
models:
- name: qwen3-local
use: langchain_ollama:ChatOllama
The parser treated any column-0 line as the end of the `models:` section,
so the first model ended it and nothing was detected. That is the layout
new users get from the recommended setup path, so `make dev` still synced
without `--extra ollama`. Model entries are now recognised before the
section-end test, the same ordering `tools_include_name()` already uses
for the unindented tools list (#4367).
Separately, every sequence item reset the key indent, including items
inside a model option. With `stop:` / `- END` before `use:`, the key indent
jumped to the nested item's and the model's own `use` was skipped, so
detection depended on key order within the model.
The first sequence item under `models:` now fixes the model-list indent;
only items at that indent start a model and set where its keys sit.
Nested sequence items and deeper mappings are skipped without moving it.
Regression tests cover the real setup-wizard output via
`build_minimal_config()`, a hand-written unindented list, and `use:`
after a nested `stop:` list — all three fail on the previous parser — plus
a guard that a `- use:` nested inside a model option is still ignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(web-fetch): resolve relative URLs in extracted Markdown
Pass the request URL through Jina and Browserless extraction and resolve
link/image destinations before Readability removes document base tags.
Preserve the optional legacy API and fallback text behavior.
Fixes#5307
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
* fix(web-fetch): address provider and base URL review feedback
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
* fix(web-fetch): preserve HTML source when resolving destinations
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
---------
Signed-off-by: tiammomo <26957354+tiammomo@users.noreply.github.com>
* fix(agents): make the injected current-date timezone configurable
## Why
The date reminder injected into the lead and subagent prompts (DynamicContextMiddleware / SubagentDateContextMiddleware) was formatted with the server's local wall clock. DeerFlow containers default to UTC, so a user in Asia/Shanghai chatting in the 00:00-08:00 window was told that 'today' is the previous day - the model then reasons, plans, and date-stamps against the wrong day.
## What changed
- _format_current_date() now reads the optional DEER_FLOW_DATE_TIMEZONE env var (IANA name, e.g. Asia/Shanghai) and renders the date in that zone.
- Unset = unchanged server-local behavior; invalid names log a warning and fall back to server-local.
- Documented the knob in config.example.yaml, the module docstring, and the DynamicContext entry in agents/middlewares/AGENTS.md.
## Surface area
- [x] Agents / LangGraph - prompt-layer date context only; message shape and midnight-update behavior unchanged
- [ ] Frontend UI / Backend API / Sandbox / Skills / Dependencies
- [x] Default behavior change (opt-in via env var - no behavior change unless set)
## Bug fix verification
- New tests: test_format_current_date_honors_configured_timezone (UTC 20:30 -> 2026-09-03 in Asia/Shanghai), test_format_current_date_defaults_to_server_local_without_env, test_format_current_date_invalid_timezone_falls_back.
- Existing mocked-datetime tests pass unchanged (no env -> datetime.now() path).
## Validation
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py: 31 passed.
- blocking_io/test_dynamic_context_middleware.py: 2 pre-existing abefore_agent failures reproduce identically on clean main (blockbuster os.listdir detection on this host); the other 2 pass.
- ruff format + ruff check clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): avoid passing tz to datetime.now when no timezone is configured
CI (backend-unit-tests shard 2) failed in test_tool_error_handling_middleware.py::test_subagent_chain_injects_date_without_memory_and_coalesces_for_strict_provider because its _FrozenDateTime.now() subclass override accepts no arguments, while _format_current_date() called datetime.now(None) even when DEER_FLOW_DATE_TIMEZONE was unset.
- _format_current_date() now calls datetime.now() with no arguments unless a timezone is actually configured, preserving the exact legacy call shape for every datetime-subclass test fake.
- The configured-zone path still calls datetime.now(tz) and converts via astimezone(tz).
- Updated the no-env unit test to assert datetime.now() is called without arguments.
Validation: python -m pytest tests/test_dynamic_context_middleware.py + the previously failing strict-provider test: 32 passed. ruff clean.
* fix(agents): declare the effective current-date timezone in the assembly descriptor
## Why
Maintainer review on the DEER_FLOW_DATE_TIMEZONE change (#5154): the knob is
prompt-affecting, yet both DynamicContextMiddleware and SubagentDateContextMiddleware
were invisible to the agent assembly descriptor - describe_middleware() fell back to
{"probed": true} for unset, UTC, and Asia/Shanghai alike, so deployments that inject
different dates shared one assembly fingerprint and release observers could not
distinguish or audit the behavior change.
## What changed
- Both middlewares now implement release_policy_parameters() -> dict[str, object],
declaring {"current_date_timezone": <name>} as required by the module's middleware
self-description contract.
- The declared value is the normalized effective zone: a configured, valid
DEER_FLOW_DATE_TIMEZONE is reported by its IANA key (ZoneInfo.key); otherwise the
server-local zone is resolved to its IANA key when the platform exposes one and to
its tzname label otherwise (fixed-offset hosts), with "UTC" as the final fallback.
- Added both middlewares to _MIDDLEWARE_DECLARATIONS in
backend/tests/test_middleware_release_policy.py so the existence check and the
construct-and-canonical-hash check cover them.
## Verification
- New tests: test_date_middlewares_declare_configured_timezone (Asia/Shanghai),
test_date_middlewares_declare_utc_timezone, plus resolved-server-local assertions
for the unset and invalid-env paths; both middlewares agree in every case.
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py: 70 passed.
- Regression spot-check: tests/test_agent_assembly_descriptor.py,
tests/test_tool_error_handling_middleware.py, tests/test_system_message_coalescing_middleware.py:
102 passed.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): stabilize the declared date timezone and simplify the formatting path
## Why
Follow-up review on #5154 (willem-bd). The release-policy declaration added in
884cec4b resolved the observability gap but pinned far less identity than its
docstrings claimed, and the formatting path carried a production no-op.
## What changed
- The declared label is now stable and unambiguous: a configured, valid
DEER_FLOW_DATE_TIMEZONE is reported by its IANA key; without one, the
server-local zone is resolved to a real IANA key from the TZ env var or the
/etc/localtime symlink (Linux/macOS); when no key is recoverable (Windows,
stripped containers) the declaration falls back to a stable
`server-local(+-HH:MM)` sentinel carrying the current UTC offset. It never
reports a bare abbreviation - datetime.now().astimezone() yields only a
fixed-offset timezone whose tzname (e.g. CST, EST/EDT, CET/CEST) is
ambiguous or DST-churns, which the assembly descriptor docstring says must
not happen.
- Dropped the redundant astimezone(tz) in _format_current_date():
datetime.now(tz) already returns the instant expressed in tz. The
configured-zone test now fakes datetime.now(tz) semantics (the fixed instant
converted into the requested zone) instead of relying on that conversion.
- Documented why the knob is an env var, not a config-schema field: it is read
at runtime by both date-context middlewares so an operator can point a
container at another zone without mounting a config.yaml (module docstring +
config.example.yaml note).
- AGENTS.md: fixed the glued DynamicContext sentence (missing separator).
- Added tzdata>=2025.1 to the harness runtime dependencies (with uv.lock) so
ZoneInfo works on stripped containers / Windows without an OS zone database.
## Verification
- New tests: test_server_local_timezone_name_reads_tz_env,
test_effective_timezone_sentinel_uses_offset_when_local_zone_is_not_resolvable;
reworked test_format_current_date_honors_configured_timezone to exercise the
real datetime.now(tz) path.
- cd backend && python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_agent_assembly_descriptor.py
tests/test_tool_error_handling_middleware.py: 140 passed.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): offload subagent date injection off the event loop
## Why
Follow-up review on #5154 (willem-bd, P2): SubagentDateContextMiddleware.abefore_agent()
called _inject() directly, so enabling DEER_FLOW_DATE_TIMEZONE could synchronously
read the OS timezone database (or the tzdata wheel) on a cold cache - filesystem
work on the async subagent execution path whenever no assembly observer resolved the
zone first.
## What changed
- SubagentDateContextMiddleware.abefore_agent() now offloads the injection via
asyncio.to_thread with the same bounded timeout DynamicContextMiddleware uses
(issue #3402); on timeout it logs and skips the date update for that run instead
of blocking the loop.
- Narrowed the exception handling in _date_timezone() and the TZ-env branch of
_server_local_timezone_name() to configuration-shaped failures
(ZoneInfoNotFoundError / ValueError / OSError). Previously a blanket
`except Exception` also swallowed BlockingError raised by the blocking-I/O
regression gate, mislabeling a loop-blocking call as an invalid timezone and
silently degrading to server-local - which made the new regression anchor
useless. Other exceptions now propagate.
## Verification
- New blocking-I/O regression anchor
(backend/tests/blocking_io/test_subagent_date_context_middleware.py): drives a
real create_agent graph under the strict Blockbuster gate with the knob enabled
and asserts the date reminder is injected. Verified it fails (BlockingError) when
the offload is reverted and passes with it in place.
- python -m pytest tests/blocking_io/test_subagent_date_context_middleware.py:
1 passed. The two pre-existing os.listdir failures in
tests/blocking_io/test_dynamic_context_middleware.py reproduce unchanged on this
host (same as clean main).
- python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_tool_error_handling_middleware.py
tests/test_agent_assembly_descriptor.py: 139 passed; the single
ToolReceiptMiddleware-ordering failure reproduces with the change stashed
(local extensions registry, unrelated to this PR).
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
* fix(agents): read the direct /etc/localtime symlink target for the zone key
## Why
Follow-up review on #5154 (willem-bd, P2): on macOS, /etc/localtime commonly
points to /var/db/timezone/zoneinfo/<zone>, but Path.resolve() follows that
directory's own symlink and yields a versioned path such as
/private/var/db/timezone/tz/2026c.1.0/zoneinfo/Asia/Shanghai, which matched no
configured prefix. The server-local resolution then returned None and the
assembly descriptor fell back to a server-local(+HH:MM) sentinel even though
the IANA key was available - conflating zones that share an offset and making
DST-based fingerprints unstable.
## What changed
- _server_local_timezone_name() now reads the direct symlink target via
os.readlink("/etc/localtime") instead of Path.resolve(), so macOS' unversioned
zoneinfo path is seen as-is and its IANA key is preserved.
- The zone key is taken from whatever follows the last "/zoneinfo/" segment,
which also handles Apple's canonical versioned path when a direct target
already carries it, and relative targets are normalized against /etc.
- Removed the now-unused Path import and the fixed zoneinfo prefix tuple.
## Verification
- New tests: test_server_local_timezone_name_reads_direct_macos_symlink_target,
test_server_local_timezone_name_reads_apple_versioned_symlink_target, and
test_server_local_timezone_name_normalizes_relative_symlink_target.
- python -m pytest tests/test_dynamic_context_middleware.py
tests/test_middleware_release_policy.py tests/test_agent_assembly_descriptor.py:
105 passed (75 after re-running the first two on the merged main). The blocking
subagent anchor still passes; the two pre-existing os.listdir blocking failures
on this host are unchanged.
- ruff check + ruff format clean.
## AI assistance
**Tool(s) used:** Codex (coding agent)
**How you used it:** analysis, implementation, and regression tests produced with AI assistance; reviewed before commit.
- [ ] I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(ci): split backend unit tests into parallel CI shards
Split the single offline backend `make test` job into four GitHub Actions
matrix shards (SPLITS=4, GROUP=1..4) via pytest-split, so the ~12k-test suite
runs in parallel instead of in one 15-minute job. Each shard runs on its own
runner with its own Postgres/Redis services; fail-fast: false lets a failing
shard report its owned tests without cancelling its peers.
`make test` stays the canonical full-suite entry point; CI now calls the new
`make test-shard SPLITS=4 GROUP=N`. tests/blocking_io remains owned solely by
the dedicated blocking-I/O workflow (excluded via --ignore), extending #5105.
Fixes#5088
* test(ci): make backend test shards duration-aware and pin the contract
Make `make test-shard` an explicit least_duration split that READS
backend/.test_durations (read-only for shards, so concurrent CI jobs never
race writes on it), and add `make test-shard-durations` to regenerate that
file from the full offline suite. Update the CI unit-test workflow contract to
call `make test-shard SPLITS=4 GROUP=<n>` and assert the shard command carries
--splits 4, --group 2, -m "not live", --ignore=tests/blocking_io and
--splitting-algorithm least_duration. Verified on the real 13,140-test normal
suite that the four shards are pairwise disjoint and their union equals the
unsplit suite.
Refs #5088
* test(ci): fail fast when the duration baseline is missing
`make test-shard` now requires backend/.test_durations and exits with a clear
error instead of letting pytest-split silently degrade to an even (count-based)
split. Harden the CI contract test to pin `--durations-path=.test_durations` and
to assert the repo ships the committed duration baseline.
Refs #5088
* docs: trim backend/AGENTS.md within guidance budget
* test(ci): add backend test duration baseline
Add the duration baseline generated by a full offline backend run on a
GitHub-hosted ubuntu-latest runner (the same runner type the shards use), so
`make test-shard` balances the four matrix shards by real wall-clock cost.
Refs #5088
* test(ci): make the duration writer honor DURATIONS_FILE
`test-shard-durations` now writes `--durations-path=$(DURATIONS_FILE)` instead of a
hard-coded .test_durations, so the reader and writer stay consistent when the
path is overridden.
Refs #5088
* test(ci): address sharding review feedback
* test: isolate subagent execution capacity state
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(deps): depend on renamed tenki package instead of tenki-sandbox
tenki-sandbox has been removed from PyPI and republished as tenki. Its old wheel URL still resolves, so existing lockfiles keep installing and the breakage is invisible to anyone with a warm lock; any fresh resolution fails with 'tenki-sandbox was not found in the package registry'.
tenki 1.0.2 still ships the tenki_sandbox module, so the imports in community/tenki/provider.py and sandbox.py are unchanged.
Fixes#5081
* fix(tenki): point install guidance at the renamed distribution
The rename to `tenki` left the user-facing remediation still naming the
removed package. `_import_client` raised "pip install tenki-sandbox" on the
missing-extra path — the exact instruction this change proves now 404s on
PyPI, handed to the user at the exact moment they need it to work.
Update that message and the remaining `tenki-sandbox` references in the
provider, sandbox adapter, README, sandbox AGENTS.md and the test docstring.
The imported module stays `tenki_sandbox`, so the distribution and module
names now differ; each mention says so rather than just swapping the string.
No behavior change beyond the error text.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tenki): migrate the provider to the 1.x workspace-only API
Renaming the dependency was not enough. tenki 1.0.2 keeps the tenki_sandbox
module name but not its contract: Client.create dropped project_id and has no
**kwargs to absorb it, and IdentityWorkspace no longer carries `projects`
(the attribute is gone from the package entirely). Both configuration paths
therefore failed before a sandbox could be created — explicit project scope
raised TypeError, and automatic scope raised AttributeError walking
workspace.projects.
Scope is now the workspace alone. _resolve_scope returns a single workspace id,
auto-selecting when the account has exactly one, and project_id is gone from
create_kwargs and from the documented config surface.
A stale project_id in config.yaml warns rather than fails. SandboxConfig is
extra="allow", so simply not reading the key would leave it scoping nothing
with no signal; it also used to short-circuit the identity lookup, so operators
with more than one workspace need to know they must now set workspace_id.
The suite passed against the broken provider because the fake client took
**kwargs and swallowed the project_id the real SDK rejects. The double now
mirrors 1.0.2 — keyword-only, no **kwargs — so an unexpected argument is a
TypeError in tests exactly as it is against the SDK. Reintroducing the old
create call fails 20 tests; before this change it failed none.
Verified against the exact locked wheels: every other kwarg the provider
passes (name, workspace_id, sticky, wait, max_duration, image, cpu_cores,
memory_mb, env) and every SDK surface it touches (who_am_i, Identity.workspaces,
wait_ready, exec, close, the fs API, the four terminal exception classes) is
unchanged in 1.0.2.
Reported by willem-bd in review.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(config): drop sandbox.project_id from the Tenki example
The canonical example still documented project_id as a supported optional key
after the provider stopped honouring it, so an operator following it could set
the key, get no scope from it, and hit a workspace-resolution failure with
nothing in the example to explain why.
Replaced with a migration note rather than a silent deletion: someone upgrading
already has the key in their config.yaml and needs to know it is inert now and
that workspace_id is what scopes a sandbox on Tenki 1.x.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Aniket Wagh <aniketwaghh@users.noreply.github.com>
* feat(extensions): let an out-of-tree extension observe what the agent did
DeerFlow's extension system can contribute middleware, services and routes,
but an extension cannot answer basic questions about a run without reaching
into host internals. Several of the facts it would need are destroyed by the
operations that produce them:
* The middleware chain injects and rewrites a lot of context — date
reminders, recalled memory, compaction summaries, durable-context data,
image payloads, activated skill bodies. Downstream, none of it is
attributable: at the model-call boundary an injected HumanMessage is
indistinguishable from the user's own, and anything wanting to tell them
apart has to pattern-match prompt wording, which breaks on the next copy
edit.
* Two runs of "the same agent" are only comparable if the chain enforced the
same limits, prompts and thresholds. Recovering that from outside means
reading private attributes and guessing which of them change behaviour — a
guess that rots silently as middlewares gain fields.
* The lead-agent factory resolves a model after runtime overrides, renders a
prompt, filters tools through authorization and composes a stack, all
inside one synchronous call, and none of it survives: a middleware sees its
neighbours but not the prompt, the run worker sees a graph but not what
went into it.
* Summarization is destructive by design. N messages leave the context and
one summary enters it; afterwards only the summary exists, so "which
messages became this?" is not reconstructible.
This adds seven neutral facilities so those facts are recorded where they are
still true, and releases the contract package as 0.2.0.
Message provenance
Producers stamp `deerflow_content_kind` / `deerflow_producer_kind` onto the
messages they inject or rewrite. Stamping is unconditional — a fact whose
presence depends on whether an observer is installed is not a fact — and the
keys are server-owned, so provenance cannot be forged from a request.
Middleware self-description
Twelve middlewares declare their own behaviour-affecting parameters through
a duck-typed `release_policy_parameters()`. Long text is hashed rather than
embedded: a declaration is an identity, not a copy of the prompt.
Agent assembly descriptor
`assemble_lead_agent()` returns the graph plus a descriptor whose fingerprint
answers "did anything about this agent change between these two runs?".
`make_lead_agent()` keeps its graph-only signature — it is the LangGraph
Server ABI declared in langgraph.json. Tools and skills are sorted before
hashing because their assembly order is incidental; middlewares are not,
because stack order decides what wraps what. Host build identity is reported
but excluded from the fingerprint, so a redeploy does not invalidate every
agent's identity.
Context compaction observation
Summarization emits the content hashes of the messages it is about to remove
joined to the summary that replaced them. Content is the only identity
available at that seam: the summary does not become a message, and what later
projects it into a request renders it bounded and escaped rather than
verbatim.
Neutral policy, transform and MCP-source facts
Guardrail decisions are published to runtime context under a `__`-prefixed
key; result-rewriting middlewares append a declared, ordered transform trail;
MCP tools carry their credential-free logical origin.
Extension route identity
Contributed routes are session-authenticated and cannot opt out, but
"logged in" and "administrator" are different questions. Extensions get a
neutral projection of the caller rather than the host's auth context, and
`require_admin` fails closed when identity cannot be determined.
Extension-owned tables
An extension that persists data owns its own MetaData and migration chain, so
its tables are absent from Base.metadata and `alembic revision --autogenerate`
proposes dropping them. Extensions declare a table prefix, which is rejected
at registration if it would shadow a host table.
The contract package stays dependency-free and imports no host code; every new
Protocol method has a default so later additions remain additive. The loader's
pre-1.0 rule requires an exact major.minor match, so extensions written against
0.1 are now refused at startup with an actionable install hint rather than
loading into a host that implements a different surface.
uv.lock records the contract package's new version, so `uv sync --locked` still
resolves on a fresh checkout.
* fix(backend): sort gateway service imports
