* feat(memory): honcho backend config parsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(memory): honcho v3 http client
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(memory): honcho memory manager (workspace-per-user, fail-closed identity, async offload)
- HonchoMemoryManager implements the MemoryManager contract (add/get_context/
search/get_memory/shutdown_flush + aadd/aget_context/asearch offloaded via
asyncio.to_thread), signatures verified against manager.py's tier-1/tier-2/
async abstracts.
- Workspace resolution: workspace_overrides[user_id] else
workspace_prefix + sanitize_id(user_id); missing/empty user_id fails closed
(no-op write, empty read) rather than falling back to a shared workspace.
User peer: user_peer_overrides[user_id] else sanitize_id(user_id).
- get_context self-truncates to max_injection_chars and raises
MemoryManagerError only under failure_policy.read=fail_closed; default is
log-and-return "".
- Restore backends/honcho/__init__.py to the noop direct-import convention
(MANAGER_CLASS = HonchoMemoryManager) now that honcho_manager.py exists,
replacing Task 10's temporary lazy __getattr__ scaffold.
- Fix Task 10 deferred docstring minor: sanitize_id docstring now states the
grammar allows up to 100 chars while this helper caps at 64.
- 19 new tests appended to test_honcho_memory_backend.py (write/read/async/
lifecycle/factory-discovery); 27/27 pass. Verified end-to-end that
manager.py's drop-in backend scanner resolves "honcho" with no core edits.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(memory): collision-resistant identity derivation, exception containment, passive-writes flag
Task review findings (2 Critical + 1 Important), all fixed in the same worktree:
- CRITICAL (cross-user bleed): sanitize_id is lossy -- "user.name@example.com"
and "user-name@example.com" both sanitized to the same string, merging two
users' memory into one workspace/peer. Add _stable_id() (sanitize_id output
+ 8-hex-char SHA-256 suffix of the raw id) and use it on the default
(non-override) path in _workspace/_user_peer; workspace_overrides /
user_peer_overrides still match on the raw key, unchanged. The hash suffix
also guarantees a non-empty result for a raw id that sanitizes to "" (e.g.
"!!!"), so _user_peer can no longer return "". Documented in the manager's
isolation docstring.
- CRITICAL (exception containment): client.py's _post() called response.json()
outside the try block, so a 200 with a non-JSON body raised a bare
JSONDecodeError that would escape add() with no upstream handler. Wrap the
parse and raise HonchoRequestError (mirrors Mem0Client._request). Broadened
the manager's four boundary excepts from `except HonchoRequestError` to
`except Exception` (mirrors openviking_manager.py's broad-guard precedent),
with `except MemoryManagerError: raise` first so a contract error is never
swallowed or double-wrapped.
- IMPORTANT: added requires_passive_writes_in_tool_mode: ClassVar[bool] = True
-- Honcho's only write path is passive add() (no fact CRUD hooks), so tool
mode must keep MemoryMiddleware writes flowing to the deriver. Mirrors
mem0_manager.py's identical flag/rationale.
Minors addressed: get_memory(user_id=None) empty-shape-with-no-calls test;
empty-string user_id tests for add()/get_context(); dedicated collision test
proving two colliding raw ids resolve to different workspaces/peers.
10 new tests (37/37 total pass); RED verified by stashing only the
implementation files (tests import the not-yet-existing _stable_id, so the
whole module fails to collect) before restoring the fix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(memory): blocking-io anchor for honcho backend; docs + config example
- Adds test_honcho_memory_backend.py in tests/blocking_io/ with fake-client blocking IO
- Mirrors openviking anchor structure and conftest conventions
- Updates backends/README.md with honcho row and config keys section
- Updates config.example.yaml with honcho commented block
- Updates backend/AGENTS.md with honcho memory backend bullet
- Documents workspace resolution (prefix + collision-resistant sanitized id)
- Documents tool mode passive write retention via MemoryMiddleware
- Documents async entrypoint offloading via asyncio.to_thread
- Documents fail_closed vs fail_open recall failure policy
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(memory): wire close() to shutdown hook; correct honcho README defaults and tool-mode note
- HonchoMemoryManager.close() releases the HTTP client, mirroring
mem0_manager.py's pattern and the base MemoryManager.close() shutdown hook.
- README: fix workspace_prefix (deerflow-u-), message_char_limit (8000),
max_injection_chars (6000), and base_url (default http://localhost:8000,
not required) against backends/honcho/config.py; add missing
timeout_seconds/connect_timeout_seconds rows; replace the "middleware
mode only" claim with wording matching reality (tool mode supported,
search implemented, passive writes retained via MemoryMiddleware).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(memory): honor failure_policy.read on all honcho recall paths; review nits
Addresses PR #4730 review feedback:
- search() and get_memory() now route through a _read_or_fallback policy
gate (mem0's pattern), so failure_policy.read: fail_closed raises
MemoryManagerError on every recall path as documented; get_context()
uses the same helper, preventing future drift.
- Session ids use the collision-resistant _stable_id derivation; bare
sanitize_id would merge threads like "t.1"/"t-1" into one session.
- HonchoClient accepts a transport kwarg (Mem0Client precedent) so tests
inject httpx.MockTransport through the constructor.
- Config: empty/null workspace/peer override values fail fast at parse
time instead of silently falling through to the default derivation.
- _UTC_NOW_FIELDS 1-tuple replaced by a plain _UTC_NOW_FORMAT constant.
- README: user_peer_overrides row described the wrong target (it
overrides the user's own peer, not assistant_peer); document the
non-empty constraint on override values.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(memory): qualify honcho isolation claim for shared workspace_overrides
The module docstring claimed users cannot see each other's memory by
construction, unconditionally. That holds for the default
one-workspace-per-user derivation, but a workspace_overrides entry mapping
several users to one workspace shares that workspace's search index:
search() uses Honcho's workspace-scoped /search (no peer filter), while
get_context()/get_memory() stay peer-scoped via working_representation.
State the asymmetry in the docstring, the README Workspace Resolution
section, and the workspace_overrides table row.
Docs-only; no behavior change (review follow-up on #4730).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test(llm-error): rename a stand-in, not the shared FakeError
`exc.__class__.__name__ = "ReadError"` on a `FakeError` instance renames the
class itself, so `FakeError` stays named "ReadError" for the rest of the
session and every later test asserting error_type == "FakeError" fails.
Declaration order hides it: the renaming test runs after its victims.
Use the existing _ReadError stand-in, which is already named "ReadError" and
is how the sibling _max_attempts_for test builds the same case. Add an autouse
fixture so a future slip fails the test that causes it.
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* style: format FakeError guard
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* test(tool-output): stop expressing "unwritable path" as a magic absolute path
`test_returns_none_on_invalid_path` and `test_fallback_when_disk_write_fails`
both need an `outputs_path` that `os.makedirs` refuses to create, so they can
reach `_externalize`'s `except OSError: return None` branch. They spell that as
the literal path `/dev/null/cannot-mkdir-here`, which only works where
`/dev/null` is a character device.
On Windows it is an ordinary relative path, so `os.makedirs` succeeds, both
tests fail, and the suite writes real files to `C:\dev\null\cannot-mkdir-here\
.tool-results\` -- outside any temporary directory, at the drive root. Running
the backend suite a few times leaves dozens of stray files behind.
The comment above the first test records that this is the second time the
same assumption has broken: `/nonexistent/...` was silently created by `mkdir
-p` when CI ran as root in a container, and `/dev/null/...` was the fix. Both
encode a guess about the environment rather than the condition under test.
Use a regular file as the parent component instead. Creating a directory below
a file fails with an `OSError` subclass on every platform -- `NotADirectoryError`
(errno 20) on POSIX, `FileNotFoundError` (errno 2) on Windows -- so the branch
is reached deterministically, and the path lives inside the test's own
`TemporaryDirectory`, so nothing is written outside it.
Verified both spellings on Linux (WSL Ubuntu, non-root) and Windows; only the
file-as-parent form fails on both. The two tests still have teeth: dropping
`_externalize`'s `except OSError` guard makes both fail rather than pass.
Tests only -- no production code or documented behaviour changes.
* test(tool-output): touch the blocker file instead of writing content
Only its existence as a regular file matters for os.makedirs to fail
below it, so touch() states that directly.
_enrich_thread_message_page inlined its own turn_duration loop that
stamped EVERY AI message of a run, re-introducing #4152 on the
/messages/page endpoint (the legacy /messages and /history endpoints
already route through stamp_turn_duration_on_last_ai after #4163, but
that fix missed the page path introduced earlier in #4065). A
multi-step turn thus rendered the same run lifetime beside every
intermediate AI message, reading as repeated thinking latency.
Replace the inline loop with the shared stamp_turn_duration_on_last_ai
helper so all three message endpoints agree: the run's wall-clock
duration lands on its final visible AI message only.
Admission derived the live-entry count as `HLEN - 3`, where 3 was the number
of `meta:*` fields written 35 lines earlier in initialize(). Nothing tied the
two together, so adding a fourth meta field would shift the capacity ceiling
by one.
Name the offset `META_FIELD_COUNT` next to initialize(), and add a guard test
asserting a freshly initialized ledger holds exactly those three fields, plus
one pinning that a hard_limit of N admits exactly N reservations.
References #4575
Co-authored-by: icn5381 <255778606+icn5381@users.noreply.github.com>
* feat(extensions): observe task lifecycle and system model calls
PR 1 (#4636) gave extensions a middleware chain, and a middleware only sees
what passes through the agent graph. Two runtime surfaces stay invisible to
it: when a lead run or a subagent begins and ends, and the DeerFlow-owned
model calls made outside the graph. This slice adds both, with no new
Gateway surface -- routers, services, and the reference extension stay in
PR 3.
Contract (deerflow-extension-api 0.1.1)
---------------------------------------
Two contribution kinds join `middlewares` on the registry:
`task_lifecycle` (`on_task_start` / `on_task_stop`, receiving a `TaskInfo`
and a conservative `TaskOutcome` of completed / aborted / failed) and
`system_model_observer` (`on_system_model_call`, receiving a
`SystemOperationKind`, a `SystemModelRequest` snapshot, and a
`SystemModelResult` carrying either the response or the provider exception
plus a duration).
`SystemModelRequest.messages` normalizes to a tuple at construction. Goal
evaluation and memory extraction pass a message list while title generation
and summarization pass one prompt string, and a bare `str` already satisfies
`Sequence` -- without normalization an observer iterating `request.messages`
would silently walk characters. Copying also makes the frozen snapshot
immutable in fact rather than only by declaration, since observations may run
after the call site returns and keeps mutating its own list.
Registry marks and rollbacks become per-bucket and positional, so an
`install()` that fails after registering two different kinds cannot leave one
of them behind. `needs_task_store` now covers all three kinds: a deployment
that registers only lifecycle hooks still gets a task store.
Task lifecycle
--------------
The lead worker notifies start after the run has started and stop after
completion persistence and the completion hook, but before clearing the
finalizing barrier and publishing the stream end -- holding the barrier
across stop is what keeps a same-thread replacement run from overlapping this
task's lifecycle. Cancellation raised out of the stop notification is
deferred, not propagated in place, so a cancelled run still clears the
barrier and emits its end frame. A subagent with a parent `run_id` wraps its
execution in the same pair inside `finally`, reporting `parent_task_id` so a
delegation tree is reconstructable; a subagent without a `run_id` (embedded
client, standalone LangGraph Server) logs and skips rather than inventing a
parent. Contributors run in registration order inside one shared 3s budget
and every failure is logged and failed open.
System model calls
------------------
Four kinds cover the model calls the middleware chain cannot see: goal
evaluation, memory extraction, title generation, and summarization. Each site
reports both terminal paths without changing the provider exception the host
observes, short-circuits on `has_system_model_observers`, and passes the live
task store when the runtime has one (detached work gets an isolated store).
The sync summarization half stays unobserved on purpose -- it and its only
host caller are the sync side of an async-only runtime, so notifying there
would block a thread on a call site the host never reaches; the reason is
recorded at the call site.
The DeerMem backend must stay vendorable and cannot import the extension API,
so it reports through a new `MemoryCallbacks.on_memory_llm_result` host hook
that the DeerFlow-side callbacks translate into an observation.
Notification loop
-----------------
Extension resources must be touched on the loop that created them, but
subagents can execute on isolated loops and DeerMem runs on a worker thread.
The Gateway registers its serving loop before any runtime dependency starts
and resets it last through the exit stack, so every startup-failure and
cancellation path is covered. Awaited hooks raised on another loop are
dispatched across with `run_coroutine_threadsafe` and awaited under the same
budget; synchronous sites submit fire-and-forget work. Shutdown stops
accepting detached observations before the memory flush -- that flush runs on
a worker thread and can emit memory observations -- while keeping the loop
alive for awaited task hooks until run and subagent drain completes.
Tests
-----
`test_extension_task_lifecycle.py`, `test_extension_subagent_lifecycle.py`,
and `test_extension_system_model_calls.py` cover ordering, fail-open, budget
exhaustion, snapshot binding under a concurrent singleton replacement, the
loop-dispatch and shutdown-suspension paths, and both terminal paths at every
call site. `test_gateway_run_drain_shutdown.py` pins the stop-before-barrier
and drain ordering.
* fix(extensions): decide notification fail-open by origin, observe cancellation
`_notify_each` only guarded `Exception`, so a contributor letting a
`CancelledError` escape — an extension implementing an internal timeout with
cancellation, say — skipped its successors and reached the worker's
deferred-interrupt path, ending an otherwise successful run as cancelled.
Fail-open is about where a failure came from, not its base class: only a
genuine cancellation of the host task increments `Task.cancelling()`, so
propagate on that and contain everything else. `KeyboardInterrupt` /
`SystemExit` still propagate.
`observe_system_model_call` skipped observers on cancellation for the same
base-class reason, leaving goal / title / summarization silent on a terminal
path that is routine — interrupt/rollback admission and shutdown both cancel
the run task, with the provider tokens already spent. Awaiting observers there
is unreliable (a repeated cancel interrupts that await before any of them
runs), so report through the same non-blocking submission the synchronous
memory bridge uses, then propagate the cancellation untouched.
DeerMem keeps `BaseException` around its provider call, now with the reason
recorded: that path runs on a worker thread, where cancelling the awaiting
side never interrupts the running thread, so `CancelledError` cannot arrive
at all. Its host-hook wrapper narrows to `Exception` — only the hook's own
failures are non-fatal, and an observability path must not swallow a process
teardown signal.
* fix(extensions): warn on budget exhaustion, scope observer logs by task, propagate teardown
Review response on #4684:
- The memory observation bridge caught BaseException, which would swallow
a teardown signal raised while dispatching; it now catches Exception,
matching the boundary the DeerMem-side call site documents and tests.
- A notification-budget timeout raised mid-hook fell into the generic
hook-failure path and logged an asyncio-internal traceback; it now logs
a warning like the pre-hook budget skip, while a TimeoutError a
contributor raises on its own stays classified as a hook failure.
- System model observer logs passed the operation kind as the task id,
so log lines said "task goal/title/..."; they now carry the task
scope id alongside the kind.
DanglingToolCallMiddleware._message_tool_calls collected the raw
additional_kwargs tool_calls payload whenever structured tool_calls was
empty, even when invalid_tool_calls was non-empty. The raw payload is a
fallback serialization of the SAME calls — the OpenAI serializer reaches
for it only once both structured views are empty, which is exactly the
gating _normalize_tool_call_ids documents and implements. Collecting it
alongside a same-id invalid entry counted the call twice and emitted two
placeholder ToolMessages for one id — the duplicate-id shape strict
OpenAI-compatible providers reject with HTTP 400, the failure this
middleware exists to prevent.
Gate the raw collection on both structured views being empty, aligning
_message_tool_calls with _normalize_tool_call_ids.
The docker/lark-cli-init/README.md referenced
docs/superpowers/specs/2026-07-21-lark-sandbox-init-container-design.md,
but that design spec was never committed to the repository — it is absent
across the full git history. The link has been broken since it was
introduced in #3971. The README already documents the init-container
behavior standalone, so the dangling reference is removed.
Co-authored-by: icn5381 <255778606+icn5381@users.noreply.github.com>
* feat(mcp): add durable task runtime foundation
* fix(chart): sync embedded config version
* fix(mcp): isolate task polls during shutdown
* feat(mcp): track consecutive poll errors on mcp_tasks
poll_attempt_count grows on every claim (successful polls included), so it
cannot drive a failure backoff without misjudging normal long tasks. Add
consecutive_poll_error_count: incremented when a claim is released after a
poll error, reset to zero by any applied snapshot. The backoff/terminal
policy that consumes it lands with the first concrete driver.
* fix(mcp): harden durable task lifecycle
* fix(mcp): preserve tracked task on dedup conflict
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: bound MCP server bring-up timeouts and exclude externalized tool outputs from delivery verification
Two related robustness fixes:
1. MCP server bring-up was unbounded. tool_call_timeout only covered
session.call_tool(); tool discovery (subprocess spawn + initialize +
tools/list) and persistent stdio session initialization could hang
forever, blocking agent construction (and on the Gateway event loop,
the whole process). Add a per-server session_init_timeout
(default DEFAULT_MCP_SESSION_INIT_TIMEOUT = 60s, null disables) that
bounds both discovery and pooled-session initialization. The session
pool's existing cancellation handling tears down a session stuck
mid-creation in its own task.
2. ToolOutputBudgetMiddleware externalizes oversized tool outputs into
outputs/.tool-results/ (configurable tool_output.storage_subdir). The
workspace-change scanner and run delivery verification counted those
files as produced artifacts, so any run that externalized a tool output
without also presenting a real artifact failed with
"Artifact delivery incomplete". Exclude TOOL_RESULTS_DIRNAME via a
shared constant (mirroring BROWSER_FRAMES_DIRNAME) and thread the
configured storage_subdir through snapshot capture so both
workspace-changes events and delivery verification stay clean.
* review: enforce single-segment tool_output.storage_subdir; document discovery-timeout cleanup
Address review feedback:
1. A custom tool_output.storage_subdir with a path separator (e.g.
cache/tool-results) silently no-oped the workspace-scanner exclusion:
os.walk yields one-segment dirnames, so a nested value never matched and
its files were counted as produced artifacts again. ToolOutputConfig now
validates storage_subdir as a single directory name (rejects separators,
.., absolute, empty) with tests, so the exclusion is always sound.
2. The discovery-timeout path now documents why cancellation is safe, mirroring
the session-init note: discovery runs inside the adapter's nested async
context managers, and stdio_client's finally terminates the process tree
(SIGTERM->SIGKILL on POSIX, process-tree on Windows), so a timed-out npx
subprocess and its children are reaped rather than accumulating.
* review: log session-init timeouts and align API response model default with runtime config
Address second-round review feedback:
1. A session-init timeout raised TimeoutError without any log, unlike the
discovery timeout which logs a WARNING. Wrap the bounded get_session in a
try/except that logs the timeout (server name + seconds) and re-raises, so
operators can diagnose tool-call failures caused by hung MCP sessions.
2. McpServerConfigResponse.session_init_timeout defaulted to None while
McpServerConfig defaults to 60s: a server created via PUT /api/mcp/config
without the field was persisted with null (no timeout) while the same
server created in the config file got 60s. Align the response-model default
to DEFAULT_MCP_SESSION_INIT_TIMEOUT so API-created and file-created servers
behave the same; an explicit null still opts out.
* review: narrow the discovery-timeout handler to the bounded wait_for path
The except TimeoutError clause covered both the bounded wait_for branch and
the bare discovery branch. With session_init_timeout opted out (None), a
TimeoutError raised by discovery itself would hit the %.1f format with None:
logging raises TypeError internally, the WARNING is silently dropped, and a
--- Logging error --- traceback goes to stderr.
Narrow the handler to wrap only the wait_for call, where the branch condition
guarantees the timeout value is not None. A discovery-internal TimeoutError on
the opted-out path now falls through to the generic failure handler and is
reported as 'tool discovery failed' with exc_info. Covered by a regression
test that asserts the skip is reported without any broken format.
* feat(channels): add Buzz (Nostr) channel connector
Adds a Buzz (https://github.com/block/buzz) channel so DeerFlow can join a
Nostr-relay workspace as a member: it answers @mentions in channels, replies
to DMs, and streams answers by editing one message in place.
* app/channels/buzz_nostr.py — pure NIP-01 helpers: canonical event ids,
BIP-340 signing/verification, chat/edit/auth builders, relay frames.
* app/channels/buzz.py — BuzzChannel: one NIP-42-authenticated websocket,
channel discovery (kind 39000) with one subscription per channel, live
membership tracking (44100/44101), per-channel replay watermarks, and
replies posted once then edited in place (kind 40003).
* app/channels/buzz_run_policy.py — same-thread serialization, mirroring
the Feishu precedent.
Inbound is gated in order: signature verification, self-drop, /connect
bind-and-return, pubkey allowlist, then mention / DM / mention-free /
thread-follow. Off by default; needs the new optional `buzz` extra
(coincurve, lazily imported), which detect_uv_extras resolves from
channels.buzz.enabled the same way it already handles channels.discord.
Two relay behaviours drove the design and are worth knowing when reviewing:
a global {"kinds":[9]} subscription receives nothing from buzz-relay and a
multi-value "#h" filter receives nothing either, so one REQ per channel is
required; and a single global `since` cursor skips quiet channels, so
watermarks are per channel.
Signed-off-by: Ajay R <ajayr@formbuddy.com>
* fix(channels): only publish assistant messages from the IM stream
`_accumulate_stream_text` decided what streamed `messages-tuple` payloads
become displayable text by rejecting ONLY payloads whose `type` contained
"tool", so it published everything else. DeerFlow writes hidden model
context into the messages channel as ordinary messages -- memory recall and
the rewritten user turn as hidden HumanMessages (DynamicContextMiddleware),
the `<durable_context_data>` block as another (DurableContextMiddleware) --
and LangGraph fans those state writes out on the messages stream, so they
reached every streaming IM channel as the assistant's reply.
Proved live on a Buzz relay: the connector published a `<memory>` fact block
and, in another run, a verbatim echo of the user's own inbound message.
Affects Feishu, Telegram, WeCom and Buzz; worst on Buzz, where each update
is an immutable public Nostr event that a corrective edit cannot unpublish.
Invert the filter to an allowlist of assistant message types. Two new pure
helpers keep it testable:
- `_stream_payload_type` resolves the type from both shapes the function
already handles: the `model_dump()` shape the gateway emits, and
LangChain's `to_json()` constructor shape whose own `type` is the literal
"constructor" and whose class name is the tail of the `id` path.
- `_is_assistant_stream_type` matches "ai"/"assistant" by PREFIX, not
substring -- ordinary words contain "ai" ("chain", "domain"), and a
substring test would admit a foreign type name by accident.
The bare-`str` branch is removed: an untyped payload cannot be attributed to
the assistant, nothing in DeerFlow produces one (serialize_messages_tuple
always emits `[message_dict, metadata]`), and a runtime that emitted raw text
deltas would emit hidden context the same way. Per-message-id buffering and
merging are unchanged.
Tests pin both directions, including multi-chunk merging across one message
id, so the allowlist cannot silently kill streaming, plus an end-to-end
`_handle_streaming_chat` test asserting the live payload never reaches an
outbound message.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Ajay R <ajayr@formbuddy.com>
* chore(helm): bump config_version to 33 in chart values and README
config.example.yaml moved to 33 for the buzz channel block; the chart's
embedded config example and its README copy track it (config_version only
drives the outdated-config warning, per scripts/check_config_version.sh).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Ajay R <ajayr@formbuddy.com>
---------
Signed-off-by: Ajay R <ajayr@formbuddy.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
InputSanitizationMiddleware's text extraction only collected dict blocks
with type == "text", so a HumanMessage whose content list carried a bare
str item (a shape message_content_to_text treats as text and some IM/SDK
clients send) yielded no text at all — the request passed through
unwrapped and unescaped, letting forged framework tags (<system-reminder>
etc.) reach the model untouched. The sibling rfind-fallback path in
_process_request already neutralized bare strings individually, and both
ToolResultSanitizationMiddleware and ToolOutputBudgetMiddleware treat
bare strings as text; the extraction helper was the odd one out.
Collect bare string blocks alongside text-block dicts (skipping empty
items, matching message_content_to_text), merging them into the single
sanitized text block on rebuild while interleaved non-text blocks keep
their positions.
* feat(extensions): add middleware plugin foundation
* fix(extensions): stop config resolution from masking extension loading
`create_app()` resolved the configured plugin list inside the fail-open
guard around `load_extensions()`. CI has no `config.yaml` (gitignored and
never generated by the workflow), so `get_app_config()` raised
`FileNotFoundError` there and was swallowed as an extension failure --
`load_extensions()` never ran at all, and the four `create_app()` tests in
`test_extension_app_loading.py` passed locally but failed on every runner.
Resolve the plugin list before the guard. Only an absent `config.yaml` is
tolerated, mirroring `_resolve_trace_enabled_for_app_construction()`:
`create_app()` runs at import time, and lifespan still performs strict
config loading before serving. A `config.yaml` that exists but fails to
parse or validate now propagates instead of being reported as an extension
failure -- reporting it as the latter silently dropped a `required: true`
extension rather than failing the boot.
Make the tests config-independent with an autouse `stub_app_config`
fixture, following the existing pattern in `test_gateway_lifespan_shutdown.py`,
and cover both new branches of the config-resolution boundary.
* fix(extensions): bind the run's extension snapshot through subagent delegation
The lead-agent path resolves one immutable loaded-extension snapshot per run
and binds it through task-store allocation and graph construction, but the
subagent path re-read the process-wide singleton at execution time. In
production both are the same object, yet a `set_loaded_extensions()` between
the lead run's start and a subagent's execution (test teardown, a future
hot-reload path) would let one run mix two extension generations — exactly what
the documented invariant exists to prevent.
The graph-build binding is a ContextVar scoped to synchronous construction, so
it has already exited by the time a tool delegates; the snapshot has to travel
through runtime context instead. The run worker publishes it under the
host-internal `EXTENSION_SNAPSHOT_CONTEXT_KEY` (written after the caller merge,
popped when the run has none, so a caller-supplied value is never
authoritative), `task_tool` reads it back through the type-checking
`resolve_run_extensions()`, and `SubagentExecutor` binds it at construction.
Callers outside the Gateway run path — embedded `DeerFlowClient`, standalone
LangGraph Server — install no snapshot and keep the existing
`get_loaded_extensions()` fallback.
* refactor(extensions): defer the ordering table by call, not by a lying tuple
`CORE_ORDERING_CONSTRAINTS` was a `tuple` subclass that overrode only
`__iter__` and resolved into a class-level `_resolved` side channel. A tuple
cannot populate its own storage after construction, so the instance stayed the
empty tuple it was built as: `len()` was 0, `bool()` was False, `in` was always
False, indexing raised, slicing and `reversed()` came back empty, and it
compared unequal to the plain tuples tests substitute for it — all while
iteration yielded the real constraints. Only `assert_ordering` consumed it, and
only by iterating, so the split went unnoticed.
The sibling `_AnchorTable(dict)` uses the same idea soundly because dict is
mutable: `self.update()` fills the real storage, making every inherited
operation correct. That trick does not survive the port to an immutable type.
Replace it with `core_ordering_constraints()`, matching how `stack.py` defers
the same kind of table via `_anchors()`. The deferral is kept — it is about
dependency direction, not just cycles: `extensions/` is the layer the
middleware layer calls into, so a module-scope `agents.middlewares` import here
points the dependency backwards and closes a cycle as soon as any middleware
imports something under `extensions/` at module level. Resolution stays at
`assert_ordering` time, which already runs inside the middleware builder.
Tests pin both halves: the returned value is a plain tuple whose len/bool/
membership/indexing/reversal/equality agree with iteration, and a subprocess
probe asserts importing `extensions.ordering` does not load the middleware
layer while calling the function does.
* fix(frontend): restore user message above same-run steps after mid-run reload
After a mid-run page reload, replayed messages-tuple steps can reach the
merged list before the turn's human message (the retained replay buffer
may have dropped it), and the live-only human is woven in before the next
shared history anchor. The steps panel then renders above the user
message it belongs to and the per-turn token row splits one run in two.
restoreLocalTurnMessageOrder only covers the locally submitted turn, so
add restoreReconnectedTurnMessageOrder for the reconnected path: a
visible AI/tool step above the last visible human is provably misplaced
when the same run also appears below the human (canonical history is
seq-sorted), or when the step is run_id-less (live-only) and a
run_id-less step follows the human. Pagination orphans from older turns
(#4399) and resent turns after an interrupted run fail both checks and
keep their position.
* fix(frontend): never move a completed turn's terminal answer below the next human
Branch-seeded threads and mocked feeds can share one run_id across turns,
so the same-run sandwich check alone misfired on completed multi-turn
history: a previous turn's final answer was pulled below the next user
message, stripping the turn of its branchable assistant group (e2e
branch-thread.spec.ts).
Only steps after the last terminal assistant answer (visible content, no
tool calls) in the segment are now candidates for reordering. A
still-streaming text step can look like a terminal answer until its tool
call arrives (#4304); it then stays above the human until canonical
history heals the order — an accepted transient, far safer than moving a
completed answer.
The instance-client signal is a one-level lexical-scope analysis, and issue #4296
enumerates the cases it deliberately does not report. Four of them were only
described in prose: a handle reached through a container item, a factory return, a
locally aliased constructor, or a dynamic getattr, plus sinks invoked as anything
other than name.method(...).
Pin them in test_python_declared_false_negatives_stay_unreported alongside the cases
already covered, so each is asserted against the runtime oracle -- the client really
is called and the scanner really is silent -- rather than assumed. Re-widening or
narrowing the model now has to change this test.
No behaviour change.
Two related fixes in the reasoning extraction path of
core/messages/utils.ts:
1. splitInlineReasoning's first pass stripped every closed
<think>...</think> pair unconditionally, so a message that discusses
the tag literally in markdown inline code (e.g. "Wrap your reasoning
in `<think>...</think>`") had its code span hollowed out and the
inner text shipped to the Reasoning panel. The streaming pass already
guards backtick-adjacent openers; apply the same guard to the
closed-pair pass so both passes agree on what counts as literal tag
talk.
2. getAssistantTurnCopyData fell back to reasoning via
`content ?? reasoning`, but extractContentFromMessage never returns
null, so the fallback was dead code and a reasoning-only turn (e.g.
stopped mid-thinking) rendered no copy button at all - inconsistent
with getMessageCopyData, which does copy reasoning in that case. Use
the same empty-string check it uses.
* feat(frontend): reopen the skill list after a skill is selected
Selecting a skill closed the composer's skill list for good: `/` no longer
reopened it, so a skill could not be looked up or swapped without deleting
the chip first.
The list now reopens from the editable text beside the chip, and picking an
entry swaps the chip rather than stacking a second activation, since the wire
format carries exactly one leading /skill. Builtin commands are withheld in
that state because they own the whole composer line, and Enter navigates the
list before submitting except while an IME is composing.
The trigger is unchanged: a slash still opens the list only at the start of
the input.
* fix(frontend): keep builtin names reserved in the reopened skill list
Withholding the builtin list from getMatchingSkillSuggestions in chip mode
also disabled the reserved-name filter it drives, so a custom skill named
after a builtin command became selectable there. Nothing rejects such a name
at install time, and submitting the resulting chip runs the command instead
of the skill.
Pass the builtin list as before and drop the builtin entries from the result
instead. The new regression covers both sides of the reservation, and the
reopen test now waits for the list before pressing Enter.
* fix(frontend): hide skills the slash parsers refuse from the picker
The composer picker reserved only the two builtin command names, while both
slash parsers refuse the seven names in the shared contract. A skill named
bootstrap, help, memory, models, new or status was therefore offered, could
be selected into a chip, and submitted — and then activated nothing, because
parse_slash_skill_reference drops the name on the way in. The turn reached
the model as literal text with no skill loaded and no error anywhere.
Reserve the contract names alongside the builtin ones, so the picker cannot
offer what the parsers will not honour.
* feat(checkpoint-cache): delta-mode checkpoint history cache with recursive compose
Read-only, invalidation-free cache for LangGraph delta-channel history
({writes, seed}) at the get_delta_channel_history choke point:
- database.checkpoint_cache config (memory|redis; max_entries 0=disabled;
redis bounded by TTL, Gateway/async only)
- memory LRU backend (copy-on-read, zero-serde hit path) and redis backend
(lazy import, degrades to all-miss on outage)
- CachedHistorySaver: recursive composition from the nearest warm ancestor
(depth budget 8), caching each level; depth-0 cold chains delegate one
inner fast-path walk. Entries keyed by immutable
(db, thread, ns, checkpoint_id, channel) — no invalidation, coherent
across workers
- provider wiring: wraps in delta mode only (async + sync), full mode
untouched; sync path is memory-only
- bench opt-in: DEERFLOW_CHECKPOINT_BENCH_HISTORY_CACHE=1
sqlite bench (500 updates, payload 2KB): write phase 2.28x at f=250,
1.32x at f=10; one delegated walk per thread cold start.
* chore(config): bump config_version to 32 for database.checkpoint_cache
The checkpoint history cache feature added the database.checkpoint_cache
section to config.example.yaml; bump the schema version so existing
deployments get the outdated-config warning and can run make config-upgrade.
* chore(helm): bump config_version to 32 in chart values and README
* fix(checkpoint-cache): purge thread history entries on delete paths
Addresses review on #4638: delete_thread/prune removed source-of-truth
checkpoints but left the thread's materialized history payloads in the
cache (memory: until LRU eviction; redis: until TTL, default 1 day) — a
data-lifecycle gap for tenant offboarding / GDPR-style erasure.
- Cache contract gains thread-scoped adelete_thread/delete_thread
(lifecycle purge, not invalidation; entries remain immutable)
- Memory backend: stem scan over the LRU map; redis: SCAN MATCH + UNLINK,
outage degrades to TTL-bounded retention without raising
- CachedHistorySaver purges on delete_thread/adelete_thread and
prune/aprune (prune rewrites chains, so pre-prune histories must go);
delete_for_runs stays delegation-only (run->thread mapping unavailable,
no in-tree callers), documented in code
- ttl_seconds description documents the residual-retention window
- Tests: thread-scoped purge on both backends, saver-level delete/prune
purge, prefix-safety (t1 vs t10), redis outage degradation, and the
pinned no-purge behavior of delete_for_runs
* fix(checkpoint-cache): stable db identity, prefix-aware sync singleton, explicit zero TTL
Addresses Copilot review on #4638:
- checkpoint_cache_db_hash now hashes the credential-free postgres
identity (host:port/database + schema): credential rotation no longer
changes the cache namespace (cold cache + orphaned keys until TTL).
Unparseable URLs fall back to the raw string.
- The sync-path memory cache singleton is also keyed by its key_prefix:
a namespace change (db identity change or operator override) recreates
the cache instead of leaving stale-prefix entries unreachable and
unpurgeable.
- ttl_seconds=0 is now an explicit, documented opt-out of redis expiry
(SET without EX; redis maxmemory policy only) instead of a silent
'ttl_seconds or None' coercion.
Tests: credential-rotation hash stability, unparseable-URL fallback,
prefix-change singleton recreation, same-prefix singleton reuse, and
zero-TTL wire behavior (ex=None).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
