* fix: surface length-capped model responses
* fix: avoid the influence of the mid-turn
* fix: correcting semantic annotations
* fix: add ModelLengthTerminationDetector to compatible providers
* fix:delete redundancy code
* fix:supplementing log information improves observability
* fix: align the document and complete the assertions.
* fix: unit test
* fix: revert AGENTS.md
* fix: unit test
* fix: add annotation and skip AIMessage has empty content
* fix(gateway): stop persisting Overwrite wrappers into empty reducer channels on branch
Thread branching (and POST /state on a never-written channel) wraps copied
reducer values in Overwrite. Upstream BinaryOperatorAggregate.update seeds
an empty (MISSING) channel with values[0] verbatim without unwrapping, so
Union-typed channels (sandbox/goal/todos/promoted) stored the wrapper
literally and the next consumer crashed with TypeError: 'Overwrite' object
is not subscriptable (#4380). Patch the channel to unwrap the first write
(mirroring DeltaChannel semantics), and stop copying thread-scoped channels
(sandbox, thread_data) into branches: the parent's sandbox_id would bind
the branch to the parent's workspace and release lifecycle.
* refactor(checkpoint): drop private _get_overwrite import for a local Overwrite check
Importing langgraph's underscored _get_overwrite at module top level meant an
upstream refactor that drops it - plausibly the same release that fixes the
bug - would fail this module's import and crash startup before the probe can
stand the patch down. Replace it with a local helper on the public Overwrite
type, and fix two test docstring nits.
* refactor(checkpoint): write patch flags via their constants to avoid drift
Both saver patches read their "already patched" idempotence flag through a
module constant (_PATCH_FLAG / _BINOP_PATCH_FLAG) but wrote it as a hard-coded
attribute literal, so renaming the constant would silently break the guard and
double-apply the patch. Write via the same constant (setattr), dropping the
now-unneeded attr-defined ignores.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): unwrap Overwrite-wrapped sandbox state in after_agent
Fork-restored checkpoints can deliver the sandbox channel still wrapped
in langgraph.types.Overwrite: the rollback restore applies replace-style
writes through a state-mutation graph in delta checkpoint mode, and
after_agent/aafter_agent then crash subscripting the wrapper
("TypeError: 'Overwrite' object is not subscriptable") on the next
sandbox tool run in the forked conversation. Unwrap before reading the
sandbox id, and pin both hooks against an Overwrite-wrapped state.
Refs #4380 (bug 1 of 2; the history-loss half is a separate display path)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(sandbox): don't release fork-restored parent sandboxes
The Overwrite-wrapped value replays the parent thread's sandbox state, so releasing it from the forked run would evict the parent's warm sandbox. _unwrap_sandbox now reports the wrapped form, and both after_agent hooks skip the release for it while keeping the normal path unchanged.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
---------
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: add lark cli integration
* fix: polish lark integration actions
* feat: support lark incremental permissions
* fix: detect lark authorization completion
* fix: harden lark integration install
* feat: expand lark auth scopes and reuse host auth in sandbox
Default lark auth to least-privilege (recommend=false, base sign-in only)
and expose the full set of lark-cli --domain business domains as native
--domain grants instead of a 4-domain read-only mapping. Resolve the
skill pack from the latest larksuite/cli GitHub release at install time
with content-hash integrity, and surface version/runtime drift in status.
Share the per-user lark-cli config/data profile between the Gateway
Settings auth flow and agent conversations by mounting the integration
dirs into the AIO sandbox and injecting the matching env for lark-cli
commands, with an allowlisted extra_mounts path in the provisioner/K8s
backend and traversal guards on integration paths.
* style: fix lint issues from ruff and prettier
Sort imports in the provisioner PVC test and re-wrap two long i18n
description strings to satisfy backend ruff and frontend prettier CI.
* fix(lark): address managed integration review feedback
* fix(frontend): stabilize integrations settings e2e
* test(sandbox): isolate remote backend legacy visibility check
* test: fix backend unit failures after merge
* Harden Lark integration review fixes
* Format Lark integration E2E test
* fix(lark): harden sandbox credential exposure and status disclosure
Address willem_bd's security review on PR #3971:
- Mount the per-user lark-cli config dir (long-lived appSecret) read-only
into the AIO sandbox; only the refreshable-token data dir stays writable.
- Redact host filesystem paths (install_path, cli.path) from
GET /lark/status and the config/auth complete responses for non-admin
callers, fail-closed on any auth error.
- Document the npm postinstall trade-off (--ignore-scripts is not viable
because @larksuite/cli fetches its platform binary in postinstall).
- Document the sandbox credential trust boundary in AGENTS.md and README,
pointing at the sidecar-broker follow-up (#4338).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Branch creation seeds the new thread's run-event feed from its checkpoint
so inherited history survives the first run (#4380). Every seeded row
carried one shared run id, but run_id is a *turn* identity to the feed's
consumers, not a provenance tag: regenerating the inherited answer
resolves that row's run id as the superseded source, and
GET /messages/page then drops every row carrying it. One shared id for
the whole seed therefore deleted the complete inherited history on a
branch's first regenerate, leaving only the regenerated turn.
Group seeded rows into one synthetic run per inherited turn
(branch-seed-{thread_id}-{n}), a new turn opening at each persisted human
message — the same boundary a real run has, including the allowlisted
hidden ask_clarification reply, which resumes as its own run. Supersession
is then confined to the turn actually regenerated.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(summarization): own the run model for compaction; bound failure
With summarization.model_name: null the summary model resolved to
config.models[0] while the executing model is selected per run; when they
differ and models[0]'s provider is broken (expired key, quota, outage)
compaction silently failed every triggered turn and context grew unbounded
until the main provider 400s the run (#3103's shape), even though the run's
own model was healthy.
Model ownership is now sourced from the builders, not re-derived at runtime:
- The lead, subagent, and manual /compact builders each pass the resolved run
model into create_summarization_middleware(run_model_name=...). The middleware
no longer reads runtime.context / get_config(), which do not carry a custom
agent's or a subagent's resolved model, so a custom-agent lead run and a
distinct-model subagent now summarize with their own model, not models[0] /
the parent's. Runtime re-resolution and the per-name model cache are removed.
- model_name: null summarizes with the run's own model; an explicitly configured
summary model generates and falls back to the run model on failure. The
fallback is built lazily after the primary fails and its construction is
guarded, so a broken fallback cannot skip a healthy primary or escape the
automatic failure boundary.
Failure is bounded and side-effect-safe:
- An empty or whitespace-only response is treated as a generation failure, not a
valid summary, so compaction never removes all history for an empty replacement.
- compact_state/acompact_state take raise_on_failure independent of force: the
manual /compact path always surfaces a generation failure (even force=false)
and routes it to the existing ContextCompactionFailed path (HTTP 500 ->
frontend error toast) instead of an unconsumed response reason. The automatic
path leaves compaction state unchanged.
- before_summarization hooks fire only after a replacement summary exists.
SummarizationConfig.model_name, config.example.yaml, and docs/summarization.md
document the final lead/subagent/manual ownership rules.
Part of RFC #4346 (section A). Evaluating fraction/triggers against the run
model's profile (profile ownership) is a separate follow-up.
* fix(summarization): manual /compact model ownership + fail-open construct/parse
Manual /compact carried only agent_name, so it derived the run model from the
custom-agent model or config.models[0] and missed the request-selected model the
run path uses (request -> custom-agent -> default). Carry model_name through
ThreadCompactRequest and the frontend compact call, resolve with the same
precedence, and move the custom-agent config read off the event loop (asyncio
.to_thread) with user_id so the strict blocking-IO gate is not bypassed by the
broad except.
Make one summary attempt own its full lifecycle so the fail-open boundary covers
construction and response parsing, not just invocation: build each candidate model
lazily and guarded (a raising constructor falls through to the healthy run model
instead of breaking agent construction), build the model_name:null primary from the
run model rather than config.models[0], and run response text extraction inside the
invocation try so a failing .text accessor falls back instead of escaping compaction.
Adds factory-level constructor-failure, response-extraction-failure (sync/async), and
route-path model-ownership tests.
* fix(subagents): align prompt and middleware subagent limit; allow min of 1
SubagentLimitMiddleware clamped max_concurrent to [2, 4] internally, but
agent.py and client.py fed the raw config value into the system prompt, so
a user-configured 1 (or 5) produced a prompt that disagreed with the
enforced middleware limit. Lower MIN_SUBAGENT_LIMIT to 1 and clamp the raw
config value with _clamp_subagent_limit() at both the agent factory and the
embedded client so the prompt and middleware see the same value.
* fix: remove unused imports MAX_CONCURRENT_SUBAGENT_CALLS, MIN_CONCURRENT_SUBAGENT_CALLS, clamp_subagent_concurrency
* fix: harmonize clamp range [1,4] across middleware, config, and prompt path; fix lint
- Changed MIN_CONCURRENT_SUBAGENT_CALLS from 2 to 1 so prompt.py's
clamp_subagent_concurrency and the middleware's _clamp_subagent_limit
both clamp to [1,4] — eliminating the divergence where the prompt told
the model 'max 2 task calls' but the middleware enforced 1.
- Applied _clamp_subagent_limit at build_middlewares (agent.py:360) so
all 3 construction sites (agent.py:360, agent.py:450, client.py:259)
consistently clamp the config-resolved limit.
- Derived MIN_SUBAGENT_LIMIT / MAX_SUBAGENT_LIMIT from
MIN_CONCURRENT_SUBAGENT_CALLS / MAX_CONCURRENT_SUBAGENT_CALLS so the
two module-level definitions stay in sync.
- Added TestConfigParity.test_prompt_path_and_middleware_clamp_agree
regression test.
- Fixed lint.
* fix(lint): add missing imports for MIN_CONCURRENT_SUBAGENT_CALLS and MAX_CONCURRENT_SUBAGENT_CALLS
* docs+test: update AGENTS.md clamp range to 1-4; add prompt/middleware parity regression test
- backend/AGENTS.md still documented the old [2,4] clamp in two places;
updated to [1,4] to match MIN_CONCURRENT_SUBAGENT_CALLS = 1.
- Added test_apply_prompt_template_single_subagent_limit_matches_middleware:
renders the real system prompt with max_concurrent_subagents=1 and asserts
the advertised HARD LIMITS value equals SubagentLimitMiddleware's enforced
max_concurrent — the end-to-end check that would have caught the [1,4] vs
[2,4] prompt-path divergence flagged in review.
* refactor: simplify per review — restore clamp delegation, drop redundant call-site clamps
Per willem-bd's review, reduce the PR to the one behavioral change plus
docs/tests:
- _clamp_subagent_limit delegates to clamp_subagent_concurrency again
instead of inlining a byte-identical copy; with a single source of
truth the TestConfigParity sync-check class is unnecessary — dropped.
- Revert the call-site clamps in agent.py (build_middlewares,
_make_lead_agent) and client.py (_ensure_agent) to main: both
downstream consumers (SubagentLimitMiddleware.__init__ and the prompt
path) already clamp internally, and the cross-module private import
of _clamp_subagent_limit goes away with them.
- Keep MIN_CONCURRENT_SUBAGENT_CALLS = 1 (the fix), the [1, 4]
docstring updates, the AGENTS.md range corrections, and the
end-to-end prompt/middleware parity test for single-subagent mode
(docstring reworded: on main a configured 1 was bumped to 2 by both
paths — there was no divergence to fix, just a silently raised floor).
* test: fix stale comment referencing reverted agent.py/client.py call-site clamps
---------
Co-authored-by: nankingjing <nankingjing@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Enforce one queued or running scheduled-task run per task with a partial unique index. The migration resolves legacy duplicates before creating the index, and losing inserts use the existing conflict or skip outcomes.
Normalize YAML frontmatter keys in the shared parser so validation and review report malformed fields instead of failing while sorting mixed key types.
Add GIF to the view_image allowlist: map the .gif extension to
image/gif and detect the GIF87a/GIF89a magic bytes so the existing
extension/content cross-check accepts GIFs instead of rejecting them
as an unsupported format. Covered by a new success test.
* fix(runtime): stop subgraph stream frames impersonating root frames
The web frontend always requested stream_subgraphs, and since delegated
subagent graphs inherit the parent checkpoint namespace (#4215), their
values snapshots and token chunks ride the parent stream. The worker's
_unpack_stream_item dropped the namespace and published every subgraph
frame under a bare event name, so a subagent's values snapshot replaced
the whole thread view in SDK clients (#4399), its token chunks flooded
the parent message stream, and a subagent's LLM error fallback could be
mistaken for the parent run's.
Publish subgraph frames under namespace-qualified SSE event names
(mode|ns1|ns2, LangGraph Platform style) and keep root-only consumers
(file-tool chunk batcher, subagent event persistence, error-fallback
detection) on root frames only. Drop streamSubgraphs from the frontend
submit paths: subtask progress arrives via root-namespace task_* custom
events, so the flag only exposed the leak.
* test(runtime): add production-shaped subgraph stream regression tests
Address review: the namespace tests validated the publishing helpers
with hand-fed namespaces, while the #4399 regression lived in the
integration between LangGraph's delegation routing and the worker's
stream loop. Add TestWorkerSubgraphStreamIntegration: a real parent
graph delegates through the real SubagentExecutor and streams through
run_agent into a real MemoryStreamBridge, locking both stream_subgraphs
modes -- delegated frames arrive namespaced (never bare), a delegated
error fallback cannot mark the parent run as errored, and without the
flag delegated frames stay out while task_* custom events remain.
An empty assistant message from a provider safety filter (content_filter with
no content, no tool calls) was persisted into thread history and replayed to
strict OpenAI-compatible providers, which reject it with HTTP 400 ("message ...
with role 'assistant' must not be empty") — breaking every later turn until a
new chat is started.
SafetyFinishReasonMiddleware only handled the tool-call case (#3028) and
TerminalResponseMiddleware only the post-tool case (#4027), so a plain empty
content-filter response fell through both. Extend the safety middleware to
backfill a user-facing explanation when a safety-terminated message is
otherwise blank, so the persisted turn is non-empty (and the user sees why it
was blocked).
Fixes#4393
* fix(gateway): seed branch run-events so inherited history survives (#4380)
The thread feed (GET /messages, /messages/page) reads the run-event store,
but branch creation only wrote checkpoint state - a fresh branch had no
message rows, so the parent history vanished from the UI as soon as the
branch's first run refreshed the feed. Seed the branch's run_events from
the same checkpoint snapshot the branch was created from, mirroring
RunJournal's message-event contract (event types, hidden-message rules,
original-user-text restoration). Best-effort: a seeding failure degrades
to the old behavior and is reported as history_seed_mode=failed.
* docs(gateway): correct branch-seed docstring on RunJournal divergences
The "consumers cannot tell a seeded row from a journaled one" claim was
overstated for AI rows: seeded rows omit run-scoped enrichment (usage /
latency_ms / llm_call_index) and stamp caller=lead_agent rather than the
message's original caller, neither recoverable from a checkpoint message.
Rewrite the docstring to state these divergences explicitly and note they
are display-invisible today (no consumer indexes those keys; per-message
caller drives no attribution). Also add a code comment marking the
hide_from_ui filter as intentionally stricter than the live paths.
* fix(gateway): seed dict-shaped checkpoint messages + persist hidden AI/tool rows
Two review-driven fixes to build_branch_history_seed_events:
1. Checkpoint messages can arrive as model_dump()-shaped dicts (the
branch-matching helpers in threads.py already handle both BaseMessage
and dict). The seed only handled BaseMessage, so a dict-backed
checkpoint seeded nothing and the branch reported skipped_empty while
history existed. Coerce dicts back to BaseMessage via messages_from_dict
(faithful: tool_calls / tool_call_id / additional_kwargs survive);
unparseable dicts are dropped best-effort.
2. RunJournal.on_llm_end and _persist_tool_result_message persist
hide_from_ui AI/tool rows unconditionally (the frontend hides them
client-side); the hide check only gates the reconciliation pass. The
seed dropped them, so a hidden turn vanished from a forked feed and
seeded rows diverged from journaled ones. Match RunJournal and write
them, restoring true row-level parity.
Adds tests for dict deserialization, the unparseable-dict drop, and the
hidden AI/tool persistence contract.
Declaring the injected runtime arg as `Annotated[Runtime, InjectedToolArg] | None`
made the top-level annotation a Union, so LangChain no longer treated it as
injected. It leaked into the model-facing schema and pydantic raised
PydanticInvalidForJsonSchema on the ToolRuntime dataclass the moment the tool
was bound to a model. The tool is bound by default for the lead agent, so any
default run on an OpenAI-compatible provider failed at tool-bind time.
Declare runtime as a bare Runtime first param, matching every other built-in
tool (present_files, view_image, task, ...), which LangChain auto-injects and
auto-excludes from the schema. Add a schema regression test that binds the tool.
* feat(agents): database-backed storage for custom agent definitions
Add an agent_storage.backend switch (default file, behaviour-unchanged) with a
db backend that stores each custom agent as a row in the shared SQL persistence
layer, so a multi-instance deployment sees the same agents on every node
(#4331, #4357). Introduces an AgentStore interface routing all read/write
surfaces, an agents table + migration 0006, startup validation, and a file->db
importer. Follows the thread_meta store / run_events backend-switch /
0003_scheduled_tasks migration patterns; no new dependency.
* fix(agents): make db storage path production-ready (review round 1)
Addresses review feedback on the db/sync agent-storage path:
- sql.py: mirror the async engine's per-connection SQLite PRAGMAs on the sync
engine (busy_timeout=30000, synchronous=NORMAL, foreign_keys=ON, WAL) so both
engines behave identically against the shared DB; guard the engine cache with
a lock (double-checked) so concurrent first-touch cannot build duplicate
engines or register the connect listener twice.
- routers/agents.py + routers/assistants_compat.py: offload the sync-store reads
that ran on the event loop (list/get/check, update's pre-read + legacy guard +
refresh, and assistants_compat's four list routes) via asyncio.to_thread — on
db+postgres each was a network round trip stalling the loop. Writes were
already offloaded.
- file.py: translate the create() mkdir(exist_ok=False) race FileExistsError
into AgentExistsError (router 409, matching SqlAgentStore's IntegrityError
path); correct the _write docstring — per-file atomic replace, two commits
sequential not transactional.
Tests: sync-engine PRAGMA + engine-cache reuse assertions; file create-race ->
AgentExistsError; strict Blockbuster anchor over the read endpoints so a
regression back onto the loop fails CI.
* fix(agents): address round-2 review on the db store path
- update_agent tool: align the docstring/inline comment with FileAgentStore._write.
Cross-field write atomicity is db-only; the file backend commits config then
soul via two sequential os.replace (a crash between them can leave a fresh
config.yaml beside a stale SOUL.md). The dropped partial-write *reporting* is
an intentional tradeoff — the stage-then-replace safety is preserved
(test_update_agent_soul_failure_does_not_replace_config still holds).
- SqlAgentStore.update(): true upsert. Catch IntegrityError on the
insert-on-missing branch, re-fetch and apply, so two concurrent first-time
writes (e.g. two setup_agent handshakes) converge instead of surfacing a raw
UNIQUE(user_id, name) violation as a 500. Symmetric with create().
- get_agent_store(): document the graph-subprocess config-resolution invariant
(the except->file fallback is a genuine no-config path, not a mask for a
misconfigured graph process) and pin it with two tests driving the real
get_app_config() file resolution: db resolves from an on-disk config.yaml,
file fallback when config is unresolvable.
* test(agents): cover SqlAgentStore.update() write-race upsert recovery
Mandatory-TDD test for the round-2 fix in 0680340a: two concurrent first-time
update()s where the loser's insert hits UNIQUE(user_id, name). Deterministically
forces the IntegrityError recovery path by making the first _row probe miss the
committed winner, and asserts last-writer-wins instead of a surfaced 500.
* fix(mcp): use threading.Lock for OAuth token refresh to avoid cross-thread deadlock
OAuthTokenManager created one asyncio.Lock per server for the process
lifetime. The embedded/TUI sync tool-call path (DeerFlowClient.stream()
-> LangGraph's ToolNode._func -> a ThreadPoolExecutor ->
make_sync_tool_wrapper's per-call asyncio.run()) invokes
get_authorization_header from a fresh event loop on a fresh OS thread
for every concurrent tool call. asyncio.Lock binds to whichever loop
first contends on it; when a caller on a different loop later releases
or wakes a waiter, it does so without call_soon_threadsafe, so the
waiting loop's selector is never woken and that caller hangs forever
with no exception. A third concurrent caller instead raises a
synchronous RuntimeError ("bound to a different event loop"). Either
way, two concurrent OAuth-protected tool calls (including the very
first cold-start token fetch) can freeze the entire agent turn.
Gateway's async path (ToolNode._afunc) is unaffected.
Replace the asyncio.Lock with a plain threading.Lock, acquired via
asyncio.to_thread so the blocking wait never blocks the event loop,
and released synchronously in a finally block. This keeps the
single-fetch de-duplication the lock provided while making it safe
across however many event loops/threads call into the same server's
lock.
Adds a regression test that runs three threads, each with its own
event loop, calling get_authorization_header concurrently for the same
server, and asserts (with a bounded join timeout so a regression fails
fast instead of hanging the suite) that none hang or raise, and that
only one real token fetch happens.
* fix(mcp): make OAuth lock acquisition cancellation-safe
get_authorization_header acquired the per-server threading.Lock via a
bare `await asyncio.to_thread(lock.acquire)`, with the try/finally that
guarantees release only starting after that await returned. Once the
executor thread had actually started running lock.acquire(), cancelling
the awaiting caller only stopped the caller -- Python cannot interrupt a
running OS thread. CancelledError was still delivered to the caller
immediately, but the thread kept blocking until the current holder
released, then silently acquired the lock with nobody left to call
release() for it. The lock stayed locked forever and every later OAuth
token refresh for that server blocked permanently at the same line --
the exact cross-thread deadlock this lock was introduced to prevent,
reintroduced via a different path under cancellation (e.g. a caller
wrapped in asyncio.wait_for/asyncio.timeout, or task-group cancellation).
Run the acquisition as an explicit asyncio.create_task, awaited via
asyncio.shield() so cancelling the caller no longer cancels the
underlying acquisition task. If the caller is cancelled, keep
(re-)waiting on the still-shielded acquisition task -- tolerating
further cancellation during this cleanup by simply retrying -- until it
actually finishes, release the lock immediately, and only then
re-raise. This guarantees the lock is released regardless of when or
how many times the caller is cancelled: before the acquisition is even
scheduled, while queued, or after it has already been silently granted.
Adds a regression test that holds the per-server lock, starts a second
caller that has to wait for it, cancels that caller while it is
genuinely blocked in its executor thread, releases the original holder,
and asserts a third caller completes within a bounded asyncio.wait_for
and still performs exactly one token fetch. Every potentially-hanging
await is bounded so a regression fails the test quickly instead of
hanging the suite.
invoke_acp_agent had no timeout anywhere in its call path, and
ACPAgentConfig had no timeout field. If the ACP agent subprocess answers
initialize/new_session correctly but then hangs inside prompt(), the tool
call - and therefore the whole agent turn - blocks indefinitely, with the
child process left running. MCP stdio servers already guard against this
class of hang via tool_call_timeout; ACP agent invocations had no
equivalent.
Add ACPAgentConfig.timeout_seconds (default 1800, ge=1), mirroring the
shape/default of subagents.timeout_seconds, and wrap the conn.prompt()
call in asyncio.wait_for(). On TimeoutError, return a clear error instead
of hanging; exiting the spawn_agent_process context block triggers the
ACP library's own graceful-then-forceful subprocess cleanup, so the hung
process is actually terminated.
* refactor(memory): pluggable MemoryManager interface for backend onboarding
Optimize the MemoryManager interface layer so new backends (mem0/openviking)
onboard with less code and the contract stays stable as capabilities are
added. A minimal backend now implements only from_config + add + get_context
(verified by test_memory_manager_interface.py::_MinimalBackend onboarding via
the factory); the factory no longer knows a backend's private hooks.
- MemoryManager: ABC -> pydantic BaseModel; three-tier methods (tier-1
add/get_context abstract; tier-2 management defaults; tier-3 optional hooks
warm/reload/fact + on_pre_compress/on_turn_start). Dropped 3 self-serving
hooks. 6 hasattr probe sites -> direct call + try/except NotImplementedError.
- from_config classmethod: factory thins to resolve + inject storage_path +
collect host hooks + call from_config; DeerMem-specific hook consumption
moved from factory to DeerMem.from_config.
- Invariants: @model_validator (mode='tool' requires search via supports_search
ClassVar); DeerMemConfig storage_path-is-file check moved here from factory.
- Async: aadd/aget_context/asearch default to the sync path (speculative).
- Callbacks: MemoryCallbacks + LangfuseMemoryCallbacks; on_memory_llm_call
subsumes tracing_callback (same signature/timing/mutation); deleted the
tracing_callback field. DeerMem decoupled from langfuse (portability).
- noop keeps read-op empty overrides (avoids router 500s on the
disable-memory-via-noop path); only delete/export inherit the base raise.
Behavior preserved: 661 passed / 13 skipped. Docs: backends/README.md rewritten
(three-tier + from_config + callbacks); samples README updated; removed stale
private doc paths.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(memory): 501 on unsupported read/manage endpoints + accurate warm log
Review follow-up on the three-tier MemoryManager refactor.
- Read/manage endpoints (GET /memory, /memory/export, /memory/status,
DELETE /memory, POST /memory/import) and the /memory/reload fallback now
catch NotImplementedError -> 501, matching the fact-CRUD endpoints. The
hasattr->try/except migration had skipped these: they were @abstractmethod
before (every backend implemented them, so they never raised), so once they
became tier-2 default-raise a minimal backend (only add + get_context) hit a
raw 500 -- there is no global NotImplementedError handler. get_memory is
shared via _get_memory_or_501 (covers /memory, export, status, reload
fallback). noop is unchanged: its read-op empty overrides never raise.
- warm() base default returns None (tri-state: True=warmed, False=failed,
None=nothing to warm) so the Gateway lifespan logs "skipping" for a
non-DeerMem backend (e.g. noop) instead of the inaccurate "warmed
successfully" it never earned. DeerMem.warm keeps True/False.
- Tests: 6 router 501 tests (read/manage + reload fallback) + 2 lifespan
warm-log tests (None->skipping, False->warning); conformance/pluggable
assert warm() is None.
705 passed / 13 skipped; lint clean.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(memory): review follow-ups - search-flag consistency, client reload, backend_config purity
Address review feedback on the three-tier MemoryManager refactor:
- [Medium] supports_search/search drift: the invariant now requires the
supports_search ClassVar flag to MATCH whether search() is actually
overridden (type(self).search is not MemoryManager.search), so the flag
can't drift from the impl. Catches both directions at instantiation: a
backend that overrides search() but forgets supports_search=True (was a
misleading tool-mode rejection), and one that sets the flag without
overriding (was a runtime NotImplementedError on the first memory_search).
noop sets supports_search=True to match its search() override. Conformance
adds drift + consistent-backend tests.
- [Low] client.reload_memory fallback: wrap the get_memory fallback so a
minimal backend (only add + get_context) surfaces a clean NotImplementedError
("implements neither reload_memory nor get_memory") instead of an uncaught
propagation -- mirrors the router's 501. Test added.
- [Low] backend_config purity: DeerMem.from_config restores backend_config to
the pure data the host passed after model_post_init parses the injected hooks
into DeerMemConfig (self._config, PrivateAttr); the field stays serializable
(no callables/LLM) and matches the README ("host hooks NOT in backend_config").
Test asserts purity + hooks wired.
- [Low] CHANGELOG: breaking-change note that mode='tool' + non-search backend
now fails fast at startup (was silently empty) so operators recognize it on
upgrade.
- [Nit] .gitignore: drop the env-specific .tmp-pytest/ entry (--basetemp is
local-only, not make test/CI).
709 passed / 13 skipped; lint clean.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(changelog): correct memory tool-mode fail-fast note
The CHANGELOG entry said mode='tool' + a non-search backend "(e.g. noop)"
fails fast at startup, but noop overrides search() (returns []) and sets
supports_search=True (required by the consistency invariant), so noop IS
search-capable and noop+tool does NOT fail fast. The fail-fast only affects a
custom backend that onboards without overriding search(). Reworded to drop the
misleading noop example and state both shipping backends implement search().
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix(sandbox): bound E2B output synchronization resources
E2B release-time output sync pulled every changed file back from the
remote VM with only a per-file size cap and no aggregate bound, so a
pathological outputs tree (thousands of files, or many sub-cap files
summing to gigabytes, or a slow VM) could make release download
unboundedly on a hot path that runs at every agent turn end.
Add three aggregate ceilings on top of the per-file cap — total bytes,
file count, and a wall-clock deadline — enforced in the sync loop. When
a ceiling is hit the pass stops early, logs what it dropped, and defers
the rest to the next release. A truncated pass skips stale-manifest
pruning so files it never reached are reconciled next time instead of
being forgotten and re-downloaded.
Closes#4340
* test(sandbox): pin multi-pass convergence of bounded output sync
The four truncation tests each exercise a single capped pass. Add a
two-pass test that locks in the invariant the design relies on for
correctness: already-synced files are skipped before the budget check,
so they never consume the cap and the deferred tail drains over
successive releases instead of the leading files being re-downloaded
every turn. A refactor that let a skipped file consume the cap would
pass the single-pass tests but fail this one.
* feat(uploads): lazy-load historical files via list_uploaded_files tool
Replace per-turn injection of all historical upload metadata with on-demand
discovery via a new `list_uploaded_files` built-in tool, following the same
deferred-discovery pattern used by skills.
- Rename <uploaded_files> block to <current_uploads> (current-run files only)
- Add list_uploaded_files tool with include_outline: bool|list[str]
- Extract outline helpers to shared deerflow/utils/file_outline.py
- Update system prompt to reflect lazy-loading behaviour
- Historical file scan removed from UploadsMiddleware.before_agent()
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(uploads): clear uploaded_files state when no new files in current turn
When before_agent() returns None on empty turns, the LastValue
uploaded_files field retains the previous turn's filenames.
list_uploaded_files then incorrectly excludes those files as
"current-run" files, making them invisible until the next upload.
Fix: return {"uploaded_files": []} instead of None to explicitly
clear state. Add two-turn regression test covering the exact
scenario from review feedback.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: resolve CI lint errors and stale test assertion from merge
- Split long prompt line to fit 240-char limit
- Add missing `Any` import in list_uploaded_files_tool
- Remove unused `re` import in file_conversion (outline code moved)
- Remove unused `os` import in middleware test
- Fix test assertion: <uploaded_files> → <current_uploads> after main merge
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: resolve CI lint errors and stale test assertion from merge
- Split long prompt line to fit 240-char limit
- Add missing `Any` import in list_uploaded_files_tool
- Remove unused `re` import in file_conversion (outline code moved)
- Remove unused `os` import in middleware test
- Fix test assertion: <uploaded_files> → <current_uploads> after main merge
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: add current_uploads to input sanitization exempt tags
The lazy-loading PR renamed <uploaded_files> to <current_uploads>.
The anti-drift guard scans all framework XML blocks and requires each
to be either blocked or explicitly exempted. current_uploads wraps
trusted server-generated file metadata, not user input, so it belongs
in the exempt set.
Co-Authored-By: Claude <noreply@anthropic.com>
* test: regenerate replay golden after uploaded_files state change
before_agent now returns {"uploaded_files": []} instead of None,
adding uploaded_files to SSE values events. Regenerated via
DEERFLOW_WRITE_GOLDEN=1.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: review feedback — memory pipeline, stale tags, state clearing, nits
- Match both tags in memory stripping pipeline (uploaded_files|current_uploads)
- Remove stale uploaded_files from _BLOCKED_TAG_NAMES
- Clear uploaded_files on all before_agent early-return paths
- Fix ponytail: stray word in file_conversion re-export comment
- Remove dead total_omitted branch in _format_omitted_summary
- ruff format fixes
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: block current_uploads, sanitize only original user content
Per review feedback: instead of exempting <current_uploads> (which
allows user forgery), move it to _BLOCKED_TAG_NAMES and change
InputSanitizationMiddleware._process_request to scan only the
original user content (ORIGINAL_USER_CONTENT_KEY) when available.
Server-injected trusted blocks are no longer checked against the
blocked-tag denylist.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: clarify fallback reason in input sanitization comment
Co-Authored-By: Claude <noreply@anthropic.com>
* @
fix: third-round review feedback — state visibility, sanitization, regex, nits
- list_uploaded_files_tool: logger.warning instead of silent try/except
on runtime.state read failure (High)
- input_sanitization_middleware: _extract_text_from_content skips empty
text blocks to match message_content_to_text behaviour; rfind fallback
path logs warning for observability (Medium)
- memory pipeline regexes: backreference (?P<tag>)(?P=tag) in
message_processing.py and prompt.py (Low)
- file_conversion.py: re-export moved to top of file (Low)
- Tests: middleware→tool state bridge test; integrated forged-tag +
multimodal sanitization tests
PR #4174 — Follow-up issues: #4212, #4213, #4214
Co-Authored-By: Claude <noreply@anthropic.com>
@
* @
fix: 4th-round review — denylist, sanitization, scandir, nits
- Add "uploaded_files" back to _BLOCKED_TAG_NAMES (old tag still processed by
deermem; user forgery must be escaped) (consistency)
- Fix inaccurate rfind-fallback comment: UploadsMiddleware keeps string as
string, fallback is unreachable for strings (doc fix)
- Distinguish "empty string key" (upload without text) from "non-string key"
(caller forgery) so empty-text uploads never escape the server block (edge)
- Merge dual os.scandir(uploads_dir) calls into one list re-use (minor)
- Add comment on .md sibling skip known limitation: user-uploaded .md files
whose stem collides with a converted doc are hidden (boundary, no code change)
Co-Authored-By: Claude <noreply@anthropic.com>
@
* @
fix: tighten rfind-failure fallback — distinguish server blocks from user blocks
When _extract_text_from_content and message_content_to_text disagree on
multimodal list content and rfind fails, use content[0] (server-injected
<current_uploads> block) vs content[1:] (user blocks) to sanitize only
user blocks. Raw strings and non-standard dict blocks that
_extract_text_from_content misses are now also sanitized.
Non-distinguishable paths (< 2 text blocks, non-list content) still
degrade to full sanitization (safe — server block may be escaped but
user forgery never leaks). All fallback paths log via logger.warning.
Decision 18 / willem-bd 4th-round comment #3
Co-Authored-By: Claude <noreply@anthropic.com>
@
* @
fix: correct comments referencing text_blocks → content in rfind fallback
Co-Authored-By: Claude <noreply@anthropic.com>
@
* fix: 5th-round review — dead code, subagent gating, integration test, perf, consistency
- Delete unreachable ORIGINAL_USER_CONTENT_KEY guard in rfind fallback
branch (original_user_content guaranteed non-empty str at that point)
- Remove list_uploaded_files from BUILTIN_TOOLS; add include_upload_tool
param to get_available_tools(), default True; task_tool.py passes False
so subagents no longer receive a tool whose state exclusion is broken
- Add integration test exercising real create_agent graph (not mocked
runtime.state) to verify LangGraph propagates before_agent state writes
into ToolRuntime.state during same-turn tool calls
- Cache DirEntry.stat() st_size in candidates tuple to avoid second
per-file syscall in the rendering loop
- Make the upload-tag pre-check case-insensitive (content_str.lower())
to match _UPLOAD_BLOCK_RE re.IGNORECASE
PR #4174 — willem-bd 5th-round review items #1-#5
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(channels): pass files metadata through _human_input_message() for IM uploads
_human_input_message() was not passing additional_kwargs.files to the
downstream message. UploadsMiddleware read no files, wrote
uploaded_files=[], and list_uploaded_files reported same-run IM
attachments as historical files (fancyboi999 repro).
Fix: add files parameter to _human_input_message(), call site passes
files=uploaded. Regression test locks the contract.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(channels): remove legacy <uploaded_files> manual prepend to fix double-injection regression
Commit 8d86dbf6 added files= pass-through to UploadsMiddleware but
left the manual _format_uploaded_files_block() prepend in place.
Every IM attachment reached the model twice — once via the legacy
<uploaded_files> block and once via <current_uploads>.
This commit removes the manual prepend and the now-dead
_format_uploaded_files_block() function. UploadsMiddleware is the
sole upload-context producer for both IM and web paths.
Reported-by: fancyboi999 (PR review)
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: update #4212 issue body to reflect completed fixes and narrowed remaining scope
* chore: remove temporary scratch file
* fix(middleware): neutralize user-derived values inside <current_uploads> block
Upload-derived filenames, paths, outline titles, and preview text are
interpolated verbatim inside the trusted <current_uploads> wrapper,
which InputSanitizationMiddleware exempts from sanitization. A crafted
filename or document heading containing blocked authority tags would
bypass the guardrail and enter model context as trusted framework data.
Fix: call neutralize_untrusted_tags() on all four user-derived values
inside _format_file_entry(), preserving the outer <current_uploads>
wrapper untouched.
Reported-by: fancyboi999 (P1 security review)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(middleware): neutralize extension labels in omitted-file summary
Files exceeding the 10-item context cap bypass _format_file_entry().
Their extensions, derived from user-controlled filenames via
_extension_label(), were interpolated verbatim into the trusted
<current_uploads> wrapper — another path for blocked authority tags
to escape the guardrail.
Fix: neutralize extension values inside _extension_label(), the
single extraction point for all extension labels.
Reported-by: fancyboi999 (P1 security review)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(tools): neutralize user-derived values in list_uploaded_files tool result
Apply neutralize_untrusted_tags() to every model-visible user-derived value
returned by list_uploaded_files: filename, virtual path, extension, outline
titles, outline preview lines, and omitted-file extension summary.
This closes the last remaining injection bypass in the upload lazy-loading
path - the <current_uploads> block and its omitted summary were already
neutralized (previous commits), but the list_uploaded_files tool produced
a second exit for the same attacker-controlled metadata that
ToolResultSanitizationMiddleware did not cover.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(tests): add missing include_upload_tool=False to task_tool mock assertions
PR #4174 added include_upload_tool parameter to get_available_tools().
task_tool.py correctly passes include_upload_tool=False for subagents
but 5 existing tests' assert_called_once_with expectations were not
updated, causing CI failures.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(agents): per-agent model and generation settings
Let each custom agent choose its own model and sampling settings
(temperature, max_tokens) plus thinking / reasoning_effort defaults,
so agents sharing a model profile are no longer stuck with one shared
temperature and output length (#4336).
AgentConfig gains optional model_settings / thinking_enabled /
reasoning_effort (None = inherit). create_chat_model applies per-caller
model_overrides on top of the profile before the thinking/Codex
transforms; the lead agent resolves each knob with precedence
request > agent config > profile/default. The /api/agents create/update
routes persist the fields and reject an unknown model. The default lead
agent path is unchanged (no agent config -> overrides None). The agent
chat composer also stops force-overriding an agent's configured default
model with models[0].
* fix(agents): tri-state thinking control and default-model capability gating
The model-settings dialog seeded the thinking switch to false, so opening
it to tweak temperature and saving silently disabled thinking (the runtime
default is on) with no way back to inherit. It also hid the thinking /
reasoning controls whenever the agent inherited the global default model,
since `__default__` never resolved through `models.find`.
Give thinking an explicit Inherit / On / Off tri-state so an untouched save
is a no-op, and resolve `__default__` to the effective default (models[0])
for the capability check. Logic lives in the tested helpers module.
str_replace guards the replacement with `if old_str not in content`, which
cannot reject an empty old_str -- `"" in content` is always true. So an
empty old_str reached `str.replace("", new_str)`, which inserts new_str at
every character boundary, and the tool rewrote the file while still
returning "OK":
old_str='', new_str='# H\n' -> OK, file silently prepended
old_str='', new_str='X', replace_all -> OK, 'XdXeXfX XmXaXiXnX(X)X:X\nX...'
The empty-file branch above it already handles this case (`if not content:
if not old_str: return "OK"`), and the existing test states the intent
directly: "An empty old_str is a no-op edit and remains a benign OK". That
contract just never held once the file had content.
The tool is registered by default (config.example.yaml) and its schema
declares old_str as a plain string with no minLength, so a model can emit
"" legitimately; read-before-write only compares a hash and lets it past.
Check old_str first so the no-op holds whatever the file contains. The
empty-file case folds into the same not-found branch, which keeps its
message and behaviour.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
E2BSandbox.grep()'s glob handling reduced a directory-scoped pattern like
"src/*.js" down to just "*.js" before passing it to `grep --include=`,
dropping the directory-scoping prefix entirely. GNU grep's `--include`
matches by basename only, at any depth, so the search silently broadened
to every matching-extension file in the sandbox tree instead of just the
directory the caller asked for.
Keep the basename portion as a coarse `--include=` pre-filter (a superset
of the true match set) and post-filter grep's raw hits through
path_matches(), the same helper glob() already uses to enforce directory
scoping correctly, so grep and glob agree on what a directory-scoped
pattern means.
* feat(checkpoint): dual-mode checkpoint storage with LangGraph DeltaChannel
Add a restart-required database.checkpoint_channel_mode ("full" default,
"delta") that stores the messages channel via LangGraph 1.2 DeltaChannel,
cutting checkpoint storage from O(n^2) to O(n) for append-only history.
Existing full checkpoints seed delta state transparently; no data migration.
- config: mode schema + freeze-on-first-use with
CheckpointModeReconfigurationError; mode marker persisted in checkpoint
metadata; unsafe delta->full downgrade rejected fail-closed with
CheckpointModeMismatchError (run-level error, failed state read)
- state: delta message state schema; CheckpointStateAccessor centralizes
materialized reads for all consumers (threads API, branches,
regeneration, compaction, state updates, memory, goal workers)
- runtime: raw writers (run durations, interrupted title, thread goal)
parent their checkpoints to the checkpoint they derive from, preserving
delta ancestry; rollback forks the pre-run lineage through a state
mutation graph with Overwrite restores; InMemorySaver delta-history
override delegates to the base walk (fixes dropped first write after
migration, also present upstream)
- tests: conformance suite over {memory, sqlite, postgres} covering
migration replay, stable message IDs, storage shape and writer
preservation; conftest fixture isolates the frozen mode between tests;
stale config fakes refreshed
- ci: backend unit tests gain a postgres service
* fix(checkpoint): close materialization gaps in goal flow, guard public factory
- Route goal-continuation message reads through CheckpointStateAccessor:
raw channel_values reads see the delta sentinel in delta mode, which
disabled goal continuation (stand_down=no_durable_end_of_turn) after
durable assistant turns. Raw tuples remain for tuple-only metadata
(checkpoint id, pending_writes).
- Reject checkpoint_channel_mode='delta' + checkpointer in
create_deerflow_agent at construction: factory-built persisted graphs
bypass mode-marker injection and the fail-closed gate, reproducing
silent mixed-mode state loss. Delta without persistence stays allowed.
- Import the postgres saver lazily (pytest.importorskip in the fixture)
so the documented default install collects the suite; add a CI job
running pytest --collect-only on uv sync --group dev without extras.
- Fix test_checkpointer fallback test to patch get_app_config at its
use site (provider module), making it deterministic when a local
config.yaml selects a persistent backend.
* fix(gateway): preserve extension-owned channels in state mutations, bump config version
- build_state_mutation_graph / build_checkpoint_state_mutation_accessor
accept an explicit state_schema; branch and POST /state now compile the
mutation graph from the thread's effective schema
(graph_state_schema on the assistant graph). The base-ThreadState
fallback silently discarded channels contributed by custom
AgentMiddleware.state_schema on branch (data loss) and returned a
false-success 200 on POST /state.
- POST /state validates values keys against the mutation graph's
channels and rejects unknown fields with 422 instead of ignoring
them; reducer detection covers extension channels
(BinaryOperatorAggregate or DeltaChannel) so Overwrite replace
semantics work for middleware reducers in both modes.
- Endpoint regression: custom AgentMiddleware.state_schema value
survives branch, updates through POST /state, and an unknown field
receives 422.
- config_version 26 -> 27 for the new database.checkpoint_channel_mode
(example, Helm chart values + README, support-bundle fixture), so
existing installs get the outdated-config warning and
make config-upgrade merges the field; covered by a test driving the
real example file and the real config-upgrade script.
* fix(gateway): resolve assistant schema via one boundary, copy branch reducer values with Overwrite
GET /threads/{id}/state now resolves the thread's assistant_id through a
single reusable boundary (thread metadata -> assistant_id -> effective
graph), so channels contributed by a custom AgentMiddleware.state_schema
are materialized instead of dropped by the default lead schema. POST
/state uses the same boundary instead of resolving the schema ad hoc.
Branch writes wrap every copied reducer channel in Overwrite (derived
from the effective mutation graph: BinaryOperatorAggregate + DeltaChannel),
not just messages, so already-aggregated values are never re-merged.
Regression tests use a real AgentMiddleware.state_schema with a
non-identity reducer in both full and delta modes: GET /state returns the
extension value, POST /state replaces it, branch preserves it
byte-for-byte; the unknown-field 422 is a separate assertion.
* refactor(checkpoint): collapse read-path round-trips and ship dual-mode parity tests
Address review round 4 on PR #4292:
- Push ahistory/history limit through Pregel into checkpointer.alist
(SQL LIMIT) instead of materializing all rows and breaking in Python
- Fold the read-side mode-compat gate onto the returned snapshot's
metadata; only writes keep the pre-write tuple fetch (fail-closed)
- Cache factory-built accessor graphs per (assistant_id, mode) with
factory-identity revalidation; state reads no longer build a lead
agent per request
- get_thread: one snapshot fetch + one raw pending_writes fetch on the
resolved checkpoint (post-checkpoint __error__ writes never surface
in snapshot.tasks; verified empirically)
- DeerFlowClient.get_thread: single checkpointer.list walk collects
pending_writes per checkpoint instead of N get_tuple calls
- InMemorySaver delta-history patch: stand-down when the upstream
override disappears, try/except guard, validated-version warning,
guard tests
- make_lead_agent mode precedence: first freeze is owned by app_config
(client-supplied configurable key ignored); once frozen, injected
key/app_config must match or fail closed
- Rollback: lock in non-message channel restoration via fork
inheritance with a dedicated reducer-channel test
- Add tests/test_threads_checkpoint_mode.py and
tests/test_gateway_checkpoint_mode.py referenced by AGENTS.md and
the PR validation section: lifecycle parity (memory + sqlite),
per-step blob-count storage guard, gateway endpoint parity
Counted-saver tests pin checkpoint round-trips for aget/ahistory so
these regressions cannot silently return.
* fix(checkpoint): precise mode-mismatch HTTP mapping, gate E2E, and accessor resilience
- threads router: map CheckpointModeMismatchError to 409 (with cause and
thread id) and CheckpointModeReconfigurationError to 503 across all state
endpoints instead of swallowing both into a generic 500
- gate coverage: seed a real delta checkpoint into AsyncSqliteSaver and
assert aget/aupdate/ahistory fail closed; assert 409 at the HTTP boundary
through the real route stack
- rollback: compile the restore mutation graph with the thread's effective
state schema per the build_state_mutation_graph contract
- inheritance contract locks: rollback and manual compaction preserve
middleware-contributed channels via checkpoint fork cloning
- services: revalidate the accessor-graph cache against app_config identity
so config.yaml hot-reloads never serve a stale compiled graph
- services: degrade full-mode state reads to raw checkpointer reads when the
agent factory is unavailable (delta gate still applies; delta mode has no
fallback)
- deps: override websockets==16.0 (langgraph-sdk 0.4.2's <16 pin silently
downgraded 16.0 -> 15.0.1; pin is not grounded in any API incompatibility)
and bump the langchain lower bound to what the lockfile actually resolves
* fix(checkpoint): include anchor checkpoint in degraded history walk + cover get_thread
- _RawCheckpointReadAccessor.ahistory: alist(before=...) is exclusive while
pregel's get_state_history treats config.checkpoint_id as the inclusive
start; fetch the anchor explicitly so both read paths paginate identically
- extend the degraded-path gateway test: GET /thread returns raw values, and
POST /history with before starts at the anchor checkpoint
* fix(gateway): preserve degraded checkpoint timestamps
* fix(gateway): harden degraded checkpoint access
* fix(gateway): resolve assistants for checkpoint reads
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The /help string was hardcoded and had drifted from BUILTIN_COMMANDS,
omitting six real commands (help, switch, resume, uploads, artifacts,
details) and ordering the rest differently from the picker. Generate the
command line from the registry so /help can never fall out of sync again,
and add parity tests that guard against future drift.
`_count_diff_lines` skipped every line starting with "+++ "/"--- " to drop
difflib's two file headers. But a deleted hunk-body line whose content
begins with "-- " (e.g. a SQL comment `-- get users`) becomes the diff
line `--- get users`, and an added `++ ...` line becomes `+++ ...`, so
those were dropped too — undercounting the user-visible +N/-M change
summary (WorkspaceChangeSummary, per-file WorkspaceFileChange, and the
run-event string in recorder.py).
`difflib.unified_diff` always emits the two headers first, so skip them
by position (`lines[2:]`) instead. Hunk `@@` lines start with `@` and are
still ignored. Numeric/plain diffs are unaffected.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(browser): add agentic browser control
* fix(frontend): format browser view changes
* fix(browser): keep browser optional and isolate sidecar layout
* fix(browser): address PR review security and IME findings
- Nginx: add a browser-stream WebSocket location before the generic
/api/threads regex so Live upgrades instead of downgrading to HTTP
(both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
navigate, and tear down the browser session on thread deletion so a
later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
context-level route guard covering redirects, popups, iframes, and
subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
CJK candidate with Enter no longer submits the remote page form.
Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.
* fix(frontend): smooth streaming in long tool threads
* Revert "fix(frontend): smooth streaming in long tool threads"
This reverts commit f0462516eabe77f138d4027ea1c714fb226683cf.
* fix(browser): address review security and lifecycle findings
- Reject cross-origin WebSocket upgrades on the live browser stream
(Origin allow-list reuse of CORS/same-origin helpers) to close a
WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
(browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
can't permanently stall after the cumulative attempt cap.
* fix(frontend): reduce long tool thread render stalls
Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.
* fix(browser): keep live control responsive during continuous input
Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").
What:
- Frontend forwards one `click` per physical click instead of also emitting
`down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
start a rate-limited background refresh loop (leading frame + bounded cadence)
that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
inputs coalesce, and continuous input keeps refreshing before it stops.
Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.
* fix(browser): harden worker and session lifecycle
* fix(browser): address latest review feedback
* fix(frontend): preserve optimistic new-chat message
* test(e2e): preserve mocked message run ids
* fix(browser): address capability review feedback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(workspace-changes): classify a symlink replacing a file distinctly from deleted
scan_workspace_roots() skipped every symlinked path entirely
(host_file.is_symlink() -> continue), so the path was completely absent
from a snapshot instead of being recorded as a metadata-only stub the
way binary/large/sensitive-looking files already are. When an agent run
replaces a tracked file with a symlink (e.g. rm config.txt && ln -s
/some/other/path config.txt), the after-snapshot never contained that
path at all, so compare_snapshots()'s _status() saw after_file=None and
reported plain "deleted" -- silently hiding that the path is still alive
on disk, now as a symlink that can point anywhere on the host, including
outside the workspace root.
Add a "symlink" classification mirroring the existing binary/large/
sensitive pattern: scan_workspace_roots() now records a symlink as a
metadata-only FileSnapshot stub (symlink=True, symlink_target from
os.readlink(), lstat'd without ever following the link) instead of
omitting it. _status() reports "symlink_created" whenever a symlink
newly occupies a path that was not already a symlink (brand new or
replacing a prior file), so the security-relevant fact surfaces
distinctly instead of collapsing into "deleted". A symlink genuinely
removed with nothing replacing it is unchanged: still "deleted".
Verified against a real POSIX symlink (WSL; native Windows symlink
creation needs elevated privilege) driving the unmodified
scan_workspace_roots()/compare_snapshots() functions, and via a
patch-file revert/reapply cycle on this same fix to confirm the added
regression tests fail before and pass after.
* fix(workspace-changes): count symlink_created in the changed-file badge
getChangedFileCount summed only created + modified + deleted, so a run
whose only change was a symlink replacing a file (reported as the new
symlink_created status, not deleted) produced a count of 0 and
WorkspaceChangeBadge hid the badge entirely -- the exact scenario this
PR targets, with the opposite of the intended result.
Add symlink_created to the frontend WorkspaceChangeSummary interface
(already emitted by the backend) and include it in the count. The
per-file StatusIcon/statusLabel "modified" fallthrough is unaffected
and left as a follow-up, per review.
Regression test reverts cleanly to reproduce a count of 0 pre-fix.
* fix(workspace-changes): rank symlink_created in file sort and complete the type contract
sortWorkspaceChanges's statusRank had no entry for the new
symlink_created status, so statusRank[left.status] - statusRank[right.status]
evaluated to NaN for any comparison involving a symlink-created file,
violating Array#sort's ordering contract instead of producing a
deterministic order. The frontend WorkspaceChangeStatus and
DiffUnavailableReason unions, and the WorkspaceFileChange symlink
fields, also stayed narrower than what the backend now emits, so
TypeScript's satisfies Record<...> guard on statusRank could not catch
the gap.
Widen WorkspaceChangeStatus to include "symlink_created" and
DiffUnavailableReason to include "symlink", add the matching
symlink/symlink_target_before/symlink_target_after fields to
WorkspaceFileChange, and give symlink_created a rank alongside
modified in statusRank -- restoring the satisfies guard's ability to
catch a future unranked status. unavailableLabel now has an explicit
"symlink" branch (new symlinkUnavailable i18n string, en-US + zh-CN)
instead of falling through to the generic label.
Also fixes the failing e2e-tests CI check: the existing
workspace-changes.spec.ts mock summary predates the symlink_created
field, so getChangedFileCount computed 1 + 1 + 0 + undefined = NaN and
the badge rendered "Edited NaN files" instead of "Edited 2 files".
Added symlink_created: 0 to the mock to match the real backend
contract.
New sortWorkspaceChanges unit tests revert cleanly against the
unranked statusRank to reproduce the NaN-driven misordering. pnpm test
(626 tests), pnpm check, and pnpm format are clean, and the
previously-failing e2e spec plus the full e2e suite (94 tests) pass.
